---
name: sdk-test-tools
type: reference
title: "SDK-Test Batch Decompilation Tools"
description: "Automated reverse engineering pipeline for Antigravity Go binaries — from Go pclntab extraction through Ghidra MCP batch decompilation to organized C output."
tags: [decompilation, ghidra, reverse-engineering, go, binaries, batch-processing, sdk-test]
timestamp: 2026-07-21
---

# SDK-Test Batch Decompilation Tools

The sdk-test project includes a three-script pipeline for automated batch decompilation of Antigravity's Go binaries. These tools perform large-scale reverse engineering: extracting function symbols from Go's pclntab (program counter line number table), filtering out boilerplate, creating and renaming functions in Ghidra, batch-decompiling via Ghidra's MCP REST API, and saving each decompiled function as an individual `.c` file.

The pipeline is designed for Apple Silicon (M3 Pro, 24GB Unified VRAM), is fully resume-safe, and follows the project's [[sdk-test-agent-rules|Rule 01 (Astral UV & PEP 723)]] for execution via `uv run`.

---

## The Three Scripts

| Script | Role | Invocation |
|--------|------|------------|
| `decompile_all.py` | Fully autonomous end-to-end pipeline | `uv run tools/decompile_all.py` |
| `batch_decompile_all.py` | Batch preparation and filtering | `uv run tools/batch_decompile_all.py --binary localharness` |
| `run_batch_decompile.py` | Batch execution via Ghidra REST API | `uv run tools/run_batch_decompile.py --binary localharness` |

### Pipeline Relationship

```
decompile_all.py (master pipeline)
    |
    |-- [1] Extract functions from Go pclntab
    |-- [2] Filter boilerplate, keep interesting packages
    |-- [3] Create function entries in Ghidra
    |-- [4] Rename FUN_xxx to real Go names
    |-- [5] Batch-decompile via Ghidra MCP
    |-- [6] Save .c files, save Ghidra project
    |
    v
batch_decompile_all.py (standalone batch prep)
    |
    |-- Reads pclntab, filters, creates batch JSON files
    |-- Output: batches/batch_0000.json ... batch_NNNN.json
    |
    v
run_batch_decompile.py (standalone batch executor)
    |
    |-- Reads pending batch files
    |-- Calls Ghidra MCP batch_decompile
    |-- Saves .c files, updates batch status
    |-- Resume-safe: skips done batches on re-run
```

`decompile_all.py` is the "walk away" script — it does everything in one pass. The other two scripts exist for manual control: `batch_decompile_all.py` prepares batches without executing them, and `run_batch_decompile.py` processes batches that already exist.

---

## Target Binaries

The pipeline targets four Antigravity Go binaries:

| Binary | Path | Description |
|--------|------|-------------|
| `localharness` | `~/probe/sdk-test/.venv/.../google/antigravity/bin/localharness` | Production harness (local agent execution) |
| `agy` | `~/.local/bin/agy` | Terminal TUI interface |
| `language_server_ide` | `/Applications/Antigravity IDE.app/.../language_server_macos_arm` | IDE language server |
| `language_server_hub` | `/Applications/Antigravity.app/.../language_server` | Standalone app language server |

Each binary requires specific configuration parameters:

| Parameter | `localharness` | `agy` | `language_server_ide` |
|-----------|----------------|-------|----------------------|
| pclntab_offset | `0x2052f00` | `0x2fc4880` | `0x2b00d60` |
| text_start | `0x100000db0` | `0x100000eb0` | `0x100000eb0` |
| Ghidra program | `localharness` | `agy` | `language_server_macos_arm` |

These offsets are binary-specific and must be updated when binaries are upgraded (e.g., `agy` offset changed from `0x2f2a720` in v1.0.8 to `0x2fc4880` in v1.0.9).

---

## Stage 1: Go pclntab Extraction

Go binaries contain a Program Counter Line Number Table (`pclntab`) that stores function metadata: names, virtual addresses, and sizes. This is the most reliable source of function information — far more complete than Ghidra's auto-analysis.

**How it works:**

1. Read the entire binary into memory
2. Seek to the `pclntab_offset` (binary-specific)
3. Parse the pclntab header:
   - Offset 8: `nfunc` (number of functions)
   - Offset 32: `func_name_off` (offset to function name table)
   - Offset 64: `pcln_off` (offset to function table)
4. For each function entry, extract:
   - `name` — Short name (last path component)
   - `full_name` — Full Go package path (e.g., `jetski/cortex/handler.HandleMessage`)
   - `va` — Virtual address (`text_start + entry_offset`)
   - `size` — Function size in bytes (difference from next function's offset)

The `extract_functions()` function in both `batch_decompile_all.py` and `decompile_all.py` implements this parsing.

---

## Stage 2: Filtering

Raw pclntab extraction produces thousands of functions, most of which are Go runtime boilerplate, protobuf marshaling code, or standard library functions. The filtering stage keeps only interesting, proprietary code.

### Filter Criteria

**Minimum size:** 20 bytes (skip stubs and trampolines)

**Path prefix filter:** Only functions whose `full_name` contains specific proprietary paths:

| Path Prefix | Description |
|-------------|-------------|
| `jetski/` | Core: cortex, language_server, api_server, eval |
| `jetski_prod/` | Production harness |
| `gemini_coder/` | Framework, cider, proto |
| `google/internal/cloud/code` | Internal cloud code APIs |

**Package filter (batch_decompile_all.py):** An allowlist of ~40 interesting packages:

| Category | Packages |
|----------|----------|
| Core | `localagent`, `backend`, `genai`, `genairequest` |
| Auth | `auth`, `authclient`, `multicall` |
| gRPC | `language_server`, `language_server_go_grpc`, `language_server_go_proto` |
| Proto | `v1internal`, `prediction_service`, `extension_server` |
| Agent Control | `agentapi` |
| Telemetry | `tracing_go_proto`, `tracing` |
| State | `statesync`, `index`, `store` |
| Config | `config`, `configpb` |
| Model | `modelapi`, `inference` |
| Browser | `browser`, `playwright` |
| MCP | `mcp` |

**Boilerplate skip list (47 patterns):**

Functions matching these name fragments are excluded:
- Proto/serialization: `ProtoMessage`, `ProtoReflect`, `Marshal`, `Unmarshal`, `Size`, `Descriptor`
- Go runtime: `Reset`, `String`, `Error`, `Close`, `Len`, `Less`, `Swap`
- gRPC stubs: `Unimplemented`, `setRequestMethod`, `Trailer`, `Header`
- Compiler artifacts: `XXX_`, `rawDescGZIP`, `_proto_init`, `-fm`
- Type system: `type:.`, `go:info.`, `go:.`

**Getter filter:** Simple `Get*` functions under 100 bytes are excluded (likely protobuf field accessors).

The result is a curated set of functions — sorted by size descending — representing the proprietary logic of the Antigravity harness.

---

## Stage 3: Ghidra Function Creation

Go binaries have a pclntab with ALL function addresses, but Ghidra's auto-analysis often misses many of them. The `create_functions_in_ghidra()` function ensures every function we want to decompile exists as a Ghidra function entry.

For each filtered function, the script calls:
```python
ghidra_post(client, "/create_function", {
    "address": va,
    "program": program,
})
```

Progress is reported every 200 functions with created/already-exists/error counts.

---

## Stage 4: Ghidra Renaming

Ghidra initially names functions as `FUN_<address>` (e.g., `FUN_1004a3b20`). The `rename_functions_in_ghidra()` function replaces these with the real Go names from pclntab.

For each function:
1. Sanitize the Go name for Ghidra (replace `(`, `)`, `*`, `[`, `]` with safe characters)
2. Call `rename_or_label` on the Ghidra MCP
3. If that fails, fall back to `create_label`

Progress is reported every 100 functions.

---

## Stage 5: Batch Decompilation

The core operation: sending groups of function addresses to Ghidra's `batch_decompile` endpoint and receiving decompiled C code.

### How batch_decompile Works

The Ghidra MCP bridge exposes a `batch_decompile` REST endpoint:
```
GET /batch_decompile?functions=0x1004a3b20,0x1004a3c80,...&program=localharness
```

The response is a JSON object mapping addresses to decompiled C code:
```json
{
  "0x1004a3b20": "void FUN_1004a3b20(int param1) {\n  ...\n}",
  "0x1004a3c80": "..."
}
```

### Batch Sizing

- **`decompile_all.py`:** 75 functions per batch (tuned for M3 Pro)
- **`batch_decompile_all.py`:** 15 functions per default (configurable via `--batch-size`)
- **Delay between batches:** 0.05s (`decompile_all.py`) or 0.3s (`run_batch_decompile.py`)

### Output Format

Each decompiled function is saved as an individual `.c` file:

```c
// Function: handler.HandleMessage
// Full name: jetski/cortex/handler.HandleMessage
// Address: 0x1004a3b20
// Size: 384 bytes

void handler.HandleMessage(int64_t param1, ...) {
    // decompiled code
}
```

Filenames are sanitized from Go function names: `handler.HandleMessage.c`, `api_server.NewServer.c`, etc. Maximum filename length is 80 characters.

### Error Handling

- Failed decompilations (empty output, "Error: Function not found", code under 30 bytes) are counted as failures but do not stop processing
- Consecutive error threshold: 5 errors triggers abort (prevents infinite retry loops, consistent with [[sdk-test-agent-rules|Rule 03]])
- On error, the script backs off 2 seconds before retrying the next batch

### Resume Safety

Both scripts are fully resume-safe:
- Already-decompiled functions are detected by checking for existing `.c` files in the output directory
- Batch files in `batches/` track their own status (`pending` / `done` / `error: ...`)
- Re-running skips completed work automatically

---

## The Complete Pipeline (decompile_all.py)

`decompile_all.py` is the autonomous "walk away" script. For each binary, it executes six steps:

```
[1/6] Checking Ghidra status...
[2/6] Extracting Go symbols from pclntab...
[3/6] Filtering to interesting packages...
[4/6] Creating function entries in Ghidra...
[5/6] Renaming functions in Ghidra...
[6/6] Decompiling and saving...
      Saving Ghidra project...
```

**Dry run mode:** `uv run tools/decompile_all.py --dry-run` validates pclntab extraction, cleans old decompiled files, and rebuilds function indices without requiring Ghidra.

---

## The Standalone Batch Pipeline

When finer control is needed, the two-script approach works as follows:

### Step 1: Prepare Batches

```bash
uv run tools/batch_decompile_all.py --binary localharness --batch-size 15
```

This creates batch JSON files in `reverse_engineering/<binary>/batches/`:
```json
{
  "batch_index": 0,
  "program": "localharness",
  "functions": [...],
  "addresses_csv": "0x1004a3b20,0x1004a3c80,...",
  "status": "pending"
}
```

It also saves a `function_index.json` with the complete filtered function list.

### Step 2: Execute Batches

```bash
uv run tools/run_batch_decompile.py --binary localharness
```

This reads pending batch files, calls Ghidra MCP, saves `.c` files, and updates batch status to `done`.

**Options:**
- `--start 50` — Skip to batch 50
- `--limit 10` — Process only 10 batches
- `--port 8089` — Ghidra MCP port (default: 8089)
- `--delay 0.3` — Seconds between batches

Progress output:
```
[1/200] Batch 0042 (15 funcs) | 3150 fn/s | ETA 2min
```

---

## Prerequisites

- **Ghidra** running with the MCP bridge on port 8089
- **Python** via `uv` (PEP 723, no separate venv needed)
- **Binary files** accessible at their configured paths
- **httpx** library (auto-installed by `uv run`)

---

## Output Structure

```
reverse_engineering/
├── localharness/
│   ├── function_index.json        # Complete filtered function list
│   ├── batches/                   # Batch preparation files
│   │   ├── batch_0000.json        # Status: done
│   │   ├── batch_0001.json        # Status: pending
│   │   └── ...
│   └── decompiled/                # Individual .c files
│       ├── handler_HandleMessage.c
│       ├── api_server_NewServer.c
│       └── ... (hundreds of files)
├── agy/
│   ├── function_index.json
│   ├── batches/
│   └── decompiled/
└── language_server_macos_arm/
    ├── function_index.json
    ├── batches/
    └── decompiled/
```

---

## Why This Matters

This pipeline represents the automation of reverse engineering at scale. Instead of manually decompiling functions one by one in Ghidra's GUI, the scripts:

1. **Extract all functions** from Go's own metadata (more complete than Ghidra's analysis)
2. **Filter intelligently** to keep only proprietary code (excluding ~47 categories of boilerplate)
3. **Rename everything** so decompiled output uses real Go names instead of `FUN_xxx`
4. **Batch process** through Ghidra's REST API at machine speed
5. **Resume automatically** if interrupted

The result is a complete, browsable library of decompiled Antigravity source code — organized by binary, indexed by function, and ready for analysis. This is the foundation for understanding how the Antigravity harness works internally.

---

## Cross-References

- [[sdk-test-agent-rules]] — Rule 01 mandates `uv run` for all Python execution
- [[sdk-test-preprocessor]] — The preprocessor that can enrich prompts about decompilation results
- [[unit-distance-anti-contamination]] — Safety constraints that apply to reverse engineering workflows
- antigravity-2.0 — The harness being reverse-engineered