WikifitaGitHub live67e8de5
outro · sdk-test/sdk-test-tools

SDK-Test Batch Decompilation Tools

Automated reverse engineering pipeline for Antigravity Go binaries — from Go pclntab extraction through Ghidra MCP batch decompilation to organized C output.

Baixar raw

SDK-Test Batch Decompilation Tools

The sdk-test project includes a three-script pipeline for automated batch decompilation of Antigravity's Go binaries. These tools perform large-scale reverse engineering: extracting function symbols from Go's pclntab (program counter line number table), filtering out boilerplate, creating and renaming functions in Ghidra, batch-decompiling via Ghidra's MCP REST API, and saving each decompiled function as an individual .c file.

The pipeline is designed for Apple Silicon (M3 Pro, 24GB Unified VRAM), is fully resume-safe, and follows the project's Rule 01 (Astral UV & PEP 723) for execution via uv run.

The Three Scripts

ScriptRoleInvocation
decompile_all.pyFully autonomous end-to-end pipelineuv run tools/decompile_all.py
batch_decompile_all.pyBatch preparation and filteringuv run tools/batch_decompile_all.py --binary localharness
run_batch_decompile.pyBatch execution via Ghidra REST APIuv run tools/run_batch_decompile.py --binary localharness

Pipeline Relationship

decompile_all.py (master pipeline)
    |
    |-- [1] Extract functions from Go pclntab
    |-- [2] Filter boilerplate, keep interesting packages
    |-- [3] Create function entries in Ghidra
    |-- [4] Rename FUN_xxx to real Go names
    |-- [5] Batch-decompile via Ghidra MCP
    |-- [6] Save .c files, save Ghidra project
    |
    v
batch_decompile_all.py (standalone batch prep)
    |
    |-- Reads pclntab, filters, creates batch JSON files
    |-- Output: batches/batch_0000.json ... batch_NNNN.json
    |
    v
run_batch_decompile.py (standalone batch executor)
    |
    |-- Reads pending batch files
    |-- Calls Ghidra MCP batch_decompile
    |-- Saves .c files, updates batch status
    |-- Resume-safe: skips done batches on re-run

decompile_all.py is the "walk away" script — it does everything in one pass. The other two scripts exist for manual control: batch_decompile_all.py prepares batches without executing them, and run_batch_decompile.py processes batches that already exist.

Target Binaries

The pipeline targets four Antigravity Go binaries:

BinaryPathDescription
localharness~/probe/sdk-test/.venv/.../google/antigravity/bin/localharnessProduction harness (local agent execution)
agy~/.local/bin/agyTerminal TUI interface
language_server_ide/Applications/Antigravity IDE.app/.../language_server_macos_armIDE language server
language_server_hub/Applications/Antigravity.app/.../language_serverStandalone app language server

Each binary requires specific configuration parameters:

Parameterlocalharnessagylanguage_server_ide
pclntab_offset0x2052f000x2fc48800x2b00d60
text_start0x100000db00x100000eb00x100000eb0
Ghidra programlocalharnessagylanguage_server_macos_arm

These offsets are binary-specific and must be updated when binaries are upgraded (e.g., agy offset changed from 0x2f2a720 in v1.0.8 to 0x2fc4880 in v1.0.9).

Stage 1: Go pclntab Extraction

Go binaries contain a Program Counter Line Number Table (pclntab) that stores function metadata: names, virtual addresses, and sizes. This is the most reliable source of function information — far more complete than Ghidra's auto-analysis.

How it works:

  1. Read the entire binary into memory
  2. Seek to the pclntab_offset (binary-specific)
  3. Parse the pclntab header:
    • Offset 8: nfunc (number of functions)
    • Offset 32: func_name_off (offset to function name table)
    • Offset 64: pcln_off (offset to function table)
  4. For each function entry, extract:
    • name — Short name (last path component)
    • full_name — Full Go package path (e.g., jetski/cortex/handler.HandleMessage)
    • va — Virtual address (text_start + entry_offset)
    • size — Function size in bytes (difference from next function's offset)

The extract_functions() function in both batch_decompile_all.py and decompile_all.py implements this parsing.

Stage 2: Filtering

Raw pclntab extraction produces thousands of functions, most of which are Go runtime boilerplate, protobuf marshaling code, or standard library functions. The filtering stage keeps only interesting, proprietary code.

Filter Criteria

Minimum size: 20 bytes (skip stubs and trampolines)

Path prefix filter: Only functions whose full_name contains specific proprietary paths:

Path PrefixDescription
jetski/Core: cortex, language_server, api_server, eval
jetski_prod/Production harness
gemini_coder/Framework, cider, proto
google/internal/cloud/codeInternal cloud code APIs

Package filter (batch_decompile_all.py): An allowlist of ~40 interesting packages:

CategoryPackages
Corelocalagent, backend, genai, genairequest
Authauth, authclient, multicall
gRPClanguage_server, language_server_go_grpc, language_server_go_proto
Protov1internal, prediction_service, extension_server
Agent Controlagentapi
Telemetrytracing_go_proto, tracing
Statestatesync, index, store
Configconfig, configpb
Modelmodelapi, inference
Browserbrowser, playwright
MCPmcp

Boilerplate skip list (47 patterns):

Functions matching these name fragments are excluded:

  • Proto/serialization: ProtoMessage, ProtoReflect, Marshal, Unmarshal, Size, Descriptor
  • Go runtime: Reset, String, Error, Close, Len, Less, Swap
  • gRPC stubs: Unimplemented, setRequestMethod, Trailer, Header
  • Compiler artifacts: XXX_, rawDescGZIP, _proto_init, -fm
  • Type system: type:., go:info., go:.

Getter filter: Simple Get* functions under 100 bytes are excluded (likely protobuf field accessors).

The result is a curated set of functions — sorted by size descending — representing the proprietary logic of the Antigravity harness.

Stage 3: Ghidra Function Creation

Go binaries have a pclntab with ALL function addresses, but Ghidra's auto-analysis often misses many of them. The create_functions_in_ghidra() function ensures every function we want to decompile exists as a Ghidra function entry.

For each filtered function, the script calls:

ghidra_post(client, "/create_function", {
    "address": va,
    "program": program,
})

Progress is reported every 200 functions with created/already-exists/error counts.

Stage 4: Ghidra Renaming

Ghidra initially names functions as FUN_<address> (e.g., FUN_1004a3b20). The rename_functions_in_ghidra() function replaces these with the real Go names from pclntab.

For each function:

  1. Sanitize the Go name for Ghidra (replace (, ), *, [, ] with safe characters)
  2. Call rename_or_label on the Ghidra MCP
  3. If that fails, fall back to create_label

Progress is reported every 100 functions.

Stage 5: Batch Decompilation

The core operation: sending groups of function addresses to Ghidra's batch_decompile endpoint and receiving decompiled C code.

How batch_decompile Works

The Ghidra MCP bridge exposes a batch_decompile REST endpoint:

GET /batch_decompile?functions=0x1004a3b20,0x1004a3c80,...&program=localharness

The response is a JSON object mapping addresses to decompiled C code:

{
  "0x1004a3b20": "void FUN_1004a3b20(int param1) {\n  ...\n}",
  "0x1004a3c80": "..."
}

Batch Sizing

  • decompile_all.py: 75 functions per batch (tuned for M3 Pro)
  • batch_decompile_all.py: 15 functions per default (configurable via --batch-size)
  • Delay between batches: 0.05s (decompile_all.py) or 0.3s (run_batch_decompile.py)

Output Format

Each decompiled function is saved as an individual .c file:

// Function: handler.HandleMessage
// Full name: jetski/cortex/handler.HandleMessage
// Address: 0x1004a3b20
// Size: 384 bytes

void handler.HandleMessage(int64_t param1, ...) {
    // decompiled code
}

Filenames are sanitized from Go function names: handler.HandleMessage.c, api_server.NewServer.c, etc. Maximum filename length is 80 characters.

Error Handling

  • Failed decompilations (empty output, "Error: Function not found", code under 30 bytes) are counted as failures but do not stop processing
  • Consecutive error threshold: 5 errors triggers abort (prevents infinite retry loops, consistent with Rule 03)
  • On error, the script backs off 2 seconds before retrying the next batch

Resume Safety

Both scripts are fully resume-safe:

  • Already-decompiled functions are detected by checking for existing .c files in the output directory
  • Batch files in batches/ track their own status (pending / done / error: ...)
  • Re-running skips completed work automatically

The Complete Pipeline (decompile_all.py)

decompile_all.py is the autonomous "walk away" script. For each binary, it executes six steps:

[1/6] Checking Ghidra status...
[2/6] Extracting Go symbols from pclntab...
[3/6] Filtering to interesting packages...
[4/6] Creating function entries in Ghidra...
[5/6] Renaming functions in Ghidra...
[6/6] Decompiling and saving...
      Saving Ghidra project...

Dry run mode: uv run tools/decompile_all.py --dry-run validates pclntab extraction, cleans old decompiled files, and rebuilds function indices without requiring Ghidra.

The Standalone Batch Pipeline

When finer control is needed, the two-script approach works as follows:

Step 1: Prepare Batches

uv run tools/batch_decompile_all.py --binary localharness --batch-size 15

This creates batch JSON files in reverse_engineering/<binary>/batches/:

{
  "batch_index": 0,
  "program": "localharness",
  "functions": [...],
  "addresses_csv": "0x1004a3b20,0x1004a3c80,...",
  "status": "pending"
}

It also saves a function_index.json with the complete filtered function list.

Step 2: Execute Batches

uv run tools/run_batch_decompile.py --binary localharness

This reads pending batch files, calls Ghidra MCP, saves .c files, and updates batch status to done.

Options:

  • --start 50 — Skip to batch 50
  • --limit 10 — Process only 10 batches
  • --port 8089 — Ghidra MCP port (default: 8089)
  • --delay 0.3 — Seconds between batches

Progress output:

[1/200] Batch 0042 (15 funcs) | 3150 fn/s | ETA 2min

Prerequisites

  • Ghidra running with the MCP bridge on port 8089
  • Python via uv (PEP 723, no separate venv needed)
  • Binary files accessible at their configured paths
  • httpx library (auto-installed by uv run)

Output Structure

reverse_engineering/
├── localharness/
│   ├── function_index.json        # Complete filtered function list
│   ├── batches/                   # Batch preparation files
│   │   ├── batch_0000.json        # Status: done
│   │   ├── batch_0001.json        # Status: pending
│   │   └── ...
│   └── decompiled/                # Individual .c files
│       ├── handler_HandleMessage.c
│       ├── api_server_NewServer.c
│       └── ... (hundreds of files)
├── agy/
│   ├── function_index.json
│   ├── batches/
│   └── decompiled/
└── language_server_macos_arm/
    ├── function_index.json
    ├── batches/
    └── decompiled/

Why This Matters

This pipeline represents the automation of reverse engineering at scale. Instead of manually decompiling functions one by one in Ghidra's GUI, the scripts:

  1. Extract all functions from Go's own metadata (more complete than Ghidra's analysis)
  2. Filter intelligently to keep only proprietary code (excluding ~47 categories of boilerplate)
  3. Rename everything so decompiled output uses real Go names instead of FUN_xxx
  4. Batch process through Ghidra's REST API at machine speed
  5. Resume automatically if interrupted

The result is a complete, browsable library of decompiled Antigravity source code — organized by binary, indexed by function, and ready for analysis. This is the foundation for understanding how the Antigravity harness works internally.

Cross-References