Headless IDA Pro MCP Server for binary analysis and reverse engineering
Scoring was not performed
Output schemas not documented in source code. No tool descriptions specify what fields LLM should expect from responses (e.g., disasm returns what structure? Does addr_info include xrefs?). Agents cannot plan downstream tool calls or extract relevant data without documented output schemas.
Complex tools like open_idb (8 parameters) and disasm variants lack per-parameter descriptions. Parameters like 'extra_args' (array) and 'file_type' lack guidance on valid values, format, or when they are required vs optional. LLMs cannot infer constraints from names alone.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 36 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 35 | - | v1 |
Decompile tool (file: src/bin/gen_tools_doc.rs) lacks explicit description entirely, only inference from binary name. This tool scores 50 overall.
Disassembly tools (disasm, disasm_by_name, disasm_function, pseudocode_at) have ambiguous naming and overlapping scope. LLMs may conflate 'disasm vs disasm_by_name vs disasm_function' or 'pseudocode_at' without clear distinction when to use each. Tool names should make functional differences obvious (e.g., disasm_range vs disasm_by_function_name).
Address parameters in tools (open_idb, disasm, function_at, addr_info, set_comments, etc.) accept 'hex or decimal' but lack validation guidance in descriptions. LLMs may pass invalid formats; error messages should guide recovery (pattern: recovery-guide).
WRITE-risk tools (open_idb, close_idb, set_comments, rename, analyze_funcs) lack confirmation or dry-run guidance. Agents may destroy analysis or overwrite symbols without safeguards. No documented recovery paths if a rename or comment fails mid-operation.
lookup_funcs accepts 'array of addresses or function names' but provides no guidance on format, separators, or mixed-type handling. Description is 57 chars, below actionable threshold (pattern baseline: 194 chars for A+ tools). LLM may guess wrong on mixed input like ["0x1000", "main"].
No pagination or result-limit guidance documented. Tools like list_functions accept offset/limit but no mention of max results, total count in response, or guidance on result volume caps. LLM may request 1000 functions, exhausting context window.