Loading ToolBench
MCP server that exposes IDA Pro binary analysis capabilities over TCP, enabling remote analysis of binaries through a plugin bridge
IDA Codex MCP exhibits multiple critical definition gaps. While all 26 tools have names and brief descriptions, most parameter descriptions are minimal or missing entirely. Input schemas are present but sparse, many tools have empty input objects or lack type definitions for parameters. Output schemas are completely undocumented. The tool descriptions vary widely in quality: some (e.g., 'get_instance_info', 'get_pseudocode') are adequately detailed (50-80 chars), but others are vague (e.g., 'py_exec' offers no guidance on when to use it vs. direct IDA scripting, nor does it warn about the DESTRUCTIVE risk clearly in the description). Error handling guidance is absent, no tool description explains what to do on failure, what errors are retryable, or what validation is performed. The server mixes verb-noun naming (good: get_*, list_*, set_*) with overly generic helpers (py_exec). No tool documents output structure, response pagination, or how to chain results between tools. Security concerns: py_exec is a dangerous capability but descriptions lack prominent warnings about code injection risks.
Builds a call graph starting from a root function or all functions up to a specified depth
Builds a map of which functions call each other, showing all callers for each function
Discovers running IDA instances by probing a port range on a host
Finds the address of a function by its name
Retrieves raw bytes from a memory range
Retrieves the disassembly for a single instruction at an address
Returns disassembly (assembly instructions) for a function
Retrieves comments associated with a function
Missing output schemas on ALL tools. No tool documents what fields are returned, data types, or structure. LLMs cannot plan downstream calls or extract required context.
py_exec tool (DESTRUCTIVE capability) lacks security warnings in description. No mention of code injection risks, sandboxing, or permission requirements. Description is 87 chars but contains no actionable safety guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 36 | <=2025-11-25 | v2 |
Retrieves the signature/prototype of a function
Returns current IDA instance information including database path, input file, image base, bit width, and function count for instance discovery and identification
Retrieves the symbolic name at an address
Returns decompiled pseudocode for a function using Hex-Rays if available
Retrieves comments from decompiled pseudocode using Hex-Rays
Retrieves a string constant from the binary
Retrieves cross-references (xrefs) to or from an address
Uses heuristics to guess the role of a function (init, hotspot, normal) based on its name and callers
Lists all functions in the current binary with their addresses and names
Lists all available IDA targets (instances) configured or discovered
Execute Python code in IDA's Python interpreter with access to IDA modules and return result, output, and errors
Renames a function in the IDA database
Sets the active IDA target for subsequent tool calls
Writes raw bytes to a memory range
Sets or updates a comment on a function
Sets the signature/prototype for a function
Sets a symbolic name at an address
Sets a comment on decompiled pseudocode at a specific line using Hex-Rays
No error handling guidance in any tool description. Tools do not explain what errors are retryable, what to do on resource not found, or how to handle timeouts. Descriptions lack recovery context.
Parameter descriptions are minimal or missing context. E.g., 'ea' parameter appears in 15+ tools but has no explanation of what 'ea' (effective address) means or valid range. LLMs cannot infer IDA-specific terminology.
list_functions, list_targets, build_callers_map return unbounded lists with no pagination support. No mention of result limits, offset/limit parameters, or total count. Large binaries could return thousands of functions, exhausting context window.
No tool documents prerequisites or dependencies. E.g., build_call_graph requires a valid function address, but description does not mention calling find_function_ea first. Agents may call tools in wrong order.
Input schemas for most tools are either empty {} or contain parameters with no type field. E.g., build_call_graph has max_depth and root_name with type descriptions but no JSON Schema 'type' key. Schema score capped at 40 or below.
Destructive tools (set_function_cmt, rename_function, set_bytes, set_name, set_function_signature, set_pseudocode_cmt) lack confirmation/dry-run patterns. No mention of rollback, undo, or consequences in descriptions. High risk of accidental overwrites.
Parameter naming inconsistencies. Some tools use 'ea' (effective address), others use 'address'. No standardization across tools. LLMs may confuse which parameter to use for similar operations.
No documentation of which tools are idempotent vs. have side effects. set_name, set_function_cmt, rename_function do not state whether repeated calls are safe. Agents cannot safely retry on ambiguous failures.