Loading ToolBench
PGMCP has 16 tools with adequate naming (all verb-prefix style) and visible descriptions. However, tool definitions suffer from critical gaps in schema completeness, output documentation, and error handling guidance. Most tools lack proper parameter descriptions, and no tools document their return schemas. The codebase uses FastMCP but tool registration appears to lack structured output documentation. Descriptions range from adequate to minimal (e.g., 'Get binary metadata...' is 51 chars, acceptable; but 'Get symbol table information...' is only 48 chars). No tool explicitly declares what it returns beyond type inference from code. Parameter descriptions exist for pagination tools but are generic ('Page number (default 1)'). Most critically: (1) no output schemas are documented for any tool, (2) error handling is absent, tools fail silently or return bare exceptions, (3) no tool provides recovery guidance when operations fail, (4) no parameter validation messages are shown in definitions.
Analyze a function's pseudo code using AI model to generate insights and explanations
Compare pseudo code between two functions to find similarities
Get the analysis session log for debugging and monitoring
Get binary metadata including functions, memory blocks, architecture, and compiler information
Get the assembly code for a specific function
Get detailed information about a specific function including pseudo code, assembly, signature, parameters, and return type
Get SHA256 hash of a function's pseudo code for similarity tracking
Get the decompiled pseudo code for a specific function
No output schemas documented for any of the 16 tools. LLM cannot infer what fields to expect, blocking downstream tool composition and field extraction.
No error handling guidance. When a function is not found, tool fails with no recovery hint (e.g., 'Try search_functions() with a partial name'). Agents cannot self-correct.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
Get all global symbols from the binary with pagination support
Get all memory blocks information including name, start address, end address, and size
Get symbol table information including symbol names, addresses, and types with pagination
List all functions in the binary with pagination support
Force reanalysis of the binary by clearing the cache and re-decompiling all functions
Search for functions by name pattern with optional case sensitivity and pagination
Search for patterns in pseudo code across all functions using regex
Search for symbols by name pattern with pagination support
Parameter descriptions are generic and do not state constraints. 'Number of items per page (default 10, max 100)' lacks format specification for page_size (min/max bounds not explicit in schema). Repeated in 8 tools.
analyze_function_with_ai tool calls external AI API (gpt-oss:20b, line ~30) but no timeout, rate limit, or fallback is documented. If API hangs, agent blocks indefinitely.
reinitialize_analysis performs irreversible cache clearing and full re-decompilation but has NO confirmation step. An agent error could force expensive reanalysis.
Tool composition broken: get_function_details returns pseudo_code, asm_code, signature, params, return_type (inferred from code) but NO output schema makes it impossible for LLM to chain with other tools or extract fields reliably.
No description for search_functions 'case_sensitive' parameter beyond boolean default. LLM doesn't know: does it apply to regex matching, literal search, or both?
Global state mutation: binary_path is hardcoded (/Users/arkea/Study/..., line ~23) and decomp_data is global module state. Not compatible with concurrent agent requests or multi-user deployments.