Loading ToolBench
OpenCode-first graph memory and local context runtime for coding agents with MCP server implementation
CTX exposes 31 tools via HTTP MCP transport with significant definition quality gaps. While naming is generally verb-based and descriptions exist, the schema definitions are severely incomplete. Most tools lack explicit input parameter type definitions in the visible source code (crates/ctx-cli/src/main.rs shows tool names and brief descriptions but no formal JSON Schema definitions). Only 9 of 31 tools have any input schema visible in the provided source; the remaining 22 tools have no schema documentation. Output schemas are not documented for any tool. Parameter descriptions are minimal (typically 1-2 lines). Error handling and recovery guidance are absent. Several tools (menu, doctor, mcp_stdio) lack even input parameters in the visible registration. The codebase is well-structured (13 specialized crates) but the MCP integration in crates/ctx-mcp does not expose complete schema metadata in the way production tools require.
Ask a query and retrieve packed context
Run A/B benchmark on memory retrieval
Run benchmark suite on memory retrieval
Run diagnostic checks
Explain code at a given location
Build the dependency graph
Query the dependency graph
Rebuild the dependency graph
Apply pre-prompt hook to query with packed context
22 of 31 tools have no visible input schema definitions. Source code shows only tool names and brief descriptions in crates/ctx-cli/src/main.rs, but formal JSON Schema type definitions are not present in the provided codebase snapshot.
No output schemas are documented for any of the 31 tools. LLMs cannot predict what fields to extract or how to chain results to subsequent tools without knowing the response structure.
Tool descriptions are uniformly brief (30-60 chars). While present, they lack context on WHEN to use each tool, what prerequisites exist, and how to distinguish overlapping tools like 'pack' vs 'ask' vs 'hook'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Index source files in the repository
Initialize a CTX repository
Generate MCP server configuration
Serve CTX as an MCP server over HTTP
Serve CTX as an MCP server over stdio
Bootstrap memory from files
Delete a memory entry
Export memory to a file
Get a memory entry by key
Import memory from a file
List memory entries
Search memory entries
Set a memory entry
Interactive menu interface
Install OpenCode integration
Pack context for a query
Prune diff output to max lines
Prune log output to max lines
Rebuild the index for source files
Retrieve relevant code snippets for a query
Display repository statistics
Update CTX to latest version
No error handling or recovery guidance. Tools that fail (e.g., missing file, invalid graph query) provide no direction for retry, alternative approaches, or available options. LLMs will stall on errors.
Parameter descriptions lack constraint details. 'query' appears in 6+ tools but is never documented as free-form string, max length, syntax rules, or examples. 'limit' and 'max_lines' lack min/max bounds.
Overlapping tool names lack disambiguation. 'pack', 'ask', and 'hook' all take 'query', 'budget', and 'attach' params but descriptions do not explain when to pick one vs another. LLMs will guess.
No indication of which tools are idempotent vs stateful. 'index', 'reindex', 'graph_build', 'graph_rebuild' are mutually confusing, agents cannot determine which are safe to retry.
Tools like 'memory_delete' (DESTRUCTIVE), 'memory_set' (WRITE), and 'update' (WRITE) have no confirmation step or dry-run option. Agents can permanently delete memory or corrupt state with no safety net.
Generic tool names 'menu' and 'doctor' do not indicate what they do. 'menu' suggests interactive prompts; 'doctor' suggests diagnostics. Descriptions do not clarify return types or side effects.