Loading ToolBench
AI-powered code context manager. AST-based chunking with vector search for AI agents.
Nanocontext provides 19 tools with explicit schemas and descriptions. However, the implementation exhibits significant quality gaps that prevent a higher score. Naming is inconsistent and abbreviated (svec, sdeep, sreg, sregdeep, stale), these cryptic names violate verb_noun conventions and force LLMs to guess intent. Descriptions are present but generic, ranging from terse ('Vector/semantic search with type filtering') to adequate. Parameter descriptions exist but are minimal (10-30 chars typically). Output schemas are not documented, the tool definitions show inputs but omit what fields the LLM should expect back. Error handling is absent from visible code; there is no guidance on retryability or recovery paths. The server operates over STDIO only, which caps protocol readiness at 50 regardless of definition quality. Despite these issues, the tools are internally consistent (each does one thing), schemas are properly typed with enums where appropriate (e.g., type filter in svec), and the implementation is complete enough to function.
Show likely outbound calls for a symbol
Show likely inbound references for a symbol
Read code snippet from file at location
Get direct dependencies/references for a file or method
List indexed files or search them by partial name
Analyze likely impact for a file or symbol
List project memories
Show readers for a state/property path
Show direct references/callees for a symbol
Abbreviated tool names (svec, sdeep, sreg, sregdeep, stale) violate verb_noun convention and obscure intent. LLMs cannot infer whether 'svec' means 'search vectors', 'semantic vectors', or something else without reading the description. Names should be unambiguous: search_vector, search_semantic, search_regex, search_regex_deep, check_stale.
Output schemas are not documented. The ListToolsRequestSchema response includes tool definitions, but the McpServer.ts code shows tool execution returning JSON via compact() and shortening() functions with abbreviated key mappings (type→t, file→f, method→m, etc.). LLMs receive responses with abbreviated field names and have no schema documentation to parse them correctly.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Add a note to project memory
Deep vector/semantic search with full metadata
Exact text search on indexed code
Regex search on code names and signatures
Deep regex search with full metadata
Check whether the index is stale
Show indexed state/property references
Vector/semantic search with type filtering
Trace a likely execution chain for a symbol
Show writers for a state/property path
Parameter descriptions are minimal and generic (10-30 chars). Examples: 'Search query', 'Limit on number of results', 'File path', 'Location/range in file'. These do not explain WHEN to use the parameter or what format/constraints apply. A description like 'Limit on number of results (1-10)' would be actionable; as written, LLMs do not know if they should pass 1, 10, 100, or 1000.
No error handling or recovery guidance in tool definitions or implementation. There is no documentation of what errors each tool can return, whether they are retryable, or what the LLM should do next. For example, if 'refs' returns a symbol not found, should the LLM retry with a different symbol, search first, or report failure to the user?
The 'remember' tool (write operation) lacks idempotency guarantees and conflict handling. Multiple calls with the same text may create duplicate memories, or overwrites may be ambiguous. No idempotency key, version field, or merge strategy is documented.
Result limiting is inconsistent. Some tools accept 'n' (number) parameters with no documented range (svec, sdeep, sreg, sregdeep, state_refs, readers, writers). The code in McpServer.ts shows outputLimit() enforces a max of 10, but this limit is not documented in tool descriptions, forcing LLMs to discover it through trial and error.
Tool composition relies on implicit chaining. For example, 'search' and 'svec' both return search results, and 'code' accepts a 'f' (file) parameter. There is no documented guarantee that the 'f' values returned by search are valid inputs to 'code'. If the field name or format differs, chaining breaks.
The 'stale' tool has no input parameters and minimal documentation ('Check whether the index is stale'). The return type is not documented. Does it return a boolean, a timestamp, or a structured object? Without output schema documentation, LLMs cannot reliably interpret the result.