Loading ToolBench
Self-maintaining code index for LLM and human queries via MCP and CLI
Codelibrarian has 12 well-named, read-only code analysis tools with complete input schemas and descriptions. All tools follow verb_noun naming (search_code, lookup_symbol, get_callers, etc.). Descriptions are clear and action-oriented (avg ~120 chars). All parameters have types and descriptions. However, output schemas are NOT documented, LLMs cannot see what fields to expect from responses. Error handling is absent: no guidance on what to do if a symbol is not found, no recovery suggestions. No tool annotations (readOnlyHint present implicitly but not declared). Parameter descriptions lack format constraints (e.g., 'qualified_name' format, 'file_path' relative vs absolute). Missing pagination guidance for tools that return lists (search_code, get_callers, get_callees, list_symbols). No per-tool output schema documentation.
Return the number of direct callees of a symbol. Efficient alternative to get_callees when only the count is needed.
Return the number of direct callers of a symbol. Efficient alternative to get_callers when only the count is needed.
Generate a Mermaid flowchart diagram of call relationships (callers or callees).
Generate a Mermaid class hierarchy diagram for a given class, showing parents, children, and methods.
Generate a Mermaid flowchart of file-to-file import dependencies.
Find all functions/methods called by the specified symbol.
Output schemas not documented. LLMs cannot see what fields search_code, lookup_symbol, get_callers, etc. return. This forces agents to guess field names and risks extraction errors.
No error handling guidance. If lookup_symbol('nonexistent_func') fails, the tool returns nothing. No recovery hint like 'Try search_code() with a partial name.' Agents dead-end.
Pagination not documented for list-returning tools. search_code, get_callers, get_callees, list_symbols accept 'limit' but no offset/cursor. Large result sets risk context window exhaustion.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Find all functions/methods that call the specified symbol.
Get the inheritance hierarchy for a class: its parent classes and all known subclasses.
Show what modules a file imports and what other files import it.
List symbols filtered by kind, name pattern, or file. Useful for structural queries like 'all classes in module x'.
Look up a code symbol by exact name or qualified name. Returns full signature, docstring, parameters, return type, file path and line number.
Hybrid semantic + full-text search across all indexed code symbols. Returns functions, methods, and classes matching the query with file path and line number.
Parameter format constraints missing. 'qualified_name' (e.g., 'MyClass.my_method') and 'file_path' (relative vs absolute) lack format guidance in descriptions. LLMs may pass invalid formats.
No tool annotations. readOnlyHint should be declared on all 12 tools to signal to clients that these are safe, non-destructive operations.