Loading ToolBench
AI Knowledge Base System with code intelligence and memory management. Provides tools for indexing project code, searching symbols, managing memory planets/notes, and exploring code relationships.
BaseMem has 13 tools with inconsistent quality. Most tools have descriptions (good), but parameter schemas are incomplete or missing for many. Tool naming is reasonable (verb_noun pattern mostly followed), but several tools lack proper input validation guidance. Output schemas are not documented. Error handling is minimal, most tools return raw strings without recovery guidance. The code_find tool is particularly complex with 10 parameters, many undescribed or poorly constrained. Advanced tools (compute_similarity, rerank, set_memory_state, get_node, code_list_projects) are conditionally registered but lack schema visibility in the provided source. Overall, this is a C-grade server with functional tools but significant gaps in LLM-optimization and error guidance.
Trace callers/callees of a symbol. Direction: 'callers' (who calls this) or 'callees' (what this calls).
List all indexed files in the project, optionally filtered by pattern.
STEP 1 for any code question, bug, or exploration: find symbols or grep text. Use grep=True for text search. THEN read windows with code_read. Empty result means call code_init(projectRoot) once and retry.
Index project code (tree-sitter). Auto-runs on first code_find.
List all indexed code projects (optional tool, requires BASEMEM_ENABLE_ADVANCED_TOOLS=1).
Read a code file in windows (offset/limit). Offset in lines, limit<=50. Returns file content with line numbers.
code_find has 10 parameters with incomplete schema visibility. Parameters like 'useRegex', 'dead', 'source', 'references', 'grep', 'context' lack descriptions or type constraints. No enum for mutually exclusive search modes (grep vs symbol search).
Advanced tools (compute_similarity, rerank, set_memory_state, get_node, code_list_projects) have no visible input schemas in source code. Conditional registration via BASEMEM_ENABLE_ADVANCED_TOOLS env var means schemas are inferred, not explicit.
Tool naming inconsistency: 'getContext' and 'logInteraction' use camelCase; others use snake_case. LLMs expect consistent naming conventions. Should be 'get_context' and 'log_interaction'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Compute semantic similarity between two text snippets (optional tool, requires BASEMEM_ENABLE_ADVANCED_TOOLS=1).
Fetch memory context for a topic/planet. Returns goal, state, next steps, and recent notes. Call only to refresh or switch topics (auto-injected at session start).
Get a node by ID from the knowledge graph (optional tool, requires BASEMEM_ENABLE_ADVANCED_TOOLS=1).
Get diff context for code review. Pass file paths to review diffs and get context.
Log a decision, fix, or learned fact to memory. Call immediately after making a decision. End sessions with activity='done' and a summary.
Rerank search results by relevance to a query (optional tool, requires BASEMEM_ENABLE_ADVANCED_TOOLS=1).
Set memory state (hot/warm/cold) for a topic (optional tool, requires BASEMEM_ENABLE_ADVANCED_TOOLS=1).
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract structured data. E.g., code_find returns a string; what fields does it contain? code_init returns a summary string, what structure?
Error handling is minimal. Tools return raw strings on failure (e.g., 'Directory not found: {projectRoot}'). No recovery guidance, no categorization of retryable vs fatal errors. LLMs cannot determine next steps.