Loading ToolBench
Long-term memory for agentic tasks. It tells you when it doesn't know.
Two tools with partial schema visibility. 'memory' tool has a comprehensive input schema with 6 parameters (action, query, name, path, limit, scope), all typed and described. 'memory_write' tool description exists but input schema is not visible in provided source. Both tools lack documented output schemas, critical for LLM planning. Descriptions are functional but brief (67 and 54 chars respectively), below the 194-char baseline for production tools. No error handling guidance visible. Tool naming follows verb_noun pattern (memory, memory_write) but 'memory_write' is generic; 'capture', 'import', 'index' would be more specific. Risk annotations present (READ_ONLY, WRITE) but no idempotent hints. Parameter descriptions are present but lack format constraints (e.g., 'max 8192 characters' for query is stated, but no enum for 'action' values despite listing them in description).
Read-only memory tool: search, retrieve latest, verify, get, thread, neighbors, probe_status, index_status
Write-enabled memory tool: capture, import, index, demote, promote
memory_write tool has no visible input schema in source code. Cannot verify parameter types or constraints.
No output schemas documented for either tool. LLMs cannot plan downstream calls or extract required fields (e.g., does 'search' return result_id, score, content?). Violates pattern:tool and mxe:include-chaining-ids.
Tool descriptions are 50 - 67 characters, well below 194-char baseline. 'Read-only memory tool: search, retrieve latest, verify...' lists actions but does not explain WHEN to use this tool vs memory_write, or what the LLM should expect in return.
'action' parameter accepts free-form string with 10+ valid values (search, latest, thread, verify, import, capture, index_status, get, neighbors, index, demote, promote, probe_status). Should be declared as enum to prevent hallucinated actions.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | <=2025-11-25 | v2 |
No error handling guidance visible. If a search returns no results, or an import fails, what should the LLM do? No recovery hints (e.g., 'Try a broader query' or 'Check file format'). Violates pattern:recovery-guide.