Loading ToolBench
One permanent memory for every AI coding agent — a unified agentic harness with shared memory and capability-based multi-provider routing, local-first.
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. sources_list has empty input schema ({}), put_page lacks output schema documentation, and submit_job has nested object parameters without field-level descriptions. Descriptions are present but generic (10-50 chars for some). No error handling guidance, no idempotency hints, no permission scopes declared. Tool names follow verb_noun pattern (good), but parameter naming inconsistencies (e.g., 'source' vs 'source_id') and missing constraints on enums/ranges.
Store a page of content in a memory source with a given slug
List all memory sources with their federation status and page count
Submit an autopilot cycle job with specified phases (sync, extract, embed)
sources_list has empty input schema ({}). No parameters documented, making it impossible for LLMs to understand what inputs it accepts or whether it supports filtering/pagination.
Output schemas not documented for any tool. LLMs cannot infer what fields to expect (e.g., sources_list returns {sources: unknown[]} but field structure is opaque; put_page and submit_job return types are completely undocumented).
Parameter naming inconsistency: put_page uses 'source' (defaults to 'agent-memory') but submit_job uses 'source_id'. This forces LLMs to reason about field mappings and risks passing wrong parameter names.
submit_job's 'data' parameter is an object with nested 'phases' (array of strings) and 'source_id' (optional string), but no enum constraint on phases or description of valid phase values (sync, extract, embed are inferred from code, not documented).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
No error handling guidance. Tools provide no recovery hints (e.g., 'If source not found, call sources_list first'). No categorization of errors as retryable vs fatal. Agents cannot self-correct on failures.