Loading ToolBench
An MCP proxy giving AI agents safe, reversible tool execution: contracts, dry-run planning, policy, approvals, saga execution, and rewind.
Belay demonstrates solid tool definition practices with consistent naming (verb_noun pattern), explicit risk annotations, and clear parameter schemas. All 11 tools have descriptions and input schemas with typed parameters. However, descriptions are terse (avg ~60 chars, below the 194-char baseline), lack actionable context for LLM selection, and omit output schema documentation. Parameter descriptions are minimal. Error handling guidance is absent from tool definitions. Tool annotations (readOnlyHint, destructiveHint) are present and correctly applied, which is a strength. The CRM and FS tool suites are well-composed (single responsibility, clear naming), but descriptions don't explain WHEN to use each tool or what downstream tools expect.
Delete every record with `last_seen < before_year` (plan.md §10 demo tool).
Create a record. Not read-only; overwrites if `id` already exists.
Delete a record. Marked destructive; a hint never authorizes on its own.
Dump every record -- used by tests to assert the CRM's exact state.
Fetch one record by id. `existed: false` (and no `fields`) if absent.
Bulk-create/overwrite records.
Overwrite a record's fields wholesale (used both forward and as undo).
Tool descriptions are terse (avg ~60 chars) and lack actionable context. E.g., 'Fetch one record by id' does not explain when to use crm.get vs crm.export_records, what fields are returned, or what downstream tools expect. Descriptions should be 50 - 200 chars and answer: WHAT, WHEN, and WHAT NEXT.
Output schemas are not documented in tool definitions. LLMs cannot plan downstream calls or extract required fields (e.g., what does crm.get return? Is it {id, fields, existed}?). Every tool must document its response structure.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2025-06-18+ | v2 |
Delete a file. Marked destructive but a hint never authorizes on its own (spec §4.6).
List every file in the sandbox, relative to its root.
Read a file's contents as text.
Overwrite (or create) a file with `content`. Not read-only, not marked destructive.
Parameter descriptions are minimal or absent. E.g., crm.import_records has a 'records' param described only as 'Records to import', what structure? What fields are required? What happens on conflict? LLMs cannot infer this.
No error handling guidance in tool definitions. If crm.get returns 'existed: false', what should the LLM do next? If fs.write_file fails due to permission denied, should it retry or ask the user? Error responses must guide recovery.
Tool annotations (readOnlyHint, destructiveHint) are correctly applied and present in the code, which is a strength. However, they are not visible in the tool definitions provided, verify they are exposed in the MCP Tool objects returned to clients.