Loading ToolBench
Nine tools with basic schemas and descriptions, but significant gaps in parameter documentation, error handling, and output structure. All tools have descriptions (10-50 chars, below the 50-200 char baseline for LLM optimization), and input schemas are present but incomplete. No output schemas documented. Error handling is minimal, tools return raw stderr without recovery guidance. Parameter descriptions are generic ('Path to repository', 'Archive to extract from') and lack constraints, ranges, or format hints. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Naming is consistent (verb_noun: borg_init, borg_create, borg_list) but descriptions do not explain WHEN to use each tool or dependencies between them. Missing: pagination for list results, batch operations, and structured output schemas.
Check repository consistency.
Create a new backup archive.
Extract files from an archive.
Show repository or archive information.
Initialize a new Borg repository.
List repository contents or archive contents.
Mount repository or archive as FUSE filesystem.
No output schemas documented. Tools return unstructured strings (e.g., 'borg_list' returns raw stdout). LLMs cannot parse results reliably or chain tools without explicit field definitions.
Descriptions are too short (10-50 chars, baseline 50-200). 'Initialize a new Borg repository' lacks WHEN to use, prerequisites, or side effects. LLMs cannot distinguish borg_init from borg_create without richer context.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). borg_prune is marked DESTRUCTIVE but has no annotation; borg_list is READ_ONLY but unmarked. Agents cannot infer safety or retry semantics.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
Prune old archives according to retention policy.
Unmount a FUSE filesystem.
Error handling returns raw stderr without recovery guidance. 'Failed to initialize repository: <stderr>' tells LLM nothing about whether to retry, ask user, or abort. No error classification (retryable vs fatal).
Parameter descriptions lack constraints. 'Encryption mode (none, repokey, ...)' is an enum but not declared as such in schema. 'Compression algorithm' lists options in description instead of JSON Schema enum. LLMs may hallucinate invalid values.