Loading ToolBench
Human-friendly Typer CLI for Anna's Archive: search, fast MD5 download, Markdown normalization, Chroma ingestion, semantic search.
Three tools with explicit schemas and descriptions, but several quality gaps. All tools have input schemas with types and descriptions present. Naming follows verb_noun convention (search_catalog, download, search_downloaded_text). Descriptions are present but vary in quality, most fall in the 50-150 character range, which is adequate but terse. No parameter validation guidance for the 'secret_key' parameter (exposed as a tool parameter rather than server-side injection). Error handling is not documented in any tool description. Output schemas are not explicitly documented for any tool. Parameters are typed but lack format specifications and constraint guidance. The 'download' tool conflates two concerns: downloading and normalizing artifacts.
Download and normalize an artifact.
Search Anna's Archive catalog.
Return markdown snippets around a needle for an existing artifact.
Secret key exposed as tool parameter (secret_key in download tool). Credentials must never appear as tool parameters and should use server-side secret injection via environment variables or vault. Agent traces log every parameter, causing credentials to leak into logs.
Output schemas not documented. Tools return structured data but LLMs cannot determine what fields to expect, forcing them to parse unstructured responses or guess at downstream field availability. search_catalog returns SearchResult objects, download returns a normalized artifact path with metadata, search_downloaded_text returns markdown snippets, none documented in tool definitions.
Error handling not documented. No tool description indicates what happens on failure, what error types are possible, or what recovery steps to take. LLMs receive no guidance on whether errors are retryable or what to do next.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 50 | - | v1 |
download tool combines multiple concerns: fetching a file, normalizing artifacts, performing OCR, and optionally ingesting into vector store. Should be split into separate tools (download_artifact, normalize_artifact, ingest_chunks) so agents can compose them.
search_catalog limit parameter defaults to 20 but description states cap at 200 without explaining why agents should choose 200 vs 20. No guidance on when to use pagination or how pagination works. Result size is not capped in tool description.
md5 parameter in download and search_downloaded_text lacks format description. Should specify: 'A 32-character lowercase MD5 hash (e.g. a1b2c3d4e5f6... format validation missing). This forces LLMs to guess valid MD5 format and risks passing invalid values.
Tool descriptions are generic and lack WHEN context. 'Search Anna's Archive catalog' does not explain when to call this vs other search tools, what query format works best, or what fields the LLM should expect in results. Descriptions should guide LLM tool selection.