Loading ToolBench
An MCP server that provides tools for answering support questions from ingested documentation and ingesting markdown/HTML docs into a vector store.
Two tools with clear, detailed descriptions (194-char baseline met). Both have input schemas with typed parameters and descriptions. Ask tool has excellent description explaining verdict computation and output fields (claims, citations, cost). Ingest tool clearly states idempotency. However, output schemas are NOT documented, Ask returns a complex object with verdict/answer/citations/claims/cost but no schema definition visible in code. Error handling returns structured error objects (component/code/message) but lacks recovery guidance. No tool annotations (readOnlyHint/destructiveHint) despite Ask being read-only and Ingest being destructive. Parameter descriptions are good (10-20 chars each) but could specify formats/constraints more explicitly.
Answer a support question from the ingested docs. Returns the verdict (resolved or escalate) with the answer, page citations, the claim-by-claim verifier breakdown, model calls, seconds, and estimated cost. Verdict is computed BEFORE any output is returned — an escalated answer is never included.
Ingest a directory of markdown/HTML docs into the local vector store so the ask tool can answer questions from them. Idempotent: unchanged pages are skipped.
Output schemas not documented. Ask returns complex object (verdict/answer/citations/claims/cost/model_calls/seconds/estimated_cost) but no schema definition visible. LLMs cannot plan downstream operations without knowing return structure.
Missing tool annotations. Ask is read-only but lacks readOnlyHint. Ingest is destructive (writes to vector store) but lacks destructiveHint. Agents cannot distinguish safe from risky operations.
Error handling lacks recovery guidance. ErrorJson returns {error: {component/code/message}} but does not tell LLM what to do next. E.g., 'ingest not-configured' should suggest 'Configure CS_AGENT_STORE environment variable.'
Parameter 'path' in Ingest lacks format constraints. Description says '.md/.html' but does not specify: must be absolute or relative? Must exist? What happens if empty or non-existent?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Ask parameter 'question' has no length limits or format guidance. LLMs may pass empty strings, extremely long queries, or non-text payloads. Specify min/max length and expected format.