Loading ToolBench
MCP server exposing a governed answer engine for tabular data. Provides the same deterministic semantic layer and gated SQL fallback as the REST API and Streamlit UI, with tools for listing datasets, asking questions, resuming clarifications, and providing feedback.
Server has 4 tools with basic schemas and descriptions. Naming follows verb_noun convention (list_, ask, resume, feedback). Descriptions are present but brief (avg 95 chars, below 194-char baseline). All tools have input schemas with type definitions. However, output schemas are not documented, critical gap for LLM planning. The 'ask' tool supports multi-turn via thread_id, but lacks error guidance on clarification flows. Parameter descriptions are minimal; 'question' and 'message' lack format/constraint details. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. Error handling is absent from visible code.
Ask a question about one dataset. Starts a new thread when thread_id is omitted, or continues an existing one when supplied -- the same thread_id also lets a later `resume` or `feedback` call refer back to this turn.
Record a thumbs up/down on a thread's last answer. A thumbs up on a guarded-fallback answer (one with no matched semantic metric) saves it as a verified query for the fast path.
List the datasets available to ask questions against.
Answer a pending clarification for a thread returned by `ask` with resumable=true. Fails if the thread has nothing pending.
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., does 'ask' return thread_id, clarification_needed, answer?). Breaks tool chaining.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint). 'feedback' is marked WRITE but tool definition lacks destructiveHint. 'ask' and 'resume' lack idempotentHint despite thread-based state. Agents cannot reason about retry safety.
Parameter descriptions are minimal. 'question' (ask tool) lacks format/length constraints. 'message' (resume tool) lacks guidance on expected content. 'rating' enum is documented but 'comment' lacks length/format hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
No error handling guidance. 'resume' states 'Fails if thread has nothing pending' but does not explain what error the LLM receives or how to recover. No recovery guides for invalid thread_id, missing dataset, or malformed questions.
Thread-based conversation state requires clarification. 'ask' description mentions 'resumable=true' but this field is not in the input schema. Unclear whether all 'ask' calls are resumable or only some. Ambiguous for LLM planning.