Loading ToolBench
MCP server that lets an LLM answer plain-English questions against a manufacturing ERP database (demo, read-only)
Three read-only tools with clear naming (list_, describe_, run_) and reasonable descriptions (100-180 chars). All tools have input schemas with Zod validation. However, parameter descriptions are minimal or missing context. list_tables has empty input schema ({}), which is correct but offers no guidance. describe_table and run_query have single string parameters with basic descriptions but lack format constraints, examples of valid input, or error recovery hints. Output schemas are not documented, LLMs cannot predict the structure of returned data. Error handling exists (isError flag) but lacks actionable guidance. No tool annotations (readOnlyHint, idempotentHint) despite all being read-only. Descriptions mention prerequisites ('Call this first') which is good, but do not explain when to use one tool vs another.
Get the columns, types, and keys for one table. Call this before writing a query against a table.
List all tables in the ERP database with their row counts. Call this first to see what data is available.
Run a single read-only SELECT query against the ERP database. Only SELECT/WITH statements are accepted; results are capped at 200 rows. Use list_tables and describe_table first to learn the schema.
Output schemas not documented. LLMs cannot predict the structure of list_tables, describe_table, or run_query responses. Responses are JSON-stringified text blocks, not structured objects with typed fields.
Parameter descriptions lack format constraints and validation rules. 'table' parameter in describe_table has no guidance on valid table names, case sensitivity, or error recovery if table does not exist. 'sql' parameter in run_query lacks examples of valid SELECT syntax, row limit explanation, or what happens if a non-SELECT statement is passed.
Error messages are generic. When describe_table or run_query fails, the response includes err.message but does not guide the LLM on recovery. E.g., 'Table not found' should suggest 'Call list_tables() to see available tables.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
No tool annotations despite all tools being read-only and idempotent. Adding readOnlyHint and idempotentHint would signal to agents that these tools are safe to call repeatedly without side effects.
run_query truncates results at 200 rows but does not return pagination metadata (total_count, has_more, next_offset). LLMs cannot determine if results are complete or if a follow-up query is needed.