Loading ToolBench
A collection of MCP servers including Snowflake database access and weather forecasting
Server has 5 tools with basic schemas and descriptions, but significant quality gaps. All tools have descriptions (10-50 chars, below 194-char baseline), and input schemas are present but minimal. No parameter descriptions in schemas, LLMs cannot infer what 'table_name' or 'state' mean beyond the bare name. No output schemas documented. Error handling returns generic strings ('Error fetching tables: {str(e)}') with no recovery guidance. Tools lack idempotency hints, permission declarations, and composition support. Naming is acceptable (verb_noun pattern), but descriptions are too terse to guide LLM selection. No tool annotations (readOnlyHint, destructiveHint). Overall structure is functional but falls short of production quality.
Get weather alerts for a US state.
Get weather forecast for a location.
Get schema information for a specific table.
Get list of tables in LAUNDRYGO_LIVE.PUBLIC schema.
Query data from a specific table.
Parameter descriptions missing in JSON schemas. 'table_name', 'state', 'latitude', 'longitude' have no descriptions in the schema, only in docstrings. LLMs cannot read docstrings; they parse JSON Schema. This forces LLMs to guess parameter meaning.
No output schemas documented. Tools return unstructured strings (e.g., 'No tables found in LAUNDRYGO_LIVE.PUBLIC'). LLMs cannot plan downstream calls or extract structured data. Baseline: 100% of A+ tools have documented return types.
Error handling returns generic exception strings with no recovery guidance. E.g., 'Error fetching tables: {str(e)}' tells the LLM nothing actionable. Should return 'Table not found. Try list_tables() first to see available tables.' Baseline: error responses must guide next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Tool descriptions are 30-50 characters, well below 194-char baseline. 'Get list of tables in LAUNDRYGO_LIVE.PUBLIC schema' lacks context on WHEN to use this tool vs alternatives, what it returns, and prerequisites. Descriptions should be 50-200 chars for LLM optimization.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are READ_ONLY but this is not declared in the tool definition. Agents cannot distinguish safe tools from destructive ones without annotations.