Loading ToolBench
Local-First AI agent with privacy-aware routing between Ollama and Claude
Single tool with adequate naming (query_database) and reasonable description (~280 chars, within baseline 194-avg range). Input schema present with type and description. However, output schema is undocumented (returns JSON string, not structured object), and error handling lacks recovery guidance. Security checks are implemented server-side but not declared in tool metadata. No tool annotations (readOnlyHint, etc.) despite read-only semantics being critical.
Execute a read-only SQL query against the database. Args: query: SQL SELECT query to execute. Only SELECT statements allowed. ctx: FastMCP context for logging and accessing app state. Returns: JSON string containing query results as list of dictionaries, or error message if query is invalid. Security: - Rejects queries containing DROP, DELETE, INSERT, UPDATE, ALTER, TRUNCATE, CREATE, or REPLACE keywords. - Only allows SELECT statements. Example: >>> await query_database("SELECT * FROM transactions WHERE status = 'pending'") '[{"id": 1, "description": "...", "amount": 299.99, "status": "pending"}]'
Output schema undocumented. Tool returns JSON string, but LLM cannot infer structure (list of dicts with columns: id, description, amount, status, created_at). Without documented output schema, downstream tool chaining and result extraction are error-prone.
Missing tool annotations. Tool is read-only and idempotent, but lacks readOnlyHint and idempotentHint metadata. LLMs cannot infer safety properties without explicit annotations.
Error responses lack recovery guidance. Errors like 'Query rejected: Only SELECT statements are allowed' do not suggest next steps (e.g., 'Rewrite as SELECT' or 'Call describe_schema() to see available tables'). Per pattern:recovery-guide, errors must guide the LLM toward resolution.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | <=2025-11-25 | v2 |
Parameter description lacks format constraints. 'query' param description mentions 'SELECT * FROM transactions WHERE status = pending' as example, but does not state max query length, forbidden keywords explicitly, or character restrictions.
No pagination or result limit enforcement. Tool description does not state max rows returned. Large result sets can exhaust context window. Per mxe:enforce-result-limits, tools should cap results (e.g., 100 rows) and offer pagination.