Loading ToolBench
ajan-sql demonstrates solid schema-aware SQL access with 8 well-named, read-only tools. All tools have descriptions (10 - 200 chars, within baseline 194 avg). Most tools have input schemas with typed parameters. However, several parameter descriptions are missing or minimal, and output schemas are not explicitly documented. Error handling is present but generic. The server uses a custom TOON format for structured output, which is non-standard and may confuse LLMs expecting JSON. No tool annotations (readOnlyHint, etc.) despite all tools being read-only.
Return columns and types for a given table.
Return query execution plan.
Return foreign key relationships.
Return all tables in the database.
Execute a safe SELECT query.
Return example rows from a table.
Search table and column names across the available schema.
Return server runtime details for onboarding and diagnostics.
Missing input schemas for 4 tools (list_tables, server_info, list_relationships, and implicitly others). Tools without schemas cannot be validated by clients and force LLMs to guess parameter structure.
Output schemas not documented. LLMs cannot plan downstream calls or extract fields without knowing what each tool returns. The custom TOON format is non-standard and opaque to LLMs.
No tool annotations despite all 8 tools being read-only. Adding readOnlyHint=true to each tool helps clients and LLMs understand safety guarantees without reading descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2025-06-18+ | v2 |
Parameter descriptions are minimal or missing context. E.g., 'schema' param in describe_table says 'Schema name (optional, defaults to public)' but does not explain when to use non-public schemas or what happens if the schema does not exist.
Error handling returns classified error codes (NOT_FOUND, INVALID_QUERY, etc.) but does not provide recovery guidance. E.g., 'Table not found' should suggest 'Call list_tables() to see available tables.'