Read-only access to four prepared BIRD Mini-Dev SQLite databases with natural language search and SQL execution capabilities
Six tools with complete input schemas and descriptions. Naming follows verb_noun pattern (list_, search_, get_, execute_). Descriptions are clear and action-oriented (avg ~80 chars), explaining WHAT and WHEN. All parameters have types and descriptions. Output schemas are documented via Pydantic contracts (DatasetInfo, QueryResult, SchemaResult, RetrievedContext). No critical security issues, all tools are read-only with hard limits (500-row cap on execute_sql, 50-value limit on get_distinct_values). Gaps: descriptions could be more LLM-optimized (10-200 char sweet spot); no error recovery guidance in descriptions; no tool annotations (readOnlyHint, idempotentHint); parameter constraints (e.g., top_k range, limit bounds) not formally declared in schema.
Execute one validated read-only SQLite query with a hard 500-row cap.
Explore up to 50 distinct values from one validated table column.
Return authoritative columns, types, primary keys, and foreign keys.
List approved datasets, topics, tables, and local availability.
Retrieve relevant DDL, business evidence, and question-SQL examples.
Rank datasets for a natural-language question before choosing one.
No tool annotations (readOnlyHint, idempotentHint) declared. All six tools are read-only and idempotent, but FastMCP server does not expose these hints to clients. Clients cannot infer safety properties without explicit annotations.
Parameter constraints not formally declared in JSON Schema. top_k defaults to 5 but no min/max bounds; limit defaults to 50 but no bounds; max_rows defaults to 500 but no bounds. Descriptions mention limits (e.g., '500-row cap') but schema lacks minValue/maxValue properties.
Error handling descriptions missing. Tools do not document what happens on invalid dataset_id, malformed SQL, or timeout. No recovery guidance (e.g., 'Call list_datasets() first to find valid dataset IDs'). Descriptions lack error classification (retryable vs. user-fixable).
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Output schema documentation incomplete. Pydantic contracts (QueryResult, RetrievedContext) are defined but not visible in tool descriptions. LLMs cannot infer response structure from tool definitions alone, must inspect source code or runtime introspection.
get_distinct_values description vague on search parameter. States 'Optional search filter for column values' but does not clarify if it's a substring match, regex, or exact match. LLMs may pass invalid patterns.