City of Chicago Socrata open-data portal
Three tools with basic schemas and minimal descriptions. Tool names lack action verbs (datasets, query, metadata are nouns/generic). Schemas present but incomplete: 'datasets' and 'query' lack descriptions for limit/offset/where/select/group/order params. No output schema documentation. Error handling minimal, callTool throws generic errors without recovery guidance. No idempotency hints, no pagination metadata (total count, next_cursor). Transport unknown, capping protocol readiness.
Search dataset catalogue.
Resource metadata.
SoQL query.
All parameter descriptions missing. 'limit', 'offset', 'where', 'select', 'group', 'order' have no descriptions in schema. Rubric requires every parameter to have a description explaining what it controls.
Tool names lack action verbs. 'datasets', 'query', 'metadata' are nouns/generic. Should be 'search_datasets', 'execute_query', 'get_metadata' to signal intent before LLM reads description.
No output schema documented. LLMs cannot plan downstream calls or extract fields without knowing response structure. Rubric baseline: 100% of A+ tools have documented return types.
Error handling provides no recovery guidance. callTool throws 'Unknown tool' and HTTP status errors without actionable next steps. Rubric requires: 'Error responses must tell the LLM what to do next.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
Transport unknown. Package.json lists no MCP server framework or transport binding. Cannot verify HTTP, SSE, or STDIO. Caps protocol readiness at 40/100.