Read-only tools over accounting data — budget vs actual analysis, GL queries, and AP invoice tracking for a fictional wholesale distributor.
Three tools with clear, actionable descriptions (10 - 50 words each). All have input schemas with type definitions. However, output schemas are undocumented (no return type specs), parameter descriptions are minimal or absent, and error handling lacks recovery guidance. describe_schema and run_query have empty input objects or single params with no description text. monthly_close_summary accepts only one param with a format hint but no validation rules. No tool declares idempotency, permissions, or error categories. Naming is verb-forward (describe_, run_, monthly_) but generic. Overall: functional definitions suitable for read-only accounting queries, but missing LLM-optimization patterns (output schemas, param descriptions, error recovery) that would elevate to 70+.
List every table and its columns in the accounting database.
Budget vs actual for one month (YYYY-MM), worst variances first.
Run a read-only SELECT against the accounting database (500-row cap).
Output schemas undocumented. All three tools return strings or structured text, but no JSON schema or field documentation exists. LLMs cannot plan downstream calls or extract structured data.
Parameter descriptions missing or minimal. run_query's 'sql' param has a description, but describe_schema has no params and monthly_close_summary's 'month' param lacks validation rules (format, range, examples).
Error handling lacks recovery guidance. run_query returns error strings ('REFUSED', 'SQL ERROR') but does not guide the LLM on next steps (retry, ask user, call a different tool).
No tool annotations (readOnlyHint, idempotentHint). All three are read-only and idempotent, but these properties are not declared in the schema, forcing LLMs to infer safety from descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
run_query result cap (500 rows, 40KB text) is enforced but not documented in the output schema. LLMs cannot predict truncation or plan pagination.