Interact with the Petstore Swagger API
PetstoreAPI has 10 tools with basic naming and descriptions, but significant gaps in schema completeness, parameter documentation, and error handling. Tool names follow verb_noun convention (good), but descriptions are generic and lack actionable context. Most parameters lack type constraints (enums, ranges). Output schemas are undocumented. Error handling returns raw API responses without recovery guidance. The query_petstore tool is a meta-tool that delegates to other tools, violating single-responsibility principle. No tool annotations (readOnlyHint, destructiveHint) despite clear read/write distinctions.
Create a new user
Find pets by status (available, pending, sold)
Find pets by tags
Get inventory status by status
Get order by ID
Get pet details by ID
Get user by username
Login with user credentials
query_petstore violates single-responsibility principle. It is a meta-tool that delegates to other tools instead of performing a concrete action. LLMs should call specific tools directly (find_pets_by_status, get_pet_by_id, etc.), not a wrapper that re-routes.
No enum constraints on status parameters. find_pets_by_status accepts 'status' as free-form string with description 'available, pending, sold' but no formal enum. LLMs may hallucinate invalid values like 'available_now' or 'pending_approval'.
Output schemas undocumented. Tools return formatted strings via format_api_response() but LLMs cannot parse the structure. No documented return type schema for any tool. Agents cannot reliably extract IDs or fields for chaining.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2026-07-28+ | v2 |
Place an order for a pet
Query the Petstore API with natural language
Error handling returns raw API responses without recovery guidance. make_api_request() returns {'error': '...', 'details': '...'} but does not tell LLM whether to retry, ask user, or abort. No error classification (retryable vs fatal).
No tool annotations. place_order and create_user are destructive (WRITE risk) but lack destructiveHint annotation. get_* tools are read-only but lack readOnlyHint. Agents cannot distinguish safe from risky operations.