Look up and compare LLM API pricing across 128+ models and 22+ providers. Query pricing details, calculate API call costs, compare models, and find the cheapest options for your workload.
Six tools with clear verb-noun naming (get_, calculate_, compare_, find_, list_, search_) and consistent descriptions (100-150 chars each). All tools have documented input schemas with typed parameters and descriptions. Output schemas are implicit but inferable from code. Error handling is present but minimal, most errors return generic dicts without recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all being read-only. Parameter descriptions are adequate but lack format constraints (e.g., no enum for provider names, no min/max for limit). Composition is sound: tools chain well (search_llm_models → get_model_pricing → calculate_api_cost). No secrets exposed. Naming is consistent and action-oriented.
Calculate the exact cost of an LLM API call.
Compare the cost of the same workload across multiple models.
Find the cheapest models for a given token workload.
Get pricing details for a specific LLM model.
List all supported LLM providers with their model counts and cheapest pricing.
Search for LLM models by name or filter by provider.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all six tools being read-only. LLMs cannot infer safety properties without explicit hints.
Error handling returns generic dicts without recovery guidance. E.g., 'Model not found' should suggest 'Use search_llm_models() to find available models.' Current errors give LLMs no actionable next step.
Parameter 'provider' in search_llm_models lacks enum constraint. Description lists examples ('OpenAI', 'Anthropic', 'Google') but no formal enum, inviting hallucinated provider names.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Parameter 'limit' in find_cheapest_models lacks explicit min/max constraints in description. Code enforces max 20, but description only says 'default 5, max 20', no mention of minimum.
Output schemas are not formally documented. While code returns structured dicts, no JSON Schema definitions are visible for tool responses. LLMs must infer output structure from code.