Loading ToolBench
Selects optimal LLM API cost model tiers based on monthly token volume, task complexity, and latency requirements
This MCP server has a single tool with a simple schema but significant quality gaps. The tool name `select_optimal_model` lacks a verb prefix (should be `get_optimal_model` or `recommend_optimal_model`), and the description, while present, is only 122 characters, within acceptable range but generic. The input schema is visible and properly typed, but parameter descriptions are minimal (10-20 chars each) and lack actionable detail. The output schema is not documented in the tool definition, LLMs have no visibility into what fields will be returned. No error handling, validation, or recovery guidance is present. The tool operates on a single concern (model selection) but lacks composition context (e.g., what to do if monthly_token_volume is invalid). The implementation is a simple lookup table with hardcoded cost multipliers, making it brittle and unmaintainable.
Selects optimal LLM model tier based on monthly token volume, task complexity, and latency requirements
Tool name lacks action verb. 'select_optimal_model' is passive; should start with 'get_', 'recommend_', or 'choose_' to signal intent to LLM.
Output schema not documented. Tool returns a dict with 'recommended_model_tier', 'estimated_monthly_cost_usd', and 'savings_vs_current_usd', but LLMs cannot see this schema. Add explicit output schema documentation in tool definition.
Parameter descriptions are too brief (10-20 chars) and lack actionable constraints. 'Expected monthly token volume' provides no guidance on range (min/max), units, or precision. 'Maximum acceptable latency in milliseconds' doesn't clarify if values are milliseconds (ms) or seconds (s) in context.
No error handling or validation guidance. If monthly_token_volume is negative, zero, or exceeds realistic bounds, tool silently produces nonsensical costs. No actionable error messages for invalid task_complexity values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 30 | 2026-07-28+ | v2 |
task_complexity parameter description lists enum values ('LOW, MEDIUM, HIGH') in text instead of declaring as JSON Schema enum. LLMs may hallucinate invalid values like 'VERY_HIGH' or 'critical'.
Logic is oversimplified and potentially misleading. Cost model uses only task_complexity and monthly_token_volume; completely ignores latency_requirement_ms parameter in decision logic. LLMs may pass realistic latency values but see them ignored.