Loading ToolBench
Stateless MCP server for OpenRouter providing model discovery, chat, analytics, and account management tools
OpenRouter MCP server has 13 well-defined tools with consistent naming, mostly complete input schemas, and clear descriptions. All tools follow verb_noun naming convention (list_*, get_*, query_*). Input schemas are present and typed for all tools. However, output schemas are not documented in the provided code, descriptions vary in quality (some generic), and error handling recovery guidance is absent. Tool descriptions range from 78-319 characters, meeting the baseline expectation. Parameters have types and descriptions. The server uses Zod validation internally but does not expose output schema documentation to the LLM.
Send a message to an OpenRouter model and receive a response
Send the same message to multiple OpenRouter models and compare their responses
Get the schema for available analytics dimensions, metrics, granularities, and operators
Get current account credit balance and total usage
Get detailed information about a completed generation including latency, provider, and cost
Get detailed information about a specific OpenRouter model including pricing, benchmarks, architecture, and capabilities
Output schemas not documented. No tool definition declares what fields are returned. LLMs cannot plan downstream tool chaining or parse responses reliably.
Generic descriptions on utility tools. 'get_credits' (50 chars), 'get_analytics_schema' (50 chars), and 'list_app_rankings' (60 chars) lack context for when/why to call them. Baseline expectation is 50-200 chars with clear WHAT/WHEN guidance.
No error recovery guidance. Tools like 'chat_with_model' and 'compare_models' (WRITE risk) have no documented error responses or recovery steps. If an API error occurs, LLM receives no actionable guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 38 | - | v1 |
List API usage activity by date with optional filtering and grouping
Get ranked applications by popularity or trending metrics
List available endpoints for a specific model across different providers
Get ranked models by usage within a specified time period and modality
List available OpenRouter models with optional filtering by architecture, category, pricing, modalities, context length, and benchmarks
List OpenRouter providers with optional filtering by datacenter location and headquarters country
Query analytics data with dimensions, metrics, filters, and time range
Missing pagination guidance in descriptions. Tools like 'list_models' and 'list_activity' accept limit/offset but descriptions do not explain pagination behavior, max results returned, or when pagination is required.
Parameter 'q' in list_models and list_providers lacks format guidance. Free-text search parameters invite hallucinated queries. Descriptions should clarify: partial match? exact match? wildcards supported?
No API key handling guidance in tool descriptions. Tools require OPENROUTER_API_KEY but descriptions do not mention authentication prerequisites or consequences of missing credentials.