Loading ToolBench
MCP server that provides chat completion capabilities via OpenRouter API with OAuth authentication
Single tool 'chat_completion' has a reasonable description and schema, but critical gaps in parameter annotation, output schema documentation, and error handling guidance significantly reduce quality. The tool description is adequate (~88 chars) but lacks clear usage guidance and prerequisite documentation. Input schema is present with types but parameter descriptions are minimal. No documented output schema, no pagination support, no error recovery guidance. The server appears production-ready in transport (HTTP) but the tool definition itself would not pass code review for agent safety.
Generate chat completion using OpenRouter API with OAuth authentication
Output schema is not documented. LLMs cannot predict the response structure, forcing guesswork about available fields. The code shows ChatCompletionResponse and ChatCompletionStreamResponse types internally, but these are never exposed in the tool definition or documentation.
Parameter descriptions are sparse and lack constraints. 'model' description says 'The model to use (default: anthropic/claude-3.5-sonnet)' but does not document: what models are valid? Are they OpenRouter model IDs? Can the user pass any arbitrary string? What happens on invalid model? No enum, no pattern, no validation guidance.
No error handling guidance in tool description. The code includes logic to handle non-200 responses and missing authentication, but the tool description does not tell the LLM what errors to expect or how to recover. LLMs will not know if a failure is retryable, user-fixable, or fatal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | 2025-03-26+ | v1 |
No guidance on streaming behavior. The tool accepts streaming implicitly via Accept header detection, but the description does not document this. LLMs cannot reason about whether streaming is available, when to use it, or how to handle Streamable HTTP responses. The tool definition should document both response modes.
OAuth token is not visible in tool definition but is required for operation. The description mentions 'OAuth authentication' but does not explain the flow or what happens if the user is not authenticated. The code returns a clear 401 error with message 'Authentication required. Please login with OpenRouter OAuth.' but this is backend logic, not documented in the tool interface.
No pagination or result limit guidance. While this is a single-response tool (not a list), if the model generates very long output, there is no documented limit or truncation strategy. For streaming responses, the token limit is implicit in the model's max_tokens, but this should be documented.