This repository is primarily an SDK/framework for building MCP servers, not a production MCP server itself. The 5 tools evaluated are demonstration examples across different files (basic_tool.py, tool_progress.py, legacy_as_server.py). While schemas are present and descriptions exist, they are minimal and lack depth. Tool naming follows basic conventions (verb_noun), but descriptions are quite short (avg ~40-50 chars), parameter documentation is sparse, and output schemas are not explicitly documented. Error handling guidance is absent. The 'get_time' tool appears twice (duplicate in evaluation data), suggesting the actual active tool set may be smaller. As an SDK, the framework demonstrates competence, but the example tools do not meet production quality standards.
Get the current server time. This tool demonstrates that system information can be protected by OAuth authentication. User must be authenticated to access it.
Get the current server time. This tool demonstrates that system information can be protected by OAuth authentication. User must be authenticated to access it.
Get weather for a city.
Execute a task with progress updates.
Add two numbers together.
Tool descriptions are too brief (avg 40-50 chars, baseline 194 chars). Descriptions lack context for when/why to select each tool and what they return. 'Add two numbers together' and 'Get weather for a city' provide minimal guidance for LLM decision-making.
No documented output schemas visible for any tool. While input schemas are present with types and descriptions, the return value structure is not specified. LLMs cannot plan downstream tool calls or extract needed fields without knowing what each tool returns.
Parameter descriptions are minimal. The 'get_weather' tool has 'unit' parameter described only as 'Temperature unit' without specifying valid values (celsius, fahrenheit, kelvin?). LLMs will hallucinate invalid values.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 60 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 46 | - | v1 |
No error handling guidance visible. Tools do not document recovery paths, retryability, or how LLMs should respond to failures. E.g., what should an agent do if get_weather fails for an invalid city?
Duplicate 'get_time' tool listed twice from the same file (legacy_as_server.py). This suggests either incomplete enumeration or an actual duplicate in the codebase that should be resolved.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. All tools are marked READ_ONLY, but this is not reflected in the tool definitions where annotations could inform LLM behavior.