A pytest plugin for testing MCP (Model Context Protocol) servers
This is a test framework server with 8 basic tools. All tools have descriptions (10-30 chars, below the 50-200 char LLM-optimized baseline) and complete input schemas with typed parameters. However, descriptions are minimal and lack context about when/why to use each tool. No output schemas documented. Error handling present but generic. Tool names follow verb_noun convention (get_user, create_user, delete_user) which is good, but descriptions do not explain state-modifying consequences or recovery paths. Composition is sound, each tool has one responsibility. No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels (WRITE, DESTRUCTIVE). This is a typical community test server: functional but not production-grade.
Add two numbers
Create a new user
Delete user by ID
Divide two numbers
Get user by ID
List all users
Multiply two numbers
Subtract two numbers
Descriptions are too brief (10-30 chars) and lack LLM-optimized context. 'Add two numbers' does not explain when to use add vs other arithmetic tools, what the output format is, or any prerequisites. Baseline for A+ tools is 50-200 chars with WHAT/WHEN/WHY clarity.
No output schemas documented. LLMs cannot plan downstream calls or extract return values without knowing the response structure. E.g., does get_user return {id, name, email} or a JSON string?
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. delete_user is marked DESTRUCTIVE but the tool definition lacks destructiveHint annotation. create_user is WRITE but lacks annotation. LLMs cannot infer safety properties without explicit hints.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | <=2025-11-25 | v2 |
list_users has empty inputSchema properties (no parameters), but no pagination support (limit, offset, cursor). If user count grows, returning all users will blow context windows. Baseline pattern requires pagination for list tools.
Error messages in call_tool are generic ('Missing required argument', 'Unknown tool'). They do not guide recovery or suggest alternatives. E.g., 'User not found: {user_id}. Try list_users() to see available users.' would be actionable.