MCP server for TestIT (WorkItems, TestRuns, TestPlans)
Server has 15 tools with basic schemas and descriptions, but significant quality gaps. Tool names follow verb_noun convention (good). However, descriptions are minimal (10-50 chars, well below the 194-char baseline), parameter descriptions are sparse or missing entirely, and output schemas are not documented. Many parameters use overly generic 'additionalProperties: true' payloads instead of explicit field definitions. Error handling is present but not recovery-focused. No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications in metadata.
Create a new configuration.
Create a new project.
Delete a project by ID.
Get a configuration by ID.
Get a project by ID.
Get analytics for all test plans in a project.
Get a shared step by ID.
Payload parameters use 'additionalProperties: true' with no field documentation. create_project, update_project, create_configuration, update_configuration all accept opaque 'payload' objects. LLMs cannot infer required fields, valid keys, or expected structure.
Descriptions are too short (10-35 chars, baseline 194 chars). Examples: 'List all projects.' (18 chars), 'Get a project by ID.' (20 chars). Descriptions lack context on WHEN to use each tool, dependencies, or what the response contains.
Parameter descriptions are missing or minimal. 'query' in search_projects has description 'Search filter object.' but no guidance on structure, valid keys, or examples. 'filter' in get_test_run_statistics similarly underdocumented.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Get analytics for a specific test plan.
Get test result statistics for a specific test run, optionally filtered.
List configurations for a project.
List all projects.
List shared steps for a project.
Search projects by filter.
Update an existing configuration.
Update an existing project (PUT).
No output schemas documented. Tools return data but LLMs have no visibility into response structure, field types, or which fields are safe to chain to downstream tools. Violates pattern:tool requirement.
Tool annotations missing. Risk metadata (READ_ONLY, WRITE, DESTRUCTIVE) exists in server metadata but not exposed as tool inputSchema annotations (readOnlyHint, destructiveHint). LLMs cannot infer safety properties from tool definitions alone.