Automatically discover and document web application APIs
The Spectral MCP server exposes a single dynamically-registered tool with minimal definition quality. The tool is registered at runtime from discovered web application APIs, but the actual tool definitions lack critical quality attributes. The visible code shows a generic tool name pattern '{app_name}_{tool_name}' with empty input schema (no properties declared), no parameter descriptions, and only a generic description stating the tool 'corresponds to a business capability.' This violates fundamental patterns around naming clarity, parameter documentation, and schema completeness. The tool name pattern provides zero semantic information about what action the tool performs, it does not start with a verb, does not indicate whether it's a GET or POST operation, and does not convey intent. The empty input schema means agents have no guidance on what parameters to pass. The generic description fails to explain WHAT the tool does, WHEN to use it, or WHAT it returns. The code does show thoughtful error handling (auth errors direct users to login), HTTP request building with parameter injection, and timeout protection (30s), but these cannot overcome the fundamental definition quality gaps.
Dynamically registered MCP tool corresponding to a business capability exposed by a discovered web application API
Tool name provides no semantic information. Pattern '{app_name}_{tool_name}' does not start with an action verb (get_, create_, update_, delete_, search_, list_, send_). LLMs cannot infer what the tool does from the name alone.
Input schema is completely empty: {"type":"object","properties":{},"required":[]}. No properties declared, no parameter names visible, no types.
Tool description is generic and non-actionable: 'Dynamically registered MCP tool corresponding to a business capability exposed by a discovered web application API.' This fails to answer: What does it do? When should the LLM call it? What does it return? Violates pattern:tool-description.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 14 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 11 | - | v1 |
No parameter descriptions visible in the schema. Even if parameters are dynamically discovered at registration time (via ToolDefinition.parameters), the empty schema shown means agents get no guidance on what parameters are required, optional, or what they control.
Output schema is not documented in the tool definition. The _handle_call() function returns raw HTTP response text ('HTTP {status}\n\n{response.text}'), but the MCP tool definition does not declare what the response structure is. LLMs cannot plan downstream operations without knowing what fields to expect.
No error categorization or recovery guidance. Error handling returns JSON with 'error' field, but does not distinguish between retryable, user-fixable, or fatal errors. Auth errors direct users to 'spectral auth login {app_name}', which is helpful, but HTTP errors return only status code and raw response text with no guidance.
Dynamic tool registration means schemas are generated at runtime from ToolDefinition objects discovered from the filesystem. The visible code in cli/formats/mcp_tool.py shows ToolDefinition has a 'parameters' field (dict[str, Any]), but the MCP tool created in _make_mcp_tool() passes tool.parameters directly without validation or schema normalization. If a discovered tool has malformed parameters, the MCP server will expose it as-is.
No parameter-level documentation visible. The ToolDefinition model includes a 'parameters' dict, but tools using this server have no pattern for documenting what each parameter does. There is no guidance in the code to ensure parameter descriptions are populated.