Pydantic AI agent that identifies which Contrast Security application corresponds to a code repository
Critical gaps across all three tools. Tool definitions are inferred from docstrings in mcp_tools.py rather than explicit MCP schema registration. No visible input schemas, parameter type definitions, or structured output schemas. Descriptions exist but are generic (avg ~60 chars) and lack actionable guidance for LLM selection. The fs_* tool is delegated to an external server with no local schema visibility. contrast_search_applications and contrast_get_route_coverage lack parameter documentation, type constraints, and error recovery guidance. No evidence of pagination, result limits, or chaining IDs. Security: Contrast credentials passed via environment variables (correct), but no per-tool permission declarations or audit logging visible.
Get route/endpoint coverage for a Contrast application to validate against extracted routes from source code
Search Contrast applications by name, tags, or other criteria to find matching applications
Filesystem access tools provided by @modelcontextprotocol/server-filesystem MCP server for exploring repository structure and reading files
No input schemas visible for any tool. Tools are registered via MCPServerStdio with tool_prefix but no explicit schema definitions in source code. Cannot verify parameter types, constraints, or required fields.
contrast_search_applications and contrast_get_route_coverage lack parameter documentation. No descriptions of what 'name', 'tags', 'criteria', or 'application' parameters accept, their types, or valid ranges. LLMs cannot infer correct usage.
No output schemas documented. Descriptions do not specify what fields are returned, their types, or structure. LLMs cannot plan downstream tool calls or extract required data (e.g., application_id for chaining).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 29 | 2026-07-28+ | v2 |
No pagination or result limits documented. contrast_search_applications may return hundreds of applications; no limit, offset, or next_cursor parameters visible. Risk of context window exhaustion.
No error handling guidance. Tool descriptions do not explain what errors are possible, whether they are retryable, or what the LLM should do next (e.g., 'Application not found, try search_applications with partial name').