Loading ToolBench
Python code analysis MCP server providing asynchronous linting and static analysis tools for Python code
Quack has 5 tools with basic schemas and descriptions, but significant gaps limit production readiness. All tools have descriptions (10-100 chars, mostly adequate) and input schemas with type definitions. However, output schemas are completely undocumented, LLMs cannot predict response structure for planning. Parameter descriptions are minimal (e.g., 'Python code content to analyze' lacks format/constraint guidance). No error handling guidance, no recovery paths, and no indication of which operations are idempotent or destructive. Tool naming is clear (submit_code, get_job_results, list_jobs) but the three submission tools (submit_code, submit_code_for_linting, submit_code_for_static_analysis) represent composition issues, the convenience wrappers add little value and dilute focus. No pagination support despite list_jobs potentially returning unbounded results. Missing security considerations (no permission gates, no audit trail hints).
Get the results of a previously submitted job
List all jobs and their statuses
Submit Python code for analysis
Submit Python code for linting analysis
Submit Python code for static type analysis
Output schemas completely undocumented. LLMs cannot predict response structure (e.g., get_job_results returns 'status', 'job_type', 'results', 'execution_time' but this is invisible to the agent). Without documented output schemas, agents cannot plan downstream tool calls or extract required fields reliably.
Three submission tools (submit_code, submit_code_for_linting, submit_code_for_static_analysis) violate single-responsibility principle. The convenience wrappers add minimal value and force LLMs to reason about which to use. Consolidate to submit_code with job_type enum.
list_jobs lacks pagination (no limit, offset, or page parameters). If job history grows large, unbounded results will exhaust context windows. No total count or next_cursor returned.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | <=2025-11-25 | v2 |
Error responses lack recovery guidance. When job_type is invalid, the tool returns {'status': 'error', 'message': str(e)} but does not tell the LLM what valid types are or how to recover. No error classification (retryable vs. fatal).
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are read-only, but this is not declared. Agents cannot infer safety properties without explicit hints.