Loading ToolBench
MCP server for Apache Spark code optimization using Claude AI
This server exhibits major definition quality issues. Only 2 tools are defined, and both have critical gaps. Tool 1 (optimize_spark_code) has an acceptable name and schema structure, but the schema references 'spark_code' parameter while the source code shows 'code'. Tool 2 (get_spark_examples) is listed as a resource but treated as a tool, unclear what it actually does. Descriptions are present but vague (68 and 49 characters respectively, below the baseline 194 chars). Critical issue: the source code shows a DIFFERENT tool definition (analyze_performance) not listed in the evaluation inventory, indicating inconsistency between declared and actual tools. The v0-legacy/server.py file defines tools manually without using fastmcp decorators, and the v1/ directory structure suggests migration is incomplete. Parameter descriptions are minimal: 'The Spark code to optimize' provides no guidance on expected input format, size limits, or examples. No output schemas are documented. Error handling is absent, no guidance on what happens if Claude API fails, if the code is invalid, or if optimization fails.
Resource providing example Spark code snippets.
Optimizes Apache Spark code based on specified optimization level.
Tool parameter name mismatch: schema declares 'spark_code' but source code uses 'code'. LLMs will pass the wrong parameter name.
No input schema defined for get_spark_examples. Cannot determine what parameters it accepts or what it returns. Schema score must be 0.
Tool descriptions are too short and vague. 'Optimizes Apache Spark code based on specified optimization level' (68 chars) lacks context on when to use it, what it returns, and any prerequisites. Baseline is 194 chars.
Parameter 'optimization_level' lacks validation guidance. Description only says 'Level of optimization (low, medium, high)' but source code uses enum ['basic', 'advanced']. LLM will pass invalid values like 'low' or 'medium'.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 33 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 42 | - | v1 |
No output schema documented. LLMs cannot determine what fields to expect from tool responses, making it impossible to chain this tool with others or extract results reliably.
No error handling guidance. If Claude API fails, if the code is invalid, or if optimization takes too long, the LLM receives no recovery hint. Source code calls Claude with hardcoded model 'claude-3-opus-20240229' and no timeout.
Tool inventory inconsistency: v0-legacy/server.py defines 'analyze_performance' tool (not listed in evaluation), but the evaluation lists only 'optimize_spark_code' and 'get_spark_examples'. Unclear which tools are actually exposed.
get_spark_examples is listed as a 'resource' but treated as a tool. No schema, no clarity on parameters or return format. If it's a resource, it should use the resources API; if a tool, it needs a full definition.
Parameter 'spark_code' has no length limits, format guidance, or examples. LLMs may pass multi-megabyte files or malformed Python. Validate and document constraints.
Anthropic API key is read from environment (ANTHROPIC_API_KEY), good. But if a future version exposes this as a parameter, credentials will leak into logs. Document that secrets are never parameters.