Loading ToolBench
An MCP server that analyzes stock market data, generates executable Python code for visualization, and uploads generated plots to AWS S3.
Three tools with critical definition gaps. analyze_stock has a lengthy description (300+ chars) but vague parameter guidance; save_code and execute_code lack input schema visibility and have minimal parameter documentation. No tool declares output schemas. Error handling is generic ('Error: {e}') with no recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite execute_code being irreversible. Naming is reasonable (verb_noun pattern), but parameter descriptions are absent or trivial.
Analyzes the stock market data based on the user query and generates executable python code to visualize the stock data. Returns a formatted python script which is ready for execution. The query is a string that must contain the stock symbol (e.g., TSLA, NVDA, MSFT, etc.), timeframe (e.g., 1d, 1mo, 1y), and action to perform (e.g., plot, analyze, compare).
Executes the python code saved in the 'stock_analysis.py' file. Returns the output of the code execution. Automatically uploads the generated PNG file to S3.
The input is a working, executable python code in string format. Saves the generated python code to the 'stock_analysis.py' file. Makes sure that the code is a valid python file, ready for execution.
save_code and execute_code have no visible input schemas. save_code accepts 'code' (string) but schema is inferred from function signature, not declared. execute_code has empty input {} but no formal schema definition.
execute_code is marked IRREVERSIBLE (executes arbitrary code, uploads to S3) but has no tool annotation (destructiveHint) and no confirmation/dry-run pattern. Agents cannot distinguish safe from dangerous calls.
No output schemas documented for any tool. analyze_stock returns a string (Python code), save_code returns a string (confirmation message), execute_code returns a string (execution output or S3 URLs). LLMs cannot plan downstream calls or extract structured data.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
Error handling is generic ('Error: {e}'). No recovery guidance, no error classification (retryable vs fatal), no actionable messages. LLM cannot self-correct or decide next steps.
analyze_stock description includes example queries but no constraint on query format. Parameter 'query' lacks description of required fields (symbol, timeframe, action). LLM must infer structure from examples, risking malformed input.