Loading ToolBench
A Streamlit-based chatbot application with LangGraph backend supporting multiple tools including web search, stock price lookup, PDF RAG, calculator, and MCP tool integration. Supports multi-thread conversation management with SQLite persistence.
This MCP server has severe definition quality issues. Of 8 tools, 3 are exact duplicates (get_stock_price appears 3 times), indicating poor server architecture. Tool names lack verb-noun clarity (e.g., 'rag_tool' is generic and vague). Descriptions are present but many are generic or lack actionable context. Most critically, input schemas are either missing or incomplete, no parameters have explicit type definitions visible in the tool registration code. The server uses FastMCP for some tools but schemas are not visible in the provided source. Parameter descriptions exist but lack constraint information (enums, ranges, formats). No output schemas are documented. Error handling is not evident. The code shows a LangGraph agent backend but the actual MCP tool definitions are either inferred or hidden in FastMCP decorators.
Add two numbers
Perform a basic arithmetic operation on two numbers. Supported operations: add, sub, mul, div
Fetch latest stock price for a given symbol (e.g. 'AAPL', 'TSLA') using Alpha Vantage with API key in the URL.
Fetch latest stock price for a given symbol (e.g. 'AAPL', 'TSLA') using Alpha Vantage with API key in the URL.
Fetch latest stock price for a given symbol (e.g. 'AAPL', 'TSLA') using Alpha Vantage with API key in the URL.
Multiply two numbers
Duplicate tool registration: get_stock_price appears 3 times with identical definitions (tools #4, #5, #7). This indicates a tool registry bug or careless server setup. Agents will select unpredictably among duplicates, wasting reasoning cycles.
API key embedded in tool parameter (get_stock_price URL includes 'apikey=DK8YGNCN6WOC4KQR'). This exposes the key in agent logs and traces. Violates pattern:secret-injection, credentials must be server-side injected, not passed as parameters.
Generic/vague tool names: 'search_tool', 'rag_tool', 'calculator' do not follow verb_noun convention clearly. 'search_tool' could be any search; 'rag_tool' is jargon. Should be 'search_web', 'retrieve_pdf_content', 'compute_arithmetic'. LLMs rely on names to infer intent.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 40 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 39 | - | v1 |
Retrieve relevant information from the uploaded PDF for this chat thread. Always include the thread_id when calling this tool.
DuckDuckGo web search tool for retrieving information from the internet
No input schema constraints (enums, ranges, formats). 'operation' parameter in calculator is free-form string instead of enum ['add', 'sub', 'mul', 'div']. 'symbol' in get_stock_price has no enum or validation. This invites LLM hallucinations (e.g., passing 'INVALID' as operation, causing silent failures).
No output schemas documented. Agents cannot know what fields to expect from tool responses. For example, rag_tool and search_tool responses are unspecified, does search_tool return URLs, summaries, or both? Does rag_tool return page numbers? This forces LLMs to guess and retry on unexpected fields.
Contradictory documentation: rag_tool description says thread_id is 'optional' but then instructs 'Always include the thread_id when calling this tool.' This ambiguity will cause the LLM to either omit it or include it inconsistently.
No error handling guidance. What happens if search_tool times out? If get_stock_price returns an invalid JSON from the API? If calculator receives a division-by-zero? Agents have no recovery instructions, they will retry blindly or fail silently.
No pagination support for result-returning tools. search_tool and rag_tool may return large result sets. Without limit/offset and total_count, the agent cannot handle pagination. Large responses will exhaust context window.
Parameter descriptions lack constraint hints. 'search query string' is vague, what is max length? Are special characters allowed? Does it support boolean operators? LLMs cannot read JSON Schema pattern fields; they rely on text descriptions to understand constraints.