Loading ToolBench
A multi-utility chatbot built with LangGraph and Streamlit that supports RAG (Retrieval-Augmented Generation) with PDF uploads, web search, stock price lookup, and calculator tools.
This server has 4 tools with basic schemas and descriptions, but significant quality gaps. All tools have descriptions (10-100 chars) and input schemas with type definitions, meeting minimum viability. However, descriptions are generic and lack LLM-optimization guidance (e.g., 'when to use', 'prerequisites'). Parameter descriptions are minimal. Error handling is present but not recovery-focused. No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Output schemas are partially documented in code but not formally declared. The rag_tool has a critical design flaw: it relies on thread_id state stored in-memory (_THREAD_RETRIEVER_MAPPING), violating stateless tool design. Overall, this is a functional but below-average implementation typical of community projects.
Perform a basic arithmetic operation on two numbers. Supported operations: add, sub, mul, div
Fetch latest stock price for a given symbol (e.g. 'AAPL', 'TSLA') using Alpha Vantage with API key in the URL.
Retrieve relevant information from the uploaded PDF for this chat thread. Always include the thread_id when calling this tool.
DuckDuckGo web search tool for retrieving information from the internet
rag_tool relies on in-memory thread state (_THREAD_RETRIEVER_MAPPING) instead of stateless design. This breaks multi-instance deployments and violates MCP stateless principle.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot determine which tools are safe to retry or have side effects.
get_stock_price exposes API key in URL parameter. This is a security violation, keys should be injected server-side via environment variables.
Descriptions lack LLM-optimization guidance. Missing 'when to use', prerequisites, and expected output structure. E.g., search_tool description does not explain when to call it vs rag_tool.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2026-07-28+ | v2 |
Error responses lack recovery guidance. E.g., rag_tool returns 'No document indexed' but does not suggest uploading a PDF or calling search_tool instead.