Loading ToolBench
Multi-tool chatbot server integrating MCP clients, LangChain tools, OAuth integrations (Google, GitHub, Facebook, Twitter), RAG capabilities, and vector search with FastAPI backend
This MCP server exhibits significant definition quality gaps across all 8 tools. While tool names follow verb-noun conventions (search_tool, get_stock_price), descriptions are consistently minimal (10-40 characters), and input schemas are only partially visible in the source. The codebase shows integration with FastMCP and LangGraph for multi-tool orchestration, but the actual tool definitions lack the depth required for production LLM agent use. Schema information is referenced in the scoring snapshot but detailed parameter descriptions and output schemas are not visible in the provided source. Error handling, parameter constraints, and recovery guidance are absent. The server implements tool gathering and user permission management at the API layer (tools.py, user_tool_status.py) but does not expose these as part of the MCP tool definitions themselves.
Fetch Facebook profile for the connected account.
Return gender and probability for a given first name.
Retrieve current stock price information
Fetch GitHub repositories for the connected account.
Fetch basic Google profile for the connected account.
Retrieve relevant chunks from documents for this thread.
Search the web using DuckDuckGo
Tool descriptions are critically short (10-40 chars), below the 34-char baseline minimum. 'Retrieve current stock price information' and 'Fetch Facebook profile for the connected account' lack context on WHEN to use the tool, what data is returned, and any prerequisites (e.g., authentication state).
No output schemas documented. Tools like rag_tool, search_tool, and the profile tools do not expose what fields they return. LLMs cannot plan downstream tool chains without knowing which IDs (e.g., user_id, profile_id) are available for passing to other tools.
Profile tools (facebook_profile_tool, google_profile_tool, twitter_profile_tool, github_repos_tool) accept empty input schemas. No documentation of required authentication, connection state, or what happens if no account is connected. Error paths are not described.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 40 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Fetch Twitter profile for the connected account.
search_tool and rag_tool both accept a query string but do not specify limits, pagination, or result count.
No parameter descriptions visible for input schemas. The 'user_name' parameter in get_gender_of_given_name has a description ('First name to classify'), but other tools like search_tool show only type ('string') without format expectations (min/max length, required format). LLMs cannot infer whether queries should be keywords, full names, or compound phrases.
No error handling or recovery guidance. If a profile fetch fails (e.g., account not connected), no tool tells the LLM what to do next (retry, configure auth, or abandon). No categorization of retryable vs. fatal errors.
Tool composition risk: search_tool, rag_tool, and get_stock_price are generic utilities, while facebook_profile_tool, google_profile_tool, github_repos_tool, twitter_profile_tool are identity-specific. No documentation of which tools can be chained or what data flows between them. The API layer manages user permission (user_tool_status.py) but this is not reflected in tool descriptions, agents do not know which tools they are allowed to call.
Security: No tool descriptions document what credentials or permissions are required. Profile tools imply OAuth/connection state but do not explain how to authenticate or what scopes are needed. No mention of whether tokens are exposed in responses.