Loading ToolBench
A collection of MCP servers demonstrating various integrations: FastAPI integration, Math tools, RAG workflow, and Yahoo Finance data
This collection exhibits significant quality gaps across all nine tools. While tool names follow verb_noun conventions and basic schemas are present, critical issues include: incomplete or missing parameter descriptions (5 of 9 tools lack descriptions on all or most parameters), missing or trivial tool descriptions (3 tools have zero descriptions), no output schema documentation, no error handling guidance, and no input validation constraints. The code shows tool registration via FastMCP decorators with type hints, but schema quality is minimal. No pagination, no structured return types, and responses are raw JSON strings without field documentation. This is representative of early-stage experimental servers, not production-ready tooling.
Add two numbers and return the result.
Summarize key financial metrics and analysis for a given ticker.
Multiply two numbers and return the result.
Use RAG workflow to answer queries about Deep Seek
Simulate news search using a ticker and return top articles.
Three tools (get_ticker_info, get_ticker_news, analyze_ticker) lack any description whatsoever. LLMs cannot determine when to select these tools or why they differ from similar tools. Violates pattern:tool-description requiring non-empty descriptions.
All nine tools lack documented output schemas. Responses are raw JSON strings with no field definitions. LLMs cannot plan downstream tool calls or extract typed fields. Violates pattern:tool requiring schema documentation.
Five tools (get_ticker_info, get_ticker_news, web_search, rag, and others) have parameters with NO descriptions. The 'ticker' and 'query' parameters lack context about expected format, constraints, or examples. Violates pattern:tool-description requiring all params described.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Return the current stock price for a ticker.
Search the web for the given query.
No error handling or recovery guidance in any tool. None specify what to do if a lookup fails, a service times out, or input is invalid. Violates pattern:recovery-guide requiring error-driven guidance.
No input validation or constraints. Parameters like 'ticker', 'query', and numeric 'a'/'b' accept any value. No enums, patterns, min/max, or format constraints prevent hallucinated or malformed input. Violates pattern:constrained-input.
Tool 'rag' description is vague: 'Use RAG workflow to answer queries about Deep Seek'. Hardcoded domain ('Deep Seek') reduces reusability. No mention of what RAG does, when to use it vs web_search, or what format it returns. Violates pattern:tool-description.
Responses are unstructured JSON strings (all tools return str, not typed objects). No field documentation. LLM must parse the string to extract ticker, price, news items, etc. Violates pattern:response-shaper requiring structured output.
Similar tools lack clear differentiation. 'search_quote' and 'search_news' both accept a ticker parameter but serve different purposes. No descriptions explain which to use when. 'get_ticker_info' vs 'analyze_ticker' both fetch ticker data but one has no description. Violates review:name-clarity.
Numeric tools (add, multiply) accept unbounded integers with no validation. No min/max constraints, no guards against overflow or abuse. Violates pattern:constrained-input.
No pagination support. Tools like search_news and web_search return fixed lists (top 5 articles) with no way to request more results or specify limits. Violates pattern:paginated-result for discovery/list tools.