Loading ToolBench
Four READ_ONLY tools with basic functionality and naming aligned to verb conventions (get_*). Descriptions are present but minimal (15-250 chars, averaging ~120 chars). Parameters have Annotated type hints with descriptions embedded in the annotation strings, which are visible in source but not formally exposed in JSON Schema structure. No explicit schema objects are visible in the provided code excerpt, tool registration happens via register_tool() call that is cut off. Input schemas are inferred from function signatures with Annotated hints rather than explicitly documented. Error handling is minimal; no recovery guidance or error categorization. All tools are read-only (no state mutation), which reduces risk but also limits functionality. Output schema (JSONType) is not documented. Security posture adequate for read-only tools but lacks explicit scope declarations or audit logging.
Get the current datetime in ISO format with Eastern Time offset and abbreviation.
Search for instruments by symbol or description. Params: symbol (search term), projection (SYMBOL_SEARCH/SYMBOL_REGEX/etc., default symbol-search). Examples: get_instruments("AAPL"), get_instruments("AAPL .*", "symbol-regex"), get_instruments("AAPL", "fundamental").
Get market hours for specified markets (EQUITY, OPTION, etc.) on a given date (YYYY-MM-DD, default today).
Get top 10 movers for an index/market (e.g., DJI, SPX, NASDAQ). Params: index, sort (VOLUME/TRADES/PERCENT_CHANGE_UP/DOWN), frequency (min % change: ZERO/ONE/etc.).
No explicit JSON Schema objects visible in tool registration. Tool input definitions are inferred from Python function signatures with Annotated hints, not exposed as formal io.modelcontextprotocol/tools/Tool.input schemas. This violates DEFINITION QUALITY / SCHEMAS pattern and makes it impossible for non-Python MCP clients to validate inputs or introspect constraints programmatically.
Output schemas (return type JSONType) are completely undocumented. LLMs cannot infer what fields to expect from each tool's response, making it impossible to chain results to downstream tool calls or extract relevant data. For example, get_market_hours returns 'JSONType' with no indication of whether response includes 'isOpen', 'opens_at', 'closes_at', etc.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 59 | 1.17.0+ | v1 |
No error handling or recovery guidance. Tools call .upper() on user input (e.g., markets, index, sort, frequency, projection) without validation. If an LLM passes an invalid market name or index, the KeyError will bubble up with no actionable message. Descriptions mention valid enums (e.g., 'EQUITY, OPTION, BOND') but code does not validate or provide recovery paths.
Enum constraints listed as text in descriptions rather than formalized in JSON Schema enums. E.g., 'Markets (list/str): EQUITY, OPTION, BOND, FUTURE, FOREX' is human-readable but machine-unvalidatable. This violates constrained-input pattern and forces LLMs to guess valid values, inviting hallucinated options.
Tool registration is incomplete in provided source. The register() function is cut off mid-call (register_tool(server, func, resul...). Cannot verify that descriptions, schemas, and input definitions are correctly passed to the MCP server registration layer. Tool definition quality cannot be fully assessed without seeing complete registration.
Parameter descriptions include examples in enum lists but violate the 'do not include example values in descriptions' guideline. E.g., 'EQUITY, OPTION, BOND' are valid enums, not examples, but LLMs may treat the first listed item (EQUITY) as the default or most common choice, biasing their usage.