Loading ToolBench
MCP server for retrieving stock market data from Polygon.io, reading user portfolio transactions, and analyzing stock performance
The server defines 4 tools with schemas present and some descriptions, but several critical quality gaps reduce the overall score. Tool naming is adequate (verb-based), but descriptions vary in quality. Two tools (read_user_exchanges_data, portfolio_activity_summary) have descriptions that are too generic or vague. Parameter descriptions are present for tools 1-3 but lack actionable context about constraints and error cases. No error handling guidance is documented. Output schemas are partially specified but lack pagination support. The schema for get_stock_value and get_last_closing_stock_price are well-formed JSON Schema with proper type constraints, but portfolio_activity_summary lacks input schema entirely.
Returns the last recorded closing price for a given stock symbol using Polygon.io. Args: stock (StockBase): A validated Pydantic model containing: - symbol (str): Stock ticker (e.g. 'AAPL') Returns: List[float]: The last closing price wrapped in a list (e.g. [220.15]).
Retrieve daily stock data from Polygon.io for a given symbol and date range. Args: stock (StockInfoByDate): A validated Pydantic model containing: - symbol (str): Stock ticker (e.g. 'AAPL') - from_date (date): Start date in YYYY-MM-DD format - to_date (date): End date in YYYY-MM-DD format Returns: List[dict]: A list of dictionaries representing the stock's daily information.
Guides the model to analyze and summarize the user's portfolio performance
Reads the user stock exchange transaction data from a local CSV file. Returns: str: The raw contents of the CSV file. If the file is empty, returns a message: "no records were found in stock_trades file". If an error occurs, returns the exception message.
portfolio_activity_summary has no input schema at all (Input: {}). Per hard scoring rules, schema score MUST be 0. Additionally, description 'Guides the model to analyze and summarize the user's portfolio performance' (73 chars) is vague, it does not explain WHAT the tool returns, WHEN to call it, or what backend action it triggers. Below 100-char minimum for actionability.
read_user_exchanges_data lacks meaningful input schema (Input: {}) and description does not explain when to call this vs other portfolio tools. The description mentions CSV file format and error cases ('no records were found', 'returns exception message') but does not specify the file path, encoding, expected column structure, or refresh behavior. Tool is discovery-critical but underdocumented.
No error handling guidance across any tool. Descriptions do not indicate what errors are retryable (e.g., 'API rate limit, try again in 60s') vs terminal (e.g., 'Invalid symbol, use search_symbols to find valid tickers'). Stock API calls can fail for invalid symbols, missing API key, network timeouts, but callers have no recovery path documented.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 52 | - | v1 |
Output schemas not documented. Tools return 'List[dict]' or 'List[float]' but LLMs do not know the structure of each dict or how to chain results to downstream tools. For example, get_stock_value returns 'A list of dictionaries', which keys? date, close, open, high, low, volume? Without documented output schema, LLM cannot plan multi-step queries or extract correct fields.
No pagination support. If user has 5 years of daily stock data (1250+ rows), does get_stock_value return all of them, or truncated at some limit? No limit parameter, no next_cursor, no offset/page documented. Returning large result sets can exhaust context window without pagination guidance.
Parameters lack actionable constraints. For get_stock_value, symbol has minLength=1, maxLength=10, which is good, but description examples 'AAPL', 'MSFT' could mislead LLM into thinking only uppercase is valid, or only 4-5 chars. No note that symbol is case-insensitive or that it must match Polygon.io valid symbols. Description should say: 'Stock ticker symbol (e.g., AAPL, MSFT). Case-insensitive. Use get_valid_symbols() first if unsure.'
Tool composition gap: no tool to search or validate stock symbols. get_stock_value and get_last_closing_stock_price accept symbol but have no guardrail. If LLM passes an invalid symbol (typo or hallucination), tools fail with no recovery path documented. Need a 'search_valid_symbols' or 'validate_symbol' tool, or error messages must guide to it.
Date range constraints not validated in descriptions. get_stock_value accepts from_date and to_date but descriptions do not specify: is from_date required to be ≤ to_date? Can you query future dates? Can you query market-closed days? No guidance means LLM may issue invalid requests.