Loading ToolBench
A multi-server MCP client and agent framework that connects to local MCP servers (dataflow, weather) and external MCP servers (git) to enable a data science assistant agent named Scout
This server has 5 tools with basic structure but significant quality gaps. All tools have input schemas with type definitions and non-empty descriptions, meeting the minimum bar. However, descriptions are terse (averaging 75 chars, below the 194-char production baseline), lack dependency guidance, and omit error recovery hints. Parameters lack validation constraints (e.g., state codes for get_alerts, lat/long bounds for get_forecast). Tool naming follows verb_noun convention well (get_*, create_*, load_*, query_*), but parameter descriptions are minimal. No output schemas documented. The dataflow_create_new_project tool is concerning, it's a WRITE operation with no confirmation/dry-run pattern, no idempotency guidance, and minimal error documentation. Overall, the server is functional but would not pass production code review due to sparse descriptions, missing output documentation, and lack of error recovery guidance for LLM agents.
Create a new project. This will create a new directory with the project name and initialize a git repository.
Load data from a file into the session.
Query the loaded data. The data must first be loaded using the dataflow_load_data tool. The data is in the table `data`.
Get weather alerts for a US state.
Get weather forecast for a location.
Destructive operation (dataflow_create_new_project) lacks confirmation or dry-run pattern. No guidance on idempotency, error recovery, or what happens if directory already exists. LLM agents risk unintended project creation.
All descriptions are under 100 characters and lack context. 'Query the loaded data' does not explain when to call this vs dataflow_load_data, what SQL dialect is supported, result limits, or error handling. Production baseline is 194 chars.
get_alerts parameter 'state' accepts free-form strings with no enum constraint. Description says 'Two-letter US state code (e.g. CA, NY)' but does not enforce it. LLM agents may pass invalid codes like 'California' or 'cali', causing silent failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 36 | - | v1 |
get_forecast parameters 'latitude' and 'longitude' have no range constraints. LLM agents may pass invalid coordinates (e.g., lat=200, lon=400) or forget to convert from degrees-minutes-seconds format, causing API errors with no guidance.
No output schemas documented for any tool. LLM agents cannot plan downstream operations or extract required fields. E.g., does get_forecast return temperature, wind speed, precipitation chance? Do alerts include severity or affected counties?
dataflow_query_data has no pagination support and does not specify result limits. If the loaded data is large, returning all rows could exhaust context window and cause reasoning degradation. Production baseline limits results to 20-50 items.
No error recovery guidance. Descriptions do not mention what to do if file_path is invalid, SQL syntax is wrong, directory already exists, or API rate limits hit. LLM agents have no recovery strategy.
dataflow_load_data and dataflow_query_data descriptions reference a 'data' table but do not specify format (CSV, JSON, Parquet). LLM agents cannot infer what file types are supported.