Loading ToolBench
A web application and MCP server that matches datasets with compatible tools in the EOSC (European Open Science Cloud) data commons. It provides dataset search, tool matching, and a dataplayer interface for executing tools on datasets.
This server has critical gaps in definition quality. Tool descriptions are present but generic (most under 100 chars); parameter schemas are minimal, with many lacking type definitions and descriptions. Most tools expose only 1-2 parameters with sparse constraint documentation. No evidence of input validation guidance, error recovery patterns, or output schema documentation. Tool naming follows verb_noun convention (good), but parameter naming is inconsistent and lacks clarity. No evidence of pagination, idempotency guidance, or composition patterns. The codebase appears to be a React frontend with server.ts handlers, but the actual tool implementations are not visible in the provided source, making most assessments inference-based. Several tools (launch_tool, monitor_task_status, get_task_result) have no schema visible in the source snippets, forcing a conservative score.
Retrieve the list of files in a dataset
Retrieve the execution result and artifact URL for a completed tool task
Retrieve the configuration and input slots for a specific tool
Launch a tool execution task with dataset files and parameters mapped to tool input slots
Match tools compatible with provided dataset files based on file formats
Stream real-time status updates for a running tool execution task via Server-Sent Events
Search datasets in the EOSC data commons
Search for compatible tools in the tool registry
Critical: Parameter schemas severely underdescribed. Only 'query' (search_data, search_tools) and 'datasetUrl' (get_dataset_files) have explicit descriptions; 'files' (match_tools_by_data) and 'userInfo' (launch_tool) lack any description. LLM cannot infer what data structure to pass.
Critical: No output schemas documented for any tool. LLM cannot plan downstream operations (e.g., what fields does search_data return? What structure does match_tools_by_data produce?). Without documented output, chaining tools requires reverse-engineering.
High: launch_tool has 6 parameters (userInfo, toolId, datasetUrl, datasetTitle, slotMapping, files) but no parameter descriptions. 'slotMapping' and 'files' are opaque, LLM cannot construct valid input. No guidance on expected object shapes, required fields, or constraints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
High: No error handling guidance. Tools have no documented recovery paths (e.g., what if a dataset URL is invalid? What if a tool is not found?). LLM will not know whether to retry, ask user, or fail gracefully.
High: Parameter types are minimal. 'query', 'datasetUrl', 'taskId', 'toolId' are strings, but no enums, length constraints, format rules, or validation patterns. LLM can pass any string, no safeguards against malformed input.
Medium: Tool descriptions are generic and lack context. 'Search datasets in the EOSC data commons' (search_data) does not explain: What filters apply? What does the response include? When should this be called vs. match_tools_by_data? No LLM-optimized guidance.
Medium: 'match_tools_by_data' and 'get_tool_config' imply a complex multi-step workflow (search → match → configure → launch), but tool descriptions do not guide sequencing or explain dependencies. LLM must infer the flow.
Medium: 'launch_tool' is a destructive operation (initiates tool execution), but description does not say so. No guidance on idempotency, dry-run, or confirmation. LLM may not realize this is irreversible.
Low: Tool names are clear but inconsistent in specificity. 'search_data' vs. 'search_tools' are balanced, but 'match_tools_by_data' is longer. No obvious naming conflicts, but 'get_task_result' could be confused with 'monitor_task_status' if both are called near each other.