Loading ToolBench
Team-ready local IDE/LLM/MCP insights dashboard and configuration manager for detecting and managing Model Context Protocol servers across multiple IDE and LLM applications
MCPStackStudio exposes 16 tools via FastAPI HTTP transport with reasonable naming conventions and parameter schemas. However, critical gaps exist: most tool descriptions are adequate (100-200 chars) but lack specificity on when to use each tool versus alternatives. Parameter descriptions are present but often generic. Output schemas are not documented, callers must infer them from API behavior. Error handling is minimal with no recovery guidance. The server lacks error classification, actionable error messages, and documentation of what fields are returned by each tool. Tools follow verb_noun naming (get_*, add_*, update_*, delete_) which is good, but compositions like add_mcp_server + update_mcp_server + delete_mcp_server lack documented dependency chains and parameter relationships. Schema definitions visible in the code show proper JSON Schema with types and descriptions for input parameters, but no output schemas are defined. Risk annotations (READ_ONLY, WRITE, DESTRUCTIVE) are present in metadata but not exposed as tool annotations in the MCP protocol.
Add a new MCP server configuration to a target file
Delete an MCP server configuration
Retrieve all detected IDEs, LLMs, and MCP configuration data with optional refresh
Retrieve scan history with optional limit
Retrieve list of detected IDE applications and editors
Query detected issues filtered by scan ID, severity, type, and source
Retrieve list of detected LLM applications, desktop apps, and CLI tools
No output schemas documented. Tools return data (scan results, MCP configs, issues) but the MCP definitions do not specify the structure, field names, or types of returned objects. This forces LLMs to infer output structure, risking extraction errors and broken downstream tool chains.
scan_now_get and scan_now_post duplicate functionality. The naming ('scan_now_get' vs 'scan_now_post') violates the principle that tools should be distinguished by purpose, not HTTP verb. LLMs will be confused about when to call which variant.
Tool descriptions lack specificity on when to use each tool vs alternatives. For example, get_all, get_ides, get_llms, and get_mcp all retrieve detection data, but the descriptions do not explain when to call get_all vs individual getters. This forces LLM reasoning overhead and risks wrong tool selection.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 8 | - | v1 |
Retrieve list of detected MCP servers and their configurations
List available MCP configuration targets and their properties
Retrieve new MCP servers from the registry catalog
Health check endpoint for server status
Export scan report in JSON format
Export scan report in Markdown format
Trigger a system scan to detect IDEs, LLMs, and MCP configurations (GET method)
Trigger a system scan to detect IDEs, LLMs, and MCP configurations (POST method)
Update an existing MCP server configuration
Error handling missing. No tool documents error cases, recovery paths, or actionable error messages. When add_mcp_server fails (e.g. invalid config file, bad permissions), the LLM gets no guidance on what to do next.
No dry-run or confirmation step for destructive operations. delete_mcp_server performs irreversible deletion with no confirmation or preview capability. Agents have no way to safeguard against accidental deletion.
Parameter relationships undocumented. add_mcp_server and update_mcp_server have optional target_key parameter with enum values, but the description does not explain when auto-detection works vs when manual specification is required. This invites misuse.
Pagination not implemented. get_new_mcp_catalog accepts limit and cursor, but no tools document total_count or whether results are paginated. get_issues and get_history accept limit parameters but do not specify offsets, cursors, or how to fetch the next page.
Tool annotations missing in MCP protocol. Risk metadata (READ_ONLY, WRITE, DESTRUCTIVE) is tracked internally but not exposed via MCP toolAnnotations (readOnlyHint, destructiveHint, idempotentHint). This prevents clients from applying safety logic.