Loading ToolBench
Unified MCP Server Observatory - a tool discovery, security scanning, and observability platform for MCP servers
Tooldex is a discovery/observatory tool with 11 tools spanning server management, trust decisions, rescanning, and LLM-based security analysis. Most tools have descriptions (9/11), but parameter schemas are inconsistently documented. Tool names follow verb_noun convention (list_, GET, POST, DELETE). However, parameter descriptions are sparse or missing entirely for many tools, and output schemas are not documented. The two sync wrapper functions (list_tools_for, list_tools_for_all) have good descriptions but lack explicit input schema visibility in the source. HTTP endpoints (GET /api/servers, POST /api/servers/{id}/trust, etc.) are present but their parameter schemas and response structures are not visible in the provided code. This forces schema scores to 0 for tools where definitions cannot be verified.
Revoke a prior approve/deny decision, returning the server to 'pending'. Clears previously discovered tools and security data.
List all MCP servers with their metadata, tool counts, connection status, and security information
Get detailed information about a specific MCP server
Poll progress of a running (or just-finished) LLM-judge job for this server.
Start the LLM-as-judge analyzer for one server as a background job. Poll llm-scan/status for progress.
Remove all cached LLM-judge verdicts for this server and reset its displayed LLM scan state back to 'never scanned'.
Signal a running LLM-judge job to stop.
Output schemas not documented. No visible response structure definitions for any HTTP endpoint or tool. LLMs cannot plan downstream calls or extract required fields without knowing what fields are returned.
Parameter schemas incomplete or invisible. Most HTTP endpoints show parameter names and descriptions in the spec, but JSON Schema type definitions are not visible in source code. Cannot verify if types, enums, defaults, and constraints are properly declared.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Re-probe a single server, update its discovered tools, and re-run its YARA security scan. Blocked while an LLM-judge scan is running unless force=true.
Approve or deny a stdio server's ability to be spawned/probed. Records the trust decision.
Sync wrapper: probe one server and return its tool discovery result. Safe to call from sync code.
Sync wrapper: probe many servers concurrently. Returns results in the same order as the input list. Supports caching of results.
Error handling guidance missing. No visible error responses, recovery hints, or actionable error messages. Tools like rescan and llm-scan can fail (e.g., timeout, invalid server_id) but error structure is not documented.
Tool definitions inferred from spec, not directly visible in source. The 9 HTTP endpoints are described in the spec but their actual FastAPI route handlers, Pydantic models, and response schemas are not shown in the provided code. Cannot verify schema completeness or error handling implementation.
Destructive operations lack confirmation or dry-run. POST /api/servers/{server_id}/trust (allow/deny), DELETE /api/servers/{server_id}/trust (revoke), and POST /api/servers/{server_id}/llm-scan/invalidate-cache (clear cache) are write operations that modify state. No visible confirmation step or dry-run mode to prevent accidental misuse.