Loading ToolBench
FastAPI control tower for governing AI agents, MCP servers, runtime policies, and execution traces.
Server has 14 well-structured tools with consistent naming (verb_noun pattern) and complete input schemas. Descriptions are present and actionable (avg ~120 chars), exceeding the 10-char minimum. However, parameter descriptions are sparse or missing in several tools (e.g., 'decision' in approval_decision lacks enum values; 'task_summary_suffix' in replay_run lacks length validation detail). Output schemas are documented via response_model declarations. Error handling is present but generic (HTTPException with status codes, no recovery guidance). Security is strong (API key auth, permission checks visible). Composition is clean, each tool has one responsibility. Main gaps: parameter constraints not fully specified in descriptions, error messages lack actionable recovery hints, and some tools (health, me, organizations) lack pagination despite potentially returning large lists.
Lists agents accessible to the authenticated user's organization.
Submits an approval decision (approved or rejected) for a pending approval, updating the associated run status.
Lists pending approvals for the authenticated user's organization.
Returns health snapshot of the control tower service including database backend, state backend, and counts of organizations, agents, MCP servers, runs, pending approvals, and incidents.
Lists security incidents for the authenticated user's organization.
Lists MCP servers accessible to the authenticated user's organization.
Parameter 'decision' in approval_decision lacks enum constraint and validation detail. Description should specify: 'Decision must be one of: approved, rejected'.
List tools (organizations, mcp_servers, agents, policies, runs, approvals, incidents) lack pagination parameters (limit, offset/cursor) and total count in response. Large result sets will exhaust context windows.
Error responses use generic HTTPException with status codes but no recovery guidance. E.g., 404 for run_traces should suggest 'Run not found. Call runs() to list available runs.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Returns the current authenticated user information including id, name, organization_id, and role.
Lists organizations accessible to the authenticated user. Platform admins see all organizations; other users see only their own organization.
Lists runtime policies accessible to the authenticated user's organization.
Creates a new agent run by replaying an existing run with optional overrides for policy, MCP server, and task summary.
Requests a new agent run with specified agent, policy, MCP server, and task parameters. Evaluates risk factors and may require approval.
Retrieves the current runtime state of a specific agent run, including status, approval information, and cost/tool call metrics.
Retrieves tool call traces for a specific agent run, showing which tools were called and their outcomes.
Lists agent runs accessible to the authenticated user's organization.
Parameter descriptions in replay_run lack specificity. 'task_summary_suffix' should state: 'Optional suffix (max 80 chars) appended to original task summary. If omitted, original summary is used.'
Destructive operations (request_run, replay_run, approval_decision with 'rejected') lack confirmation/dry-run pattern. Agents should be able to preview impact before committing.