Loading ToolBench
Semantic context routing platform with REST API and MCP adapter for AI agents
Contex exposes 16 tools via HTTP/FastAPI with reasonable schema coverage and descriptions. However, quality is uneven: audit tools have detailed descriptions (150-200 chars) with proper parameter types and ranges, but infrastructure tools (root, health, readiness, liveness, metrics) are minimally described stubs that add no value to an agent. API key and role management tools lack specificity in parameter descriptions and output documentation. Security-related descriptions mention permission requirements ('Requires admin role') but tools lack explicit permission annotations (readOnlyHint, destructiveHint). Parameter constraints are present for some tools (limit: 1-1000, days: 1-365) but missing explanations of what values mean (e.g., why is 365 the max for days?). Output schemas are not documented anywhere, the evaluation must infer structure from endpoint names. No error recovery guidance visible (e.g., 'If you lack admin permission, contact your administrator'). Composite operations like 'assign_role' accept arrays of project IDs but do not explain the difference between assigning to specific projects vs. tenant-wide scope.
Assign a role to an API key
Create a new API key
Export audit events for compliance reporting. Requires admin role. This endpoint returns a complete export of audit events for the specified time range. Use this for compliance reports, SIEM integration, or backups. If start_time is not provided, exports events from the last N days. Maximum export is 10,000 events.
Get a single audit event by ID. Requires admin role.
Get audit event summary statistics. Requires admin role. Returns counts of events by type and severity for the specified time range.
Get current rate limit status for the authenticated API key
Infrastructure tools (root, health, readiness, liveness, metrics) are operational endpoints not agent-usable tools. They should not be exposed as MCP tools, they belong in Kubernetes or monitoring systems. Exposing them wastes agent reasoning cycles and contaminates the tool namespace.
Output schemas are not documented. query_audit_events, export_audit_events, and get_audit_summary return complex structures (paginated lists, summary statistics) but the schema visible to the LLM is empty. The agent cannot plan follow-up calls without knowing what fields are present.
Tool name 'assign_role_endpoint' violates verb_noun naming convention. Should be 'assign_role' (shorter, clearer). Suffix '_endpoint' is a code smell, it leaks HTTP routing abstractions into the agent interface.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
Comprehensive health check endpoint
List all available audit event types. Requires admin role.
List all API keys
List all role assignments
Liveness check for Kubernetes
Prometheus metrics endpoint
Query audit events with filtering. Requires admin role. Filter options: - tenant_id: Filter by specific tenant - actor_id: Filter by specific actor (API key) - event_type: Filter by event type (e.g., 'data.published', 'auth.login.success') - start_time: Start of time range - end_time: End of time range Returns paginated list of audit events, newest first.
Readiness check for Kubernetes
Revoke an API key
Root endpoint
Descriptions for key management tools (create_key, list_keys, revoke_key) do not explain use cases, permissions required, or consequences. 'Create a new API key' is a command, not a goal, when should an agent create a key? What happens if they revoke the wrong one? Revoke is marked reversible but no undo mechanism documented.
Tool definitions lack permission annotations (readOnlyHint, destructiveHint, idempotentHint). All audit tools state 'Requires admin role' in description text but do not use structured hint metadata. This prevents the agent from planning with permission constraints.
Parameter 'projects' in assign_role_endpoint is an array with no description of scope semantics. Does assigning to an empty array revoke access? Does omitting 'projects' grant access to all projects? Undocumented dependency violates parameter relationship rule.
Error handling guidance is missing. No tool description explains what to do if permission is denied, if a resource is not found, or if a rate limit is hit. Agents have no recovery path and will spiral on failures.
No pagination guidance in descriptions. query_audit_events accepts limit and offset but does not explain how to iterate through large result sets, whether there is a total count, or what the maximum safe limit is.