Loading ToolBench
A comprehensive observability and monitoring platform that provides log intelligence, error tracking, health checks, database introspection, and connector management through an MCP server interface
OpenTrace exposes 7 tools with moderate structural definition but significant gaps in LLM-facing documentation. Tool names lack clear action verbs (e.g., 'connectors', 'database', 'logs' are nouns, not verb_noun patterns). Parameter descriptions exist but are sparse (average ~40 chars, well below the 72-char production baseline). Input schemas are present but incomplete, many parameters lack type information or are described only in English text without formal enum/pattern constraints. Output schemas are entirely undocumented. Critical security issues: database and connector tools expose destructive operations without documented permission gates or confirmation patterns. The server uses HTTP transport (good) but shows no evidence of current MCP spec patterns like tool annotations, structured error recovery guidance, or idempotency markers. The codebase includes comprehensive test coverage (tool_args_test.go) but tests focus on validation coercion, not agent-facing schema quality.
Connector management: list, get, create, test, update, delete
Database introspection and management
Error management: list, detail, investigate, impact, user_errors, ranking, resolve, ignore, reopen, new
Health check management: list, uptime, create, delete
Log intelligence: search, context, attributes, stats, summary, performance, trace, compare
System overview, catch-up since your last visit, triage, diagnosis, incident timeline, and agent memory
Tool names are nouns, not action verbs. LLMs cannot infer what action the tool performs from 'connectors', 'database', 'logs', 'errors', 'healthchecks', 'overview', 'users' alone. Should be 'list_connectors', 'query_database', 'search_logs', 'list_errors', etc.
No output schemas documented. The server definition lists tool names and input parameters but does not specify what fields or structure the LLM should expect in responses. This forces LLMs to guess field names and types for downstream tool calls.
Destructive tools (connectors: create/update/delete, database: kill_query, healthchecks: delete) lack confirmation or dry-run support. No evidence of permission gates or audit logging. An agent could delete all connectors without a confirmation step.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 51 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Support view: what one customer actually hit
Parameter descriptions are minimal. Example: 'order_by' in database tool described as 'Sort: calls, total_exec_time, mean_exec_time', this is an enum expressed as free text, not a formal constraint. LLMs may hallucinate other sort fields.
No error recovery guidance. Tools do not document what errors are retryable, what the LLM should do if a lookup fails, or how to resolve common issues (e.g., 'If connector not found, call list_connectors() with type filter').
'users' tool has empty input schema ({}). No parameters documented, yet the description hints it's a 'Support view: what one customer actually hit', unclear how an LLM invokes it or filters results.
Parameter names use generic terms without type suffixes. 'id' appears in multiple tools without clarification (connector_id, healthcheck_id?). 'query' is overloaded across logs and database tools, free-text search vs SQL explain query.
No evidence of tool annotations (readOnlyHint, destructiveHint, idempotentHint) in source. Tools with write risk ('WRITE', 'DESTRUCTIVE') are labeled in the spec but not in the MCP tool definitions themselves, so the client cannot use annotations for safety.