Loading ToolBench
Java observability agent providing MCP tools for monitoring application performance, logs, and behavior via method instrumentation and tracing
obs-agent provides 10 tools with basic schemas and descriptions, but falls short of production quality. All tools have descriptions (good), but most descriptions lack actionable detail for LLM selection. Parameter descriptions are entirely absent, the schema shows property names and types but no guidance on what each parameter controls or its valid range. No output schemas are documented. Error handling is minimal; tools return generic JSON responses without recovery guidance. The two-phase watch/trace pattern (call tool, generate traffic, call results) is unusual and requires explicit agent coordination that is not well-documented in parameter descriptions. No tool annotations (readOnlyHint, destructiveHint) despite all tools being read-only. Schemas are minimal (type + property names only; no descriptions, enums, or constraints).
Known routes with current health.
Recent error requests.
Recent request metrics for a route (in-JVM ring buffer).
Search in-JVM captured logs.
Slowest recent calls.
Top threads by CPU + deadlocks.
START capturing a method call tree (Arthas-trace) for classes whose name matches a regex. Returns IMMEDIATELY; the trace stays active for ttl_seconds (default 10). Two-phase: after calling this, GENERATE TRAFFIC (hit the endpoint, or wait for requests), THEN call trace_results to read the trees. Best with a SPECIFIC class (e.g. the service/controller FQCN) - broad patterns may miss cross-class callees. Instrumentation tier.
No parameter descriptions in any tool. Schemas define property names and types only; LLMs cannot infer what 'limit', 'ttl_seconds', 'window_seconds', 'top', 'level', 'traceId' control or their valid ranges.
No output schemas documented. Tools return JSON objects (captures, trees, metrics, etc.) but LLMs have no schema to understand the response structure, field names, or types. This forces LLMs to guess at downstream field names.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Read the call trees captured by a prior trace_request call.
START capturing a method's invocations (args/return/duration/exception). Returns IMMEDIATELY; the watch stays active for ttl_seconds (default 10). Two-phase: after calling this, GENERATE TRAFFIC (hit the endpoint, or wait for requests), THEN call watch_results to read captures. Instrumentation tier.
Read the captures collected by a prior watch_method call (same class+method).
Two-phase watch/trace pattern (call watch_method, generate traffic, call watch_results) requires explicit agent coordination. Descriptions mention this but do not explain how an agent should orchestrate the steps or detect when traffic has been generated.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 10 tools are read-only but this is not declared in the tool definition, forcing LLMs to infer safety from descriptions.
Error handling is minimal. Tools return generic JSON with 'error' or 'status' fields but do not guide LLMs on recovery (e.g., 'permission denied, contact admin' vs 'invalid class name, use get_routes to list available classes').