Loading ToolBench
This server has moderate definition quality. All 6 tools are registered with names, descriptions, and input schemas using Zod validation (which map to JSON Schema). Most tool descriptions are adequate (100-300 chars), falling within production baselines (avg 194 chars). However, there are systematic gaps: (1) Output schemas are not documented, tools return text content without specifying the structure agents should expect; (2) Parameter descriptions exist but are sometimes vague (e.g., 'Path to the .hprof heap dump file' doesn't specify absolute vs relative, what happens if missing, etc.); (3) No error guidance, when a tool fails, responses are simple text with no recovery hints for the LLM; (4) No pagination for list_* tools despite domain patterns suggesting large result sets; (5) Tool descriptions lack WHEN/WHY guidance, they explain WHAT the tool does but not when an agent should choose it over alternatives. Naming is strong (all verb_noun, action-oriented). Security is reasonable for this domain (no credentials exposed as parameters).
Run Eclipse MAT analysis on a heap dump file. Generates reports including leak suspects, memory overview, and top memory consumers.
Download a heap dump from a running Kubernetes pod. Executes jmap inside the pod to generate an HPROF heap dump, then copies it to the local machine for analysis.
Get a text summary of an analysis report suitable for AI review. Extracts key findings from HTML reports into a readable format.
List all downloaded heap dumps available for analysis.
List all generated analysis reports. Reports can be opened in a browser for human review.
Execute an OQL (Object Query Language) query against a heap dump. OQL is SQL-like syntax where classes are tables, objects are rows, fields are columns. Basic syntax: SELECT * FROM <class> [WHERE <condition>] Examples: - SELECT * FROM java.lang.String (all strings) - SELECT * FROM java.lang.String s WHERE s.@retainedHeapSize > 10000 (large strings) - SELECT * FROM java.lang.Thread (all threads) - SELECT * FROM java.util.HashMap WHERE size > 100 (large hash maps) Special attributes: - @retainedHeapSize: memory retained by this object - @usedHeapSize: shallow heap size - @length: array length Common useful queries available as presets (use query_preset parameter).
Output schemas not documented. Tools return text responses via 'content' field, but the LLM cannot determine what data structure to expect (e.g., does list_heap_dumps return JSON embedded in text, plain text table, or structured list?). This forces the LLM to parse unstructured output.
list_heap_dumps and list_reports lack pagination, limit parameters, and total counts. In production, heap dumps and reports can accumulate; returning unbounded lists risks context window exhaustion and delays agent reasoning.
Error responses are unstructured text with no recovery guidance. When a tool fails (e.g., 'Query failed: error message'), the LLM receives no hint about whether to retry, call a discovery tool first, or fix input. Errors should be categorized (retryable, user-fixable, fatal) with actionable next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 54 | - | v1 |
Parameter descriptions lack constraint details. 'Path to the .hprof heap dump file' does not specify: absolute vs relative paths, what error if file missing, max size, or expected file format. LLMs will guess and fail.
Tool descriptions lack WHEN/WHY guidance. They state WHAT each tool does but not when to choose it over alternatives or what prerequisites exist. E.g., download_heap_dump's description doesn't mention that kubectl must be installed or the pod must be running Java.
Destructive operation (download_heap_dump) has no confirmation or dry-run mode. While less critical than delete operations, downloading large dumps can fail or be network-expensive without warning.