Loading ToolBench
Local runtime evidence and debugging for coding agents
jolink-runtime provides 6 tools with comprehensive schemas and detailed descriptions. java_runtime and java_application have exceptionally long, LLM-optimized descriptions (500+ chars) that explain when to use them, prerequisites, and safe usage patterns. All tools have proper JSON Schema with typed parameters and enums. However, several tools lack descriptions entirely (java_fast_test, java_status, java_debugger, java_processes have minimal or generic descriptions under 100 chars), and output schemas are not documented. Error handling guidance is absent from tool descriptions. Tool naming is clear and verb-based (run, attach, debug), but composition could be tighter, java_application and java_runtime overlap significantly in functionality.
Run, observe, and debug a local Java application. Launch directly or from a supported IntelliJ IDEA Maven/Gradle project, restart the target, inspect status and logs, and verify code changes against actual runtime behavior before making further assumptions. When repeated edits fail or a fix needs verification, obtain runtime evidence before applying another patch. For deeper investigation, attach to an existing JVM, set breakpoints or exception watches, inspect stack frames and variables, and resume suspended threads. For an HTTP application launched by run/restart, provide ready_port; if startup_state is starting, call status until TCP readiness is observed before arming an HTTP trigger. After editing existing Java method bodies in a project_path launch, update accepts a background reload Attempt and returns reload_started with a reload_id; call status until last_reload is terminal, then verify with a fresh request. Treat runtime outputs as bounded observations; separate observed facts from interpretations and unverified conclusions. wait_event blocking can arm JDWP event requests, start an optional local HTTP request only after arming, and await an event in one call; use arm then await when an external action must occur between them. Always resume a suspended JVM or call cleanup_debug_state after inspection.
Attach to an existing JVM, set breakpoints or exception watches, inspect stack frames and variables, and resume suspended threads. Use for deeper investigation of running Java applications.
Run selected Java tests without an application launch. Supports Maven/Gradle projects and JUnit/TestNG tests.
List and query local Java processes for debugging and attachment.
Minimal descriptions for 4 of 6 tools. java_fast_test, java_status, java_debugger, and java_processes have descriptions under 100 chars, providing insufficient context for LLM tool selection. Baseline for A-grade tools is 194 chars average.
No output schemas documented. Tools return complex nested structures (breakpoint_id, suspension_id, stack frames, variables) but responses are not formally specified. LLMs cannot plan downstream calls or extract required fields without documented return types.
Functional overlap between java_application and java_runtime. Both support run, stop, restart, attach, detach, status, logs, breakpoint, exception, wait_event, threads, stack, variables, resume, cleanup_debug_state. This violates single-responsibility and forces LLMs to reason about which to use.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2025-06-18+ | v2 |
An LLM-facing Java debugger for observing and controlling a running Java application using real execution-time evidence. Use this tool to investigate bugs, unfamiliar code, actual execution paths, exceptions, call stacks, variables, object state, data flow, runtime configuration, permission context, and application behavior when source code, logs, tests, or error messages are insufficient. Before performing a runtime investigation with breakpoints, exception watches, event waiting, threads, stack frames, variables, object inspection, resume, or debug-state cleanup, first load skill_view("java-runtime:observation") unless that skill has already been loaded in the current context. The skill defines evidence boundaries, suspension lifecycles, investigation principles, and safe usage rules. Lifecycle, connection, and monitoring actions such as run, stop, restart, attach, detach, status, and logs do not require loading the skill. Choose and combine Runtime actions dynamically according to the current task; no fixed debugging workflow is required. Stack frames, variables, and object references remain valid only while the corresponding suspension and debug connection remain active. Do not reuse them after resume, cleanup_debug_state, detach, stop, restart, process replacement, or connection reset. Do not leave application threads suspended unnecessarily. Variable entries use value_state='observed' for successfully read values, including Java null; value_state='partial' when a structured value was only partly read; and value_state='unavailable' when no reliable value could be read. For partial or unavailable values, inspect the error on the variable entry or inside its structured value. Object inspection is shallow by default to avoid flooding results; use include_this=true or increase max_value_depth only when relevant. Read error_code, retryable, warnings, nearby_locations, suggested_next_step, next_action, and suggestions when deciding how to continue or recover. Use the exact identifiers returned by the Runtime: breakpoint_id for breakpoints, request_id for exception watches, and suspension_id for active suspensions. These identifiers are not interchangeable. The model does not need to understand JDWP or other protocol internals.
Observe process, test, state, and log information for the Java runtime.
Error handling guidance absent. Tool descriptions do not explain what to do on failure (retryable vs fatal), what error codes mean, or how to recover. Baseline A+ tools include recovery hints like 'Try search_users() if not found.'
Parameter descriptions lack constraint details. Many numeric params (timeout, tail, frame_index, item_limit) have ranges but descriptions do not explain why or what happens if violated. E.g., 'timeout: Seconds to wait' should be 'timeout: Seconds to wait (0.1 - 300; longer waits block the agent).'