Loading ToolBench
MCP server for autonomous Java reverse engineering via LLM agents. Provides tools for bytecode manipulation, decompilation, assembly, call graph analysis, comment management, and scripting using Recaf's APIs.
This server demonstrates solid foundational work with clear, verb-based tool naming and consistent parameter documentation. However, several critical quality gaps prevent a higher score. Tool descriptions are generally adequate (average ~110 chars, within the 10-1024 baseline), but output schemas are undocumented, LLMs cannot predict return structures or plan downstream tool chains. Error handling lacks recovery guidance and categorization. Security considerations are partially addressed but lack explicit permission gates and audit trail specifications. The tool set is well-composed with minimal overlap, but the lack of output documentation and recovery patterns marks this as a B-grade implementation rather than A-grade.
Assemble JASM bytecode text and replace an entire class in the workspace
Assemble JASM bytecode text and replace a method's bytecode in the workspace
Attach to a running JVM by PID or display name. At least one of 'pid' or 'displayName' must be provided.
Disconnect from the currently attached JVM
List all running JVMs on the local system with PID, display name, and main class
Load classes from the attached JVM into the current workspace for analysis
Build or rebuild the call graph for the current workspace
Output schemas are not documented. LLMs cannot predict what disassemble-method, attach-list-vms, or callgraph-path return, forcing them to guess at field names and structure for downstream operations. This breaks tool chaining and increases hallucination risk.
Error handling lacks recovery guidance and error categorization. Tools do not indicate whether failures are retryable (network), user-fixable (invalid class name), or fatal (JVM already attached). LLMs cannot decide whether to retry, ask the user, or abandon.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 62 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 57 | - | v1 |
Find all methods called by the specified method. Requires the call graph to be built first (use callgraph-build).
Find all methods that call the specified method. Requires the call graph to be built first (use callgraph-build).
Find a call path between two methods using BFS on the call graph. Returns the shortest chain of method calls from source to target. Requires the call graph to be built first (use callgraph-build).
Delete the comment on a class or class member in the current workspace
Get the comment on a class or class member in the current workspace. Omit memberName to get the class-level comment. If memberName is provided without memberDescriptor, returns comments for all members with that name.
Search for comments in the current workspace by regex or text pattern
Set a comment on a class or class member (field/method) in the current workspace. For members, provide memberName plus memberDescriptor and memberType.
Disassemble an entire class to JASM format
Disassemble a specific method to JASM bytecode representation
attach-connect has mutually exclusive parameters (pid vs displayName). Description states 'at least one must be provided' but does not clarify behavior if both are provided, or what happens if neither matches. This creates ambiguity for the LLM.
Destructive tools (assemble-method, assemble-class, comment-delete) do not support dry-run or confirmation steps. An agent mistake could permanently corrupt bytecode or lose comments. No confirmation_before_execute pattern is present.
Stateful tools (attach-connect, attach-load-classes, attach-disconnect, callgraph-build) do not document how agent state is managed across multiple requests. Are connections persistent per agent session? Per MCP server instance? What happens if the agent disconnects mid-operation?
Tool descriptions lack dependency hints. For example, callgraph-callers, callgraph-callees, and callgraph-path require callgraph-build to be called first, but this is buried in the description of each tool rather than stated as an explicit prerequisite. LLMs may attempt to call callgraph-callers without building the graph first.
attach-load-classes and callgraph-build have empty input schemas ({}). While this is valid, the descriptions are vague about what state they operate on (which workspace? which attached JVM?). Parameter validation rules and scope are not explicit.
Security: Tools do not declare required permissions (e.g., 'read:bytecode', 'write:bytecode', 'admin:jvm_attach'). Without explicit scope declarations, agents cannot be configured with least-privilege access.
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are not present in schema definitions. These would help LLMs reason about side effects and retry safety.
comment-search accepts a regex pattern but does not validate it before execution. An agent could pass an invalid regex, causing a runtime error. The description should state valid regex syntax or provide error guidance.