Loading ToolBench
A Java-based MCP server that exposes tools for workspace file operations, process execution, Java shell evaluation, memory management, and AI-powered code analysis tasks
CastCLI defines 20 tools with consistent naming (verb_noun pattern) and descriptions present for all. However, input schemas lack explicit type declarations and constraints. Most parameter descriptions are adequate (50-150 chars) but lack format/range guidance. Output schemas are undocumented. Error handling is minimal, no recovery guidance or actionable error messages visible. Tool composition is sound (each does one thing), but parameter validation rules are not stated in descriptions. Security considerations (credentials, permissions) are not evident in tool definitions.
Perform first-pass failure triage. Returns the most likely causes in ranked order, evidence for and against each cause, the smallest useful verification steps, and relevant file paths when supplied.
Returns the current date and time in an IANA time zone such as America/New_York
Draft a candidate patch for the requested change using only the supplied files. Output a unified diff followed by a short assumptions/verification note. Does not claim the patch was applied or tested. Does not modify credentials, authorization policy, or production data.
Evaluates a Java snippet in JShell. Use only for calculation and isolated Java experiments.
Delete an existing writable memory using its ID and current version.
Draft focused tests for the requested behavior using the supplied source context. Returns test code or a unified diff plus a compact edge-case table. Does not claim tests were run.
Lists files inside the workspace using a glob such as **/*.java
Input schemas lack explicit type declarations. Parameters show only 'type':'string' or 'type':'integer' with no constraints (min/max, enum, pattern, length). LLMs cannot infer valid ranges or allowed values.
Output schemas are not documented. Tools like summarizeFiles, analyzeFailure, draftPatch, generateTests, reviewDiff, mapChangeImpact delegate to LLM and return unstructured text. LLMs cannot plan downstream tool calls or extract structured data.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Map the likely impact of changing the named symbol from literal repository-search results. Separates confirmed references from inferred impact, groups findings by production code/tests/docs, cites file:line evidence, and identifies additional searches needed.
Reads a UTF-8 text file inside the configured workspace
Recall relevant shared project memories using hybrid keyword and vector search.
Recall long-term session summaries and turnover context recorded across sessions and agents.
Persist a reusable decision, preference, finding, or architectural fact in shared project memory. Never store credentials.
Perform a first-pass, non-security code review of a diff. Reports only actionable findings, ranked by severity, with file/hunk evidence and suggested corrections. Does not provide final correctness approval.
Runs a pre-approved build/test/VCS command. commandKey must be one of: gradle-test, gradle-build, gradle-check, git-status, git-diff, git-log. Requires tools.allowShellExec=true and, unless auto-approved, interactive confirmation.
Searches UTF-8 workspace files for a literal string and returns file:line matches
Semantically searches a pre-built embedding index of the workspace for code/text relevant to a natural-language query (e.g. 'where do we validate matchmaking tickets'), returning the most similar chunks with file path, line range, and relevance score. Complements searchWorkspace's literal-text matching. Requires the index to have been built with 'llm-harness index' first.
Bounded, read-only repository summary using supplied files. Answers questions directly, cites workspace-relative paths, distinguishes facts from inference, and keeps responses concise.
Summarize recorded session actions using the local LLM in the background and persist to long-term memory for agent turnover.
Update an existing writable memory using its ID and current version; rejects concurrent stale edits.
Writes UTF-8 text content to a file inside the configured workspace, creating parent directories as needed. Requires tools.allowWrites=true and, unless auto-approved, interactive confirmation.
Parameter descriptions lack format/range guidance. E.g., 'maxResults' accepts 1-500 but description does not state bounds. 'zoneId' expects IANA format but no validation hint. LLMs guess and pass invalid values.
No error handling guidance visible. Tools like writeWorkspaceFile, runCommand, forgetContext perform destructive/reversible operations but descriptions do not explain error recovery, retryability, or confirmation patterns.
Destructive tools (forgetContext, writeWorkspaceFile) lack confirmation or dry-run support. Descriptions do not mention approval gates or reversibility. Agents may accidentally delete or overwrite data.