Loading ToolBench
Cast exposes 29 tools with highly variable quality. Core file/bash tools (read, write, edit, bash) have excellent descriptions (150-300 chars) with clear parameter docs and risk annotations. However, 8 tools (memory, session_history, task, skill, todo_write, plan_done, question, web_search) lack visible input schemas in the provided source, only descriptions are shown. Many utility tools (echo, add, fails, get-last-auth-header, rich-content, page-a, page-b, fat) appear to be test fixtures with minimal descriptions (10-50 chars), dragging the average down. Parameter descriptions are present for core tools but sparse for fixture tools. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) detected. Error handling is mentioned in bash description but not systematized across tools.
Adds two numbers.
Execute a bash command in the current working directory. Returns stdout and stderr. Each call runs in a fresh shell rooted at that same directory: a `cd` does not carry over to the next call, so repeat it (or use absolute paths) whenever a command must run elsewhere. Output is truncated to last 2000 lines or 128KB (whichever is hit first). Short commands return normally. Long-running commands are automatically promoted to a managed background task instead of blocking indefinitely; use run_in_background:true to start one immediately. Background results arrive automatically, and bash_output/bash_kill can inspect or stop them. Do NOT re-run an identical command to 'double-check' a result you already have — the previous output still holds unless something changed. Running the same command repeatedly is treated as a doom loop and blocked.
Stop a background bash task.
Inspect or wait for the result of a background bash task.
Echoes back the given text.
8 tools (memory, session_history, web_search, web_fetch, task, skill, todo_write, plan_done, question) lack visible input schemas in source code.
Test fixture tools (echo, add, fails, get-last-auth-header, rich-content, page-a, page-b, fat) have minimal descriptions (10-50 chars), violating the 10-1024 char guideline and making them unsuitable for LLM selection.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. Bash, write, edit, bash_kill, ssh, task, skill, todo_write, plan_done are destructive/irreversible but lack explicit annotations to guide agent behavior.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
Performs exact string replacement in a file. You must read the file (or already have its contents from this session) before editing it. `oldString` must be the exact literal text to replace, including all whitespace and indentation — copy it verbatim from a recent `read`, don't retype it from memory. Include enough surrounding context (a few lines) to make `oldString` uniquely match one location in the file, or the edit is rejected as ambiguous. By default only the first occurrence is replaced; pass count to replace multiple.
Always returns a tool error.
Returns a huge blob.
Returns the X-Test-Token header of the last HTTP request.
Search for files matching a glob pattern in the current working directory.
Search for text patterns in files using grep.
Finds the stable catalog identifier for a named component. Use this before reading release details.
List directory contents.
Search memory with one to three distinctive search terms. Returns matching memory entries from global, project, session, or Claude Code scopes.
First page tool.
Second page tool.
Signal that the plan is complete and ready for review.
Ask the user a question and wait for their response.
Read the contents of a file. When you already know the path, call this directly — do not search with glob/ls first. Supports text files and images (jpg, jpeg, png, gif, webp, bmp — shown to you as an image in the next message; only works if the model supports vision). Output is truncated to 2000 lines or 128KB. Use offset/limit for large files. Large images are automatically downscaled to fit; only rejected if truly huge (25MB+). Each line is prefixed with its line number (`N: content`) — use the exact text (not the number) when calling `edit`. You already have the contents of every file you read earlier in this session — do NOT read the same path again unless it has changed since (e.g. you just edited it); re-use the earlier result instead. Reading the same unchanged file repeatedly is treated as a doom loop and blocked.
Reads the published release status and version for a catalog component id.
Returns every MCP content type in one result.
Search session history with one to three distinctive search terms. Returns matching entries from the current project or globally across all projects.
Load and execute a workflow skill matching the current context.
Execute a command on a remote SSH host.
Delegate a scoped task to a subagent with a specific assignment.
Write or update a structured todo list with checklist items.
Fetch and parse the contents of a web page.
Search the web using the configured search provider.
Create a new file or overwrite an entire file. Prefer `edit` for surgical changes to existing files. Do NOT fall back to write because an edit failed — retry the edit with more surrounding context in oldString instead; a from-memory rewrite tends to reproduce stale content. Automatically creates parent directories.
Parameter descriptions missing or generic for many tools. glob, grep, ls, web_search, web_fetch, task, skill, question lack detailed parameter guidance (e.g., glob pattern syntax, search provider behavior, task assignment format).
Error handling guidance inconsistent. Bash tool documents recovery (background promotion, truncation), but most other tools lack actionable error messages or recovery hints (e.g., web_search/web_fetch do not document rate limits, timeouts, or fallback strategies).