Loading ToolBench
Desktop AI studio for xAI Grok and Google Gemini/Vertex AI models with agent-based code execution, file operations, and media generation capabilities
Caelo provides 9 tools with explicit schemas and descriptions. Most tools have clear verb-noun naming (read_file, list_dir, write_file, run_command) and documented parameters. However, several parameter descriptions lack detail on constraints, formats, and error recovery. The glob and grep tools have minimal parameter documentation. Output schemas are not documented. Error handling descriptions are present but lack actionable recovery guidance. Tool composition is reasonable (file ops, command execution, media generation are separate), but descriptions could better explain when to use each tool vs. alternatives.
Replace an exact unique string in a file. Requires approval.
Generate one or more images from a text prompt and show them to the user in the chat. Use when the user asks to create, draw, design, render or imagine an image. The images are displayed automatically — do not also paste them as links.
Generate a short video. Rendering takes a while, so the job is queued — tell the user to watch the Video or Gallery tab for the result. Use when the user asks to create or animate a video. If the user just generated or attached an image and wants to animate it, the most recent image is used as the first frame (image-to-video).
Find files by glob pattern (e.g. '**/*.py') relative to workspace.
Search file contents by regular expression. Returns path:line: match.
List entries of a directory in the workspace.
glob and grep parameter descriptions lack constraint details. 'pattern' in glob has no description of valid glob syntax or limits. grep's 'pattern' lacks mention of regex engine, timeout behavior, or file size limits (8MB cap exists in code but not documented).
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract fields without knowing return structure. E.g., read_file returns 'numbered lines' but format (array? object with line numbers?) is unclear.
Error handling descriptions lack actionable recovery guidance. run_command says 'Requires approval' but does not explain what happens if approval is denied, what errors are retryable, or how to handle timeouts. No error classification (retryable vs. fatal).
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Read a UTF-8 text file from the workspace. Returns numbered lines.
Run a single program in the workspace. Requires approval. Returns exit code + combined stdout/stderr, so NEVER redirect output (no 2>&1, >, <). Shell operators are NOT allowed (no && | ; & < > $ ` ( ) { } or newlines) and such calls are rejected — to chain steps, issue separate run_command calls (e.g. 'mkdir build' as its own call, then the next command).
Create or overwrite a file with the given content. Requires approval.
run_command description mentions shell operator restrictions but does not enumerate them clearly in parameter description. LLMs may still attempt pipes, redirects, or semicolons. Constraint should be formalized in description.
generate_image and generate_video lack documentation of rate limits, quota behavior, and failure modes. What happens if the user exceeds daily image generation limits? How long does video rendering take? These details guide agent planning.