Loading ToolBench
Compiler and runtime for Juglans Workflow Language - a DSL for AI workflows, agents, and orchestration with built-in developer tools and package registry
Three tools (read_file, write_file, edit_file) with clear action-verb naming and documented schemas. All tools have input schemas with parameter types and descriptions present in src/builtins/devtools.rs. Descriptions are adequate (72-192 chars). However, output schemas are not explicitly documented, and error handling lacks recovery guidance. Parameter descriptions could be more detailed regarding constraints and validation rules. Tool composition is appropriate, each performs one specific task. No security issues detected in parameter design (no secrets exposed), but permission gating and audit logging are not evident in the code sample.
Perform exact string replacement in a file. The old_string must be unique in the file unless replace_all is true. Fails if old_string is not found or found multiple times (ambiguous).
Read a file from the filesystem. Returns contents with line numbers (cat -n format). Supports text files, returns error for binary files.
Write content to a file. Creates parent directories if needed. Overwrites existing file.
Output schemas not documented. Tool descriptions state what is returned (e.g., 'Returns contents with line numbers') but no structured schema definition is visible for outputs. LLMs cannot parse downstream results predictably.
Parameter constraints are under-specified. 'file_path' accepts absolute or relative paths but no pattern, length limit, or sanitization guidance is documented. 'limit' has no min/max bounds specified in description (code shows 2000 default but no explicit constraint on LLM-supplied values). Without constraints, LLMs may pass invalid values.
Error handling lacks recovery guidance. Code shows context-wrapping errors (e.g., 'Failed to read file: {}') but descriptions do not state what to do on failure (retryable? user-fixable?). LLM receives raw error without next-step guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 68 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 20 | - | v1 |
edit_file parameter 'old_string' is marked as requiring uniqueness unless replace_all=true, but no guidance on how LLM should handle ambiguous matches. Description states 'Fails if... found multiple times' but does not suggest recovery (e.g., 'call read_file to inspect context, refine old_string, retry').
Destructive operations (write_file, edit_file) do not support confirmation or dry-run. No dry_run parameter or confirm_before_execute pattern visible. Agents may irreversibly overwrite files without user approval.