Loading ToolBench
Production-ready relay backend for Ollama Cloud App with built-in assistant toolkits for calculations, code execution, web fetching, file operations, and visualizations
This server has 29 tools with highly inconsistent quality. Naming is generally verb-first (calculator, run_javascript, fetch_url, create_file, etc.), which is good. However, descriptions vary wildly in depth and clarity. Compute tools (calculator, run_javascript) have excellent descriptions (100-150 chars, clear use cases). File tools (create_file, write_file, edit_file, read_file, delete_file) have minimal descriptions (20-50 chars). Visualization tools (render_table, render_chart, etc.) have generic descriptions that don't explain WHEN to use each variant or how they differ. Most critically: NO input schemas are visible in the provided source code. The Swift files show tool names and descriptions but no explicit JSON Schema definitions. Without visible schemas, all tools score 0 on schema dimension. Parameters are mentioned in descriptions (e.g., 'expression', 'code', 'url') but lack formal type declarations, constraints, or ranges. Error handling is present in some tools (calculator validates allowed characters, fetch_url validates URLs and blocks private hosts) but absent in visualization tools. No tool declares permissions, audit requirements, or idempotency guarantees.
Evaluate a single arithmetic expression (digits and + - * / % . parentheses only) and return the result. For anything more complex, use run_javascript.
Create a directory at the given path. Parent directories are created automatically.
Create a new file at the given path with the provided content. Parent directories are created automatically.
Delete a file or an empty directory.
Search-and-replace within a file. Replaces the first occurrence of old_text with new_text.
Fetch the readable text of a public http(s) URL and return it (truncated to ~8000 characters). Use to read a web page or API response.
No input schemas visible in source code. All 29 tools lack formal JSON Schema definitions with type constraints, required fields, and parameter validation rules. Cannot verify schema correctness.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 34 | <=2025-11-25 | v2 |
List all files and directories in the project as a tree.
Move or rename a file or directory.
Read a file's content. Returns the full text of the file.
Render a candlestick chart.
Render an interactive bar, line, or scatter chart.
Render a comparison table with items scored across criteria.
Render a multi-component dashboard layout.
Render a horizontal flowchart.
Render a funnel chart.
Render a Gantt chart for project timelines.
Render a gauge/dial indicator.
Render a 2D heatmap.
Render a grid of metric cards with optional deltas.
Render multiple data series on one chart.
Render an interactive pie or donut chart.
Render a radar/spider chart.
Render a Sankey flow diagram.
Render an interactive styled data table.
Render a vertical timeline of events.
Render a collapsible tree diagram.
Render a waterfall chart.
Execute JavaScript in a secure on-device sandbox and return its console output. Browser-style JS (no Node.js, require, or fs); setTimeout/setInterval are supported and run on a fast virtual clock. Use for calculations, data transforms, JSON/string work, algorithms, and short timed simulations. Print results with console.log.
Replace an existing file's entire content. Creates the file if it doesn't exist.
File operation tools (create_file, write_file, edit_file, read_file, delete_file, move_file) have minimal descriptions (20-35 chars). Descriptions lack guidance on when to use each tool, what happens on error, or idempotency guarantees. delete_file is particularly risky with only 'Delete a file or an empty directory.', no warning about irreversibility.
Visualization tools (render_table through render_candlestick, 18 tools) have generic, nearly identical descriptions (40 chars each). No guidance on when to choose render_chart vs render_multi_chart, or render_pie_chart vs render_gauge. LLMs cannot distinguish between similar tools.
No error handling guidance in visualization tools. If render_chart receives invalid data (mismatched x/y lengths, non-numeric y values), no recovery path is documented. Compute tools (calculator, run_javascript) have good error handling; file tools have none.
No tool declares permissions, audit requirements, or idempotency guarantees. Destructive tools (delete_file, write_file) lack confirmation patterns or dry-run support. No scope declarations (read:files, write:files) for least-privilege agent configuration.