Loading ToolBench
A CLI tool for testing and orchestrating MCP (Model Context Protocol) servers. Runs rondo YAML files containing sequences of tool calls, resource reads, and assertions against MCP servers.
Ocarina has 10 tools across two categories: 6 test/demo tools (echo, boom, profile, count, flaky, peek) and 4 task-tracker tools (add_task, list_tasks, complete_task, delete_task). Test tools have minimal descriptions (10-30 chars) and empty input schemas. Task-tracker tools have better descriptions (40-80 chars) and partial parameter schemas, but lack parameter descriptions entirely. No tool documents output schemas. No error handling guidance. Naming is inconsistent: 'add_task' vs 'complete_task' vs 'delete_task' uses different verb patterns. Test tools lack verb-noun naming (echo, boom, profile, count, flaky, peek are all nouns or generic). Parameter descriptions are completely absent across all tools, LLMs cannot infer what 'status' means in list_tasks or 'task_id' means in complete_task. No tool declares required vs optional parameters in descriptions. Overall, definitions are skeletal and would not pass production code review.
Add a new task. Returns the created task as JSON.
always returns isError:true
Mark a task as completed. Returns the updated task as JSON.
returns an incrementing call number
Delete a task by ID. Returns the deleted task as JSON.
echo back the text arg
fails until the third call
Test tools (echo, boom, profile, count, flaky, peek) lack verb-noun naming. Names like 'echo', 'boom', 'profile' are nouns or generic verbs that do not convey action. LLMs cannot infer intent from names alone.
All 10 tools lack parameter descriptions. 'status' in list_tasks, 'task_id' in complete_task, and 'text' in echo have no descriptions explaining what they control or what values are valid. LLMs cannot determine correct usage.
Test tools (boom, profile, count, flaky, peek) have empty or minimal input schemas (objectSchema with no properties). Schemas must declare parameter types and constraints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2025-06-18+ | v2 |
List tasks. Pass status='pending' or status='completed' to filter.
read-only probe
returns structured JSON
No tool documents output schema or return type. LLMs cannot plan downstream calls or extract required fields. Task-tracker tools return JSON but structure is undocumented.
No error handling guidance. Tools like 'flaky' and 'boom' return errors but provide no recovery hints. Descriptions do not explain when to retry, what went wrong, or what to do next.