Loading ToolBench
A durable, always-running meta-agent operating system: creates, runs, evaluates, and evolves specialized agents over time.
Bakudo exposes 15 tools with basic schemas and descriptions, but falls short of production quality. All tools have descriptions (10-80 chars, mostly under 100), which is positive. However, parameter descriptions are minimal or absent, most params lack detail on format, constraints, or dependencies. Input schemas exist but are sparse: many params are typed as 'object' with only a brief description, offering no internal structure. No output schemas are documented. Error handling is not visible in the tool definitions. The tools themselves are well-named (verb_noun pattern: create_objective, list_objectives, spawn_agent_run) and logically composed, but the definitions lack the depth needed for reliable LLM reasoning. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present despite clear risk levels (WRITE, READ_ONLY, REVERSIBLE). This is a domain-specific orchestration system, not a general-purpose API, which explains some design choices, but even specialized tools need richer parameter and output documentation.
Archive a candidate agent spec with a reason
Cancel an in-progress agent run
Compare multiple candidate runs on diff size, tests, and eval score
Create a new candidate agent specification from a spec document
Create an objective and add it to a queue (default: ready queue)
List objectives from a queue ranked by priority
Parameter schemas are underspecified. Most params are typed as 'object' with only a 1-2 sentence description. No internal field definitions, constraints (min/max, enum, pattern), or format hints. Example: 'objective' param in create_objective is {type: object, description: 'Objective specification document'}, what fields does it require? What's the structure?
No output schemas documented. Tools return results (e.g., list_objectives returns ranked objectives, run_eval_suite returns a scorecard) but the response structure is not defined. LLMs cannot plan downstream tool calls or extract fields without knowing what to expect.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Decide on candidate promotion based on scorecard comparison and policy
Query the status and results of an agent run
Query the event log for a run
Query memory items by scope with optional limit
Query Temporal workflow execution details for a run
Register an agent specification version in the ledger
Run the evaluation suite on a completed run and return scorecard
Spawn an agent run against an objective, with deterministic canary routing
Write a candidate memory item and return acceptance status
Tool annotations missing. Tools are marked with Risk levels (WRITE, READ_ONLY, REVERSIBLE) in the metadata but not exposed as readOnlyHint, destructiveHint, or idempotentHint in the tool definition. LLMs cannot infer safety properties without explicit annotations.
Parameter descriptions lack actionable detail. Most descriptions are 1-2 sentences with no guidance on format, valid values, dependencies, or constraints. Example: 'queue' param in list_objectives is {type: string, description: 'Queue name to list from', default: 'ready'}, what queue names are valid? Are there constraints?
No error handling guidance visible. Tool definitions do not document what errors can occur, how to recover, or what the LLM should do next. No recovery guides, error classification, or actionable error messages.