Loading ToolBench
Claude Code runner and orchestrator — thin job lifecycle, repo management, and OTEL pipeline
agenticore has 9 well-structured tools with complete JSON schemas and consistent parameter typing. Most tools have descriptions (8/9), but many are terse or lack actionable detail for LLM selection. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present. Tool descriptions average ~120 chars, below the 194-char baseline for A+ tools. Parameter descriptions are present and typed, which is strong. Error handling is not visible in the schema, no recovery guidance documented. Security appears sound (no credential params visible). Composition is good: tools follow verb_noun pattern and have clear single responsibilities (run_task vs plan_task vs execute_plan are well-separated). Output schemas are not visible, major gap for downstream chaining.
Call the agent with a message. Returns JSON with result.
Cancel a running or queued job.
Execute a ready plan by ID. Submits a normal coding job with the plan injected as context.
Get job status, output, and artifacts.
Get a plan by ID, including its markdown content once ready.
List recent jobs with status.
List recent plans with status and name.
No output schemas documented. Tools specify inputs clearly but return types are invisible. LLMs cannot verify what fields to expect (e.g., does run_task return {job_id, status} or {job_id, status, artifacts, error}?). Breaks tool chaining, agent cannot reliably extract job_id from run_task response without trial-and-error.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). run_task, cancel_job, and execute_plan modify state, should be marked destructiveHint=true so LLMs know to request confirmation. get_job, list_jobs, get_plan, list_plans should be marked readOnlyHint=true. cancel_job and plan_task should declare idempotency.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 76 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Create an implementation plan without executing it. Runs Claude in read-only mode (Read, Glob, Grep only) to analyse the codebase and produce a markdown plan. The plan can later be executed with execute_plan.
Submit a task for Claude Code execution. Async (fire and forget) by default — returns job ID immediately. Set wait=True to block until the job completes.
Descriptions are terse and lack decision guidance. 'Get job status, output, and artifacts' (get_job) and 'Get a plan by ID' (get_plan) do not explain when to call them or how they differ from similar tools. Baseline for A+ tools is 194 chars; most here are 50 - 75 chars. LLMs cannot easily decide which tool to invoke.
Pagination semantics not documented. list_jobs and list_plans accept limit but no offset/cursor. Descriptions do not clarify sort order, whether limit is hard-capped, or how to iterate. Breaks pattern:paginated-result.
agent_completions has 17 parameters, many with unclear purpose or relationships. 'permission_mode', 'output_format', 'effort', 'fallback_model', 'allowed_tools', 'disallowed_tools' lack examples or enums. Free-form strings like effort='high' vs effort='maximum' are ambiguous. LLMs cannot determine valid values.
Error handling not documented in schema. No recovery guidance visible. If run_task fails due to invalid repo_url, what should LLM do next? If plan_task times out, is it retryable? Pattern:recovery-guide not evident.