Loading ToolBench
A collection of reusable skills and templates for multi-model routing, API resilience, and LLM integration patterns. Includes drop-in routers for task-type-based model selection and resilient Anthropic API clients with circuit breaker and exponential backoff.
Server exposes 5 tools with partial schemas and descriptions. Tool names lack action verbs (ModelRouter.complete, ModelRouter.resolveModel, ResilientAnthropicClient.messages.create, etc.), they read as class methods rather than agent actions. Descriptions exist but are generic (194-char baseline; these average ~150 chars, acceptable). Input schemas present with enums and types, but output schemas are NOT documented, critical gap. Error handling is minimal; no recovery guidance. Adapters are stubbed (NotImplementedError), making tools non-functional. No tool annotations (readOnlyHint/destructiveHint). Overall: template-quality code, not production-ready.
Routes a completion request to the appropriate model based on task type, calls the provider adapter, and emits telemetry. Returns text, model ID, token counts, cost, and latency.
Resolves the ModelSpec for a given task type from the task-type map. Raises ValueError if task type is not registered.
Returns per-key health snapshot including success rate, consecutive failures, backoff status, circuit breaker state, and overall system status (healthy/degraded/down). Used for /health endpoints and dashboards.
Interface-compatible wrapper around anthropic.messages.create() that adds round-robin key rotation, exponential backoff, and circuit breaker resilience. Automatically retries on transient errors.
Resets all per-key state including consecutive failures, backoff timers, circuit breaker flags, and error messages. Use after rotating revoked API keys.
Tool names are class method names, not action verbs. 'ModelRouter.complete' should be 'complete_with_model_router' or 'route_completion'. LLMs infer intent from verb-first names; method-style names obscure purpose.
Output schemas are not documented. CompleteResponse, TelemetryRecord, and key_stats structures are defined in code but not exposed in tool definitions. LLMs cannot plan downstream calls without knowing return fields.
Provider adapters are stubbed with NotImplementedError. Tools cannot execute. Anthropic, OpenAI, Google, and Deepgram adapters all raise 'Install X and wire Y', this is a template, not a working server.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 34 | 2026-07-28+ | v2 |
No error handling guidance. Exceptions are caught and telemetry is emitted, but error responses do not tell the LLM what to do next (retry, ask user, abort). No recovery patterns.
No tool annotations. Tools lack readOnlyHint, destructiveHint, or idempotentHint. reset_all is destructive but unmarked; get_key_stats is read-only but unmarked. Agents cannot infer safety properties.