Loading ToolBench
Tier-based tool routing for AI agents — adaptively presents tools differently by model size (0.5B to 400B+). Routes user intents to appropriate tools based on embedding similarity, model tier, and usage statistics. Bridges to Python TierEngine via persistent subprocess for high-throughput event streaming.
Tier exposes 17 tools with reasonable naming (verb_noun pattern) and basic descriptions, but critical gaps undermine production readiness. Most tools lack parameter type definitions in visible schemas, only descriptions are provided without formal JSON Schema constraints. Output schemas are entirely undocumented. Error handling is absent: no recovery guidance, no categorization of retryable vs fatal errors, no actionable error messages. The event bus domain is well-scoped, but the tool interface does not follow LLM-optimized patterns: descriptions are terse (avg ~80 chars, below the 194-char baseline), parameters lack enums/constraints, and composition assumes the agent understands event sourcing internals rather than abstracting complexity. No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. The 'compact' tool is destructive but lacks confirmation/dry-run support.
Acknowledge successful processing of an event by sequence ID.
Compact the event log by removing events before a sequence number or date.
Create a consumer group for subscribing to events.
Get dead-letter queue entries for failed events, optionally filtered by consumer group.
Get the consumer lag (difference between latest and processed sequence) for a group.
Check the health status of the event bus.
Negative acknowledge an event, marking it for retry with an error message.
Poll for events from a consumer group with optional limit and partition key.
No output schemas documented. Tools return BridgeResult with unknown structure. LLMs cannot plan downstream calls or extract required fields (e.g., event_id, sequence_id) for chaining.
Parameter schemas lack formal type definitions and constraints. Descriptions state 'array' or 'object' but no JSON Schema types, enums, or ranges. E.g., 'maxRetries' has no min/max bounds; 'eventFilter' structure is undocumented.
No error handling guidance. Tools fail silently or return generic errors. No recovery hints (e.g., 'group not found, call createGroup first'). No categorization of retryable vs fatal errors.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | <=2025-11-25 | v2 |
Publish an event to the event bus with optional correlation and causation IDs, metadata, and partition key.
Publish multiple events to the event bus in a single batch operation.
Register an event schema for validation and documentation.
Replay events from the event bus within a sequence range or time window with optional filtering.
Reset a consumer group's checkpoint to a specific sequence number.
Mark a dead-letter queue entry as resolved without retrying.
Retry processing a dead-letter queue entry.
Get statistics about the event bus including total events and consumer groups.
Subscribe a consumer group to events matching specified filters with retry configuration.
Destructive tool 'compact' lacks confirmation or dry-run. No idempotent guarantee. Agents can accidentally delete event history without safeguards.
Tool descriptions are terse (avg 60 chars vs 194-char baseline). Missing context on WHEN to use each tool and dependencies. E.g., 'subscribe' does not explain that createGroup must be called first.