Loading ToolBench
Fast, exact, local-first Codex usage facts and evidence.
Server provides 6 tools with basic schemas and descriptions, but falls short of production quality. Tool names lack clear action verbs (evidence_* prefix is vague about what 'resolve' means). Descriptions are minimal (10-50 chars) and lack context on WHEN to use each tool or WHAT they return. Parameter descriptions are sparse. No documented output schemas. Error handling guidance is absent. The usage_query tool has complex parameters (filters object, multiple arrays) but no validation rules or examples in descriptions. Overall, definitions are skeletal, sufficient for basic discovery but insufficient for reliable LLM reasoning.
Resolve an evidence selector into activities
Resolve an evidence selector into allowance observations
Resolve an evidence selector into model calls
Resolve an evidence selector into a generation-bound timeline
Resolve an evidence selector into tool invocations
Query bounded usage facts from the kernel cache
Tool names lack action verbs. 'evidence_timeline', 'evidence_calls', etc. use 'evidence_*' prefix with vague 'resolve' semantics. LLMs cannot infer intent from names alone. Should be 'get_evidence_timeline', 'list_evidence_calls', or similar.
Descriptions are too brief (10-50 chars) and lack context. 'Resolve an evidence selector into a generation-bound timeline' does not explain WHEN to call this vs other evidence_* tools, WHAT a 'selector' is, or WHAT the output structure contains. Baseline: 194 chars average for A+ tools.
No output schemas documented. LLMs cannot plan downstream calls or extract fields without knowing what each tool returns. usage_query likely returns rows/aggregates, but structure is invisible.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 33 | 2025-06-18+ | v2 |
usage_query has complex parameters (filters object, dimensions/measures arrays) with no validation rules, format hints, or examples in descriptions. LLMs will guess at structure and pass invalid payloads.
No error handling guidance. Tools do not document what errors are possible, whether they are retryable, or what the LLM should do next. A failed selector resolution returns nothing actionable.