Typed decision tools for AI agents, powered by TypeSafe Jev
Arbitype provides 9 tools with basic descriptions and input schemas, but falls short of production quality. Tool names lack action verbs (evaluate, classify, score, check, verify, gate, review, route are generic or domain-specific without clear intent). Descriptions are present but minimal (10-50 chars), below the 50-200 char LLM-optimized baseline. Parameter descriptions exist but are terse. No output schemas documented. Error handling guidance is absent. The tools appear to be wrappers around a TypeSafe Jev evaluation engine rather than discrete, composable operations. No evidence of idempotency, pagination, or recovery patterns.
Check if a state satisfies given instructions (boolean verification)
Classify a state into one of several predefined labels
Evaluate a state against structured questions using TypeSafe Jev
Gate execution based on checks against a state
Health check endpoint for the Arbitype service
Review a state against multiple checks
Route a state to one of several actions
Tool names lack action verbs. 'evaluate', 'classify', 'score', 'check', 'verify', 'gate', 'review', 'route' are domain-specific but do not clearly signal what the tool does to an LLM. Rename to verb_noun pattern: e.g., 'evaluate_state', 'classify_state', 'score_state' or more specifically 'run_evaluation', 'run_classification'.
Descriptions are too short (10-50 chars). Baseline for LLM-optimized descriptions is 50-200 chars. Current descriptions lack context on WHEN to use each tool and what distinguishes it from similar tools. E.g., 'Evaluate a state against structured questions using TypeSafe Jev' does not explain when to call evaluate vs classify vs score.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2025-06-18+ | v2 |
Score a state on an ordinal scale
Verify claims against a state
No output schemas documented. Tools return results but the response structure is not specified. LLMs cannot plan downstream calls or extract fields without knowing what fields are returned.
No error handling guidance. Tools do not document what errors can occur, whether they are retryable, or what the LLM should do next. E.g., if 'evaluate' fails due to invalid questions, should the LLM retry, ask the user, or call a different tool?
Parameter descriptions are minimal. 'state' is described as 'The current state or context to evaluate' but does not specify format (JSON object, string, nested structure?), size limits, or required fields. 'questions' is described as 'Dictionary of question IDs to question definitions' but does not document the structure of a question definition.