Loading ToolBench
Five accessibility-focused tools with clear, domain-specific descriptions (avg 180 chars, within baseline 194). All tools have input schemas with Zod type definitions and parameter descriptions. However, output schemas are undocumented, responses are JSON-stringified but no structured schema is declared. Parameter descriptions are adequate but lack format constraints (e.g., no enum for color formats despite mentioning 'hex, rgb(), named'). Error handling is absent, no guidance on invalid inputs or recovery paths. Tool names follow verb_noun pattern (check_*, validate_*) appropriately. No tool annotations (readOnlyHint, etc.) despite all being read-only operations.
Analyze images in HTML for missing, empty, or low-quality alt text. Reports which images need alt text, which use decorative (empty) alt, and which are properly labeled. Also checks linked images and input[type=image].
Check WCAG 2.x contrast ratio between a foreground and background color pair. Reports AA and AAA compliance for both normal and large text. Accepts hex (#fff, #ffffff), rgb(r,g,b), or named colors.
Check form elements for accessibility: verifies labels are associated with inputs, radio/checkbox groups use fieldset/legend, placeholder is not used as sole label, autocomplete hints on personal info fields, and detects duplicate IDs.
Validate heading structure in HTML: checks for proper h1-h6 nesting order, no skipped levels, single h1, empty headings, and heading content quality. Returns a visual tree of the heading structure.
Validate ARIA attributes in HTML: checks for invalid roles, missing required ARIA properties, redundant ARIA on native elements, aria-label on non-interactive elements, and aria-hidden on focusable elements.
Output schemas undocumented. All tools return JSON-stringified text without declaring the structure of result objects (e.g., AltTextResult, ContrastResult). LLMs cannot plan downstream operations or extract fields without explicit schema documentation.
check_color_contrast accepts color formats (hex, rgb, named) but does not constrain input with enum or regex. Parameter descriptions mention formats but lack validation rules. LLMs may pass invalid colors like 'redd' or 'rgb(999,999,999)' without guidance.
No error handling or recovery guidance. Tools do not document what happens on invalid HTML, malformed colors, or edge cases. No error messages guide LLMs to retry or correct input.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Tool annotations missing. All five tools are read-only (no side effects), but readOnlyHint is not set. This prevents clients from optimizing caching, parallelization, or safety policies.
HTML parameter descriptions are generic ('The HTML content to analyze...'). No guidance on expected HTML structure, size limits, or encoding. LLMs may pass malformed or oversized HTML without knowing constraints.