Loading ToolBench
Loudness compliance verdicts against formal published standards (EBU R128, ATSC A/85) — a pass/fail answer with exact deltas, not raw meter output.
Two well-named, focused tools with clear descriptions and complete input schemas. check_loudness has comprehensive parameter documentation with defaults and constraints; list_standards is a simple discovery tool. Output schemas are documented in docstrings. Error handling returns structured dicts with 'verdict' and 'error' fields. Main gaps: no tool annotations (readOnlyHint/destructiveHint), output schema not formally declared in JSON Schema, and parameter descriptions could be more explicit about constraints (e.g., standard enum values, stream index bounds).
Loudness compliance verdict for a media file against a formal published standard. Measures integrated loudness, loudness range, and true peak (via ffmpeg), then evaluates them against the named standard's targets and tolerances.
List the standards this tool can verdict against, with their targets, tolerances, and spec citations.
Tool annotations missing: check_loudness and list_standards lack readOnlyHint/destructiveHint/idempotentHint declarations. Both are read-only operations and should be annotated to guide agent planning.
Output schema not formally documented: docstrings describe return structure (dict with 'verdict', 'error', 'file', etc.) but no JSON Schema definition is provided. LLMs cannot parse the exact field types and structure.
Parameter 'standard' lacks enum constraint in schema: description lists valid values (EBU_R128, ATSC_A85, BS_1770) but no enum is declared. LLMs may hallucinate invalid standard names.
Parameter 'stream' lacks bounds documentation: description says 'zero-based audio stream index' but does not specify valid range (e.g., 0 - N where N is audio_streams count). LLMs may pass negative or out-of-range indices.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Error handling returns unstructured error messages: AnalysisError and ValueError are caught and returned as plain strings in 'error' field. No guidance on recovery (e.g., 'file not found, check path' vs 'ffmpeg not installed, install ffmpeg >= 5.0').