Loading ToolBench
An MCP server that exposes three audit and compliance tools: SOC 2 report summarization, ISO 42001 Statement of Applicability drafting, and control evidence evaluation. Uses Claude to process compliance documents and generate structured audit findings.
Three tools with complete input schemas and descriptions, but significant gaps in output documentation and error handling. Tool names follow verb_noun pattern (summarize, draft, evaluate). Descriptions are domain-specific and adequate (120-180 chars), exceeding the 10-char minimum but below the 50-200 char LLM-optimized range. All parameters have types and descriptions. However, output schemas are completely undocumented, LLMs cannot plan downstream calls or extract structured results. No error handling guidance, no recovery paths, no validation rules in parameter descriptions. No tool annotations (readOnlyHint present in metadata but not in schema). Parameter constraints (maxLength, maxItems) are present but not explained in descriptions, forcing LLMs to infer limits from schema alone.
Draft a Statement of Applicability section for ISO/IEC 42001:2023, given an AI system description and a list of Annex A controls to address.
Evaluate whether a piece of evidence sufficiently supports a control. Returns a sufficiency rating, coverage assessment, gaps, recommended additional evidence, and a mapping across SOC 2 / ISO 42001 / NIST AI RMF.
Summarize a SOC 2 report into structured JSON: service organization, audit period, opinion type, Trust Services Criteria, key exceptions, CUECs, subservice organizations, and risk flags. Mirrors how a Big 4 auditor reads a Type II report during vendor due diligence.
Output schemas completely undocumented. LLMs cannot infer what fields to expect, breaking downstream tool composition and forcing agents to guess at response structure.
No error handling guidance. Errors return raw exception text ('Error running X: ...') with no recovery hints, actionable next steps, or error classification (retryable vs fatal).
Parameter constraints (maxLength, maxItems) exist in schema but are not explained in descriptions. LLMs cannot read JSON Schema directly, they rely on description text to understand limits.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Tool annotations missing from schema. readOnlyHint and idempotentHint are declared in metadata but not in inputSchema, preventing clients from rendering safety warnings.
Single required parameter per tool with no defaults. 'report_text' (500KB max) and 'evidence_description' (50KB max) are unbounded in practical terms, no guidance on typical sizes or truncation behavior.