Loading ToolBench
Criterion Loom audit kernel for bounded requirement-relation and direction-binding checks.
This server exhibits severe definition quality gaps across nearly all dimensions. Of 24 tools, only 3-4 have complete input schemas with proper type definitions and descriptions. Most tools (tools 6-24) are defined in legacy code paths with minimal schema visibility. Tool names lack consistent verb-noun structure (e.g., 'semantic_guard_schema_tool' vs 'audit_requirement_relations_tool'). Descriptions are present but often cryptic and domain-specific without actionable guidance for LLM selection. Parameter descriptions are sparse or missing entirely. Output schemas are undocumented. Error handling is absent from visible code. The server conflates multiple concerns (auditing, exploration, schema retrieval, job management) without clear composition boundaries.
Audit public I/O, CLI, MCP, error, record, and repository-profile convention coverage.
Expose decided, undecided, hypothetical, inferred, value-judgment, and evidence-gap states without resolving them.
Audit a diff or change summary for traceability, meaning preservation, quality, security, tests, and docs.
Audit whether one direction-open expression has a directly bound direction. Morphology is signal-only. Numeric witnesses cannot change the primary evaluation, and machine workflow pass is not human acceptance.
Audit an implementation plan for scope, decomposition, risk, verification, validation, and completion evidence.
Audit requirements or documents with kind-aware checks.
Tool names lack consistent verb-noun structure. Names like 'semantic_guard_schema_tool', 'conventions_catalog_tool', and 'audit_result_schema_tool' are generic and do not clearly convey action. LLMs cannot infer intent from names alone.
Output schemas are completely undocumented. No tool explicitly declares what fields it returns, their types, or structure. LLMs cannot plan downstream calls or extract required data.
Parameter descriptions are sparse or missing. Many parameters (e.g., 'analysis_mode', 'morphology', 'dependency', 'profile') lack descriptions explaining valid values, constraints, or when to use them. Baseline: 100% of A+ tools have param descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
Audit a structured functional requirement with fail-closed obligation states. Provider choices are explicit. Parser and LLM candidates never acquire support or hold-mutation authority merely by being selected. The optional assurance-v1 output wraps the validated public audit in a replayable proof graph; it is not human acceptance.
Return the shared AuditResult JSON schema.
Return the machine-readable baseline convention catalog.
Check local semantic-guard environment, schemas, mappings, CI, and fixtures.
Evaluate local calibration fixtures and return structured pass/fail data.
Expose material ambiguities and necessary questions before turning an open-ended idea into a spec.
Audit completion evidence, acceptance evidence, residual risk, and release blockers.
Return a background LLM exploration job state: running, completed, failed, timed_out, or not_found.
Start the isolated LLM exploration reviewer as a pollable background job.
Run or dry-run an isolated LLM exploration reviewer for pre-spec questions. Dry-run is the default.
Build the isolated codex exec LLM reviewer command and prompt without executing it.
Return the LLM request-exploration review output schema.
Return rule ids mapped to detector and predicate surfaces.
One closed semantic-guard v1 JSON Schema.
Return one closed v1 JSON Schema by its public contract name.
Observe an operator-pinned legacy process and classify, but do not auto-resolve, deltas. This tool is disabled unless the server operator explicitly enables it and supplies one absolute legacy root through process environment. Tool callers cannot select an executable, adapter, manifest, or filesystem root.
Build a request-plan-diff-finish trace report from a JSON payload.
Audit whether the target, purpose, value, non-goals, unknowns, and validation route are understood.
No error handling guidance visible. Tools do not document what errors can occur, whether they are retryable, or what the LLM should do next. Baseline: A+ tools include recovery guidance.
Tool descriptions are domain-specific and cryptic. Phrases like 'fail-closed obligation states', 'direction-open expression', and 'assurance-v1 output wraps the validated public audit in a replayable proof graph' assume deep domain knowledge. LLMs cannot determine when to select these tools.