Loading ToolBench
aiglare: audit AI/LLM features in any JS/TS codebase for governance guardrails. Finds where model output reaches users or side-effects without confidence handling, fallbacks, validation, or human-in-the-loop.
Three well-named, domain-specific tools with clear governance focus. All have descriptions (194 - 250 chars, within baseline 34 - 392 range). Input schemas present with typed properties and enums. However, parameter descriptions are sparse or missing entirely, most params lack explanatory text beyond the name. Output schemas are not documented in the visible code. Error handling and recovery guidance are absent. Tool composition is sound (each does one thing), but LLM-facing descriptions could be more prescriptive about when to call each tool.
Audit a JS/TS repo for AI/LLM governance guardrails. Returns every surface where model output reaches a user or a side-effect, classified by sink and scored on confidence, fallback, validation, human-in-the-loop, and error isolation. Uses a repoctx index automatically if present.
Run the CI gate for a repo: reports whether any red + side-effectful surface exists (the "AI auto-triggers an irreversible action with no confirmation" case). passed=false means a build should fail.
List the AI/LLM provider registry the scanner detects (package names and inference hosts).
Parameter descriptions missing or minimal. 'path' param in ai_surface_audit and ai_surface_gate has description 'Repository path. Defaults to current working directory.' but 'sinks', 'severity', and 'compliance' lack actionable guidance on format, constraints, or when to use them. LLMs cannot infer valid enum values from names alone.
Output schemas not documented. No visible schema definition for what ai_surface_audit returns (surfaces array structure, fields like 'sink', 'guardrails', 'violations', 'confidence'). LLMs cannot plan downstream processing or extract required fields without documented return types.
No error handling or recovery guidance. Tools do not document what happens on invalid input (e.g. invalid compliance framework, nonexistent path, malformed enum). No guidance for LLM on retry strategy or fallback actions.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2025-06-18+ | v2 |
list_providers has empty input schema (no properties). Description does not explain what the tool returns or when to call it. Is it a discovery tool? Does it return package names, versions, endpoints? Unclear.
Compliance parameter in ai_surface_audit accepts enum ['soc2', 'eu-ai-act', 'nist', 'owasp'] but description does not explain what each framework covers, when to use each, or what 'violations[]' structure looks like in the response.