AI-powered open source license compliance agent. Scans repos, resolves licenses, analyzes usage context, and generates actionable compliance reports.
Comply OSS has well-structured tool definitions with clear descriptions and proper input schemas using Zod. All 4 tools are explicitly registered with descriptions (194 - 250 chars, within baseline 34 - 392 range). Input parameters have types and descriptions. However, output schemas are not documented, responses are JSON-stringified without explicit field documentation, forcing LLMs to infer structure. Error handling is minimal (generic 'Scan failed' messages). Tool names follow verb_noun convention (comply_scan, comply_explain_license, comply_diff, comply_policy) but lack action clarity, 'comply_scan' could be clearer as 'scan_repository' or 'audit_licenses'. Parameter descriptions are adequate but lack constraint details (e.g., no min/max for arrays, no format specs for paths). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all tools being read-only.
Show compliance changes since the last scan. Identifies new dependencies, removed dependencies, license changes, and new/resolved violations. Useful for reviewing what changed in a PR or after dependency updates.
Explain what a specific SPDX license means for compliance. Returns the license tier, whether it has copyleft obligations, attribution requirements, and how it interacts with different distribution models (SaaS, distributed, internal, library).
Show the current compliance policy configuration. Returns the distribution model, license rules (allow/deny/review), severity levels, and any allowlisted or denylisted packages.
Scan a repository for open source license compliance. Returns structured data about all dependencies, their licenses, policy evaluations, and flagged packages that need attention. Use this to understand the compliance posture of a codebase.
Output schemas not documented. Tools return JSON-stringified objects without explicit field definitions. LLMs cannot plan downstream operations or extract specific fields reliably.
Error handling lacks recovery guidance. Generic 'Scan failed: <message>' does not tell LLM what to do next (retry, ask user, check inputs). No error categorization (retryable vs fatal).
Tool names lack action clarity. 'comply_scan' is domain-specific but ambiguous, does it scan for compliance, scan licenses, or scan dependencies? 'scan_repository' or 'audit_licenses' would be clearer.
No tool annotations despite all tools being read-only. Missing readOnlyHint annotations prevent clients from optimizing caching and retry logic.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2025-06-18+ | v2 |
Parameter constraints underspecified. 'ecosystems' array lacks min/max length; 'path' and 'policy' lack format/pattern guidance. LLMs may pass invalid paths or oversized arrays.