Does the tool do what it says? A conformance harness for MCP servers, and an MCP server that reports what it found.
Four tools with clear, actionable descriptions (avg 156 chars, within baseline 194). All have input schemas with proper types and enums. However, naming lacks action verbs (scope, status, inspect_definition, check_now are descriptive but not verb-noun patterns). Parameter descriptions are present but minimal. No output schemas documented. Error handling guidance absent. Tools are read-only (low risk) but lack recovery patterns.
Actually run a package in a sandbox and report what it did. Slow -- minutes, not seconds -- and needs Docker. Refuses rather than running unsandboxed.
Hash a tool definition and read its wording. Takes the tool's JSON definition. Does no I/O and takes no network. Static analysis only -- it catches obvious instruction-injection wording, never careful wording.
What SayDo can and cannot settle. CALL THIS FIRST. Takes no arguments and does no I/O.
Whether a package has a SayDo conformance receipt. Takes a package URL such as pkg:npm/some-server or pkg:pypi/some-server. Returns 'unknown' when nobody has checked it, which is the usual answer.
Tool names lack action verbs. 'scope', 'status', 'inspect_definition', 'check_now' are descriptive but not verb-noun patterns (get_scope, check_status, analyze_definition, run_check). LLMs infer intent from verb-first names; these require reading descriptions.
No output schemas documented. Tools return results but LLMs cannot plan downstream calls or extract fields without seeing response structure. E.g., check_now returns findings but structure is undocumented.
No error handling guidance. Tools may fail (e.g., check_now timeout, Docker unavailable, package not found) but descriptions do not explain recovery steps or error categories (retryable vs fatal).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2025-06-18+ | v2 |
Parameter descriptions are minimal. 'purl' in status tool lacks format guidance (e.g., 'Package URL in format pkg:npm/name or pkg:pypi/name'). 'definition' in inspect_definition lacks structure hints.
check_now tool has destructive side effects (runs untrusted code in sandbox, may timeout/hang) but lacks confirmation or dry-run pattern. No idempotent/destructive hints in schema.