Loading ToolBench
Executable evidence contracts for public software claims — CLI, TypeScript SDK, CI gate, and MCP server.
Proofspec has 4 tools with clear naming (verb_noun pattern: validate, check, explain, render) and reasonable descriptions (80-180 chars). All tools have input schemas with Zod validation. However, parameter descriptions are minimal or absent, most params lack detail about format, constraints, or when to use them. Output schemas are not documented. Error handling is basic (returns JSON with 'error' field for explain, but no recovery guidance). The server lacks tool annotations (readOnlyHint, destructiveHint) despite having clear risk profiles (proofspec_check is WRITE, others READ_ONLY). No pagination, no per-item error reporting, and responses are not shaped for agent chaining.
Evaluate every claim, append independently labelled evidence receipts, verify the ledger, and return the complete report.
Evaluate the specification and return one claim with its policy, dependencies, limitations, evidence, and receipts.
Evaluate a specification and render JSON, Markdown, standalone HTML, Mermaid, or SARIF.
Validate a Proofspec configuration, including unique IDs and an acyclic claim graph, without checking evidence.
Parameter descriptions are missing or trivial. The 'config' parameter in all tools lacks detail: no mention of file path format, environment variable fallback behavior, or what happens if the file is missing. The 'claim' parameter in proofspec_explain has no description of valid claim ID format or how to discover valid IDs.
Output schemas are not documented. Tools return JSON (via jsonResult helper) but LLMs cannot see the structure of returned fields. For example, proofspec_check returns a report object, what fields does it contain? What is the structure of claims array? This forces LLMs to guess and wastes tokens on exploration.
Tool annotations are missing. proofspec_check is destructive (WRITE risk, appends receipts to ledger) but has no destructiveHint. proofspec_validate, proofspec_explain, and proofspec_render are read-only but lack readOnlyHint. Agents cannot distinguish safe from unsafe tools without explicit annotations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Error handling lacks recovery guidance. proofspec_explain returns {error: 'not_found', claim} when a claim is not found, but does not suggest how to discover valid claim IDs or offer alternatives. Per pattern:recovery-guide, errors should tell the LLM what to do next.
No pagination or result limits documented. proofspec_check and proofspec_render return full reports without documented limits. If a report contains hundreds of claims, the response could exhaust context windows. No limit parameter, no pagination, no guidance on expected response size.