Windlass: an MCP server that runs Claude Code sessions in a loop against machine-checkable success criteria, with retries, timeouts, and an audit log
Windlass defines 3 tools with explicit schemas and descriptions. All tools have non-empty descriptions (10-200 chars range) and input parameters with type definitions. However, output schemas are not documented, responses are JSON-stringified text without declared structure. Parameter descriptions are minimal (e.g., 'Task name' for 'name' param). Error handling returns isError=true with text messages but lacks recovery guidance or categorization. No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Tool names follow verb_noun pattern (run_task, check_task, list_tasks) but descriptions could better explain WHEN to use each tool vs alternatives.
Check the status of a task execution by ID.
List all running and recent task executions.
Run a Claude Code task with success criteria and automatic retry loops. Provide a task definition inline or a path to a YAML/JSON definition file.
Output schemas not documented. Tools return JSON-stringified text without declaring field types, structure, or required fields. LLMs cannot plan downstream calls or extract data reliably.
Parameter descriptions are minimal and lack format/constraint details. E.g., 'Task name' does not explain length limits, character restrictions, or uniqueness requirements. 'Task type' is ambiguous, what are valid types?
Error handling lacks recovery guidance and categorization. Errors return isError=true with plain text but do not tell the LLM whether to retry, ask the user, or abort. No actionable next steps.
No tool annotations. Tools lack readOnlyHint, destructiveHint, or idempotentHint. run_task is destructive (executes code) but not marked; check_task and list_tasks are read-only but not annotated.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
run_task accepts inline task definitions OR file paths but does not explain when to use each. Description should clarify: 'Pass a task object for simple cases; pass a file path for complex pipelines with multiple tasks.'