Loading ToolBench
MCP server for pipeline orchestration, story planning, and decision escalation
Three tools with complete input schemas and descriptions, but descriptions are terse (avg 156 chars, below the 194-char baseline). All parameters have types and descriptions. No output schemas documented. Tools follow single-responsibility principle (escalate_decision, classify_oracle_outcome, acceptance_digests are distinct). Naming is clear and verb-forward. However, descriptions lack context on WHEN to use each tool, prerequisites, and what the LLM should expect in return. Error handling guidance is absent.
Map each acceptance entry's path to a sha256 hex digest of its authoritative manifest ``source`` (never of the file on disk, which could already have been rewritten). Returns {} when the story has no acceptance entries.
Classify a test-runner outcome as "passes", "empty", "errors", or "fails_correctly" per the acceptance-oracle pattern. ``story`` is accepted for call-shape compatibility with a dispatching harness but is not part of the classification - the oracle grades the fixture run (returncode + output), never the story manifest.
Escalate a blocking decision to the overlord, which rules on the user's behalf per the decision policy. Assembles the same decision-prompt shape the main pipeline server's request_decision tool uses (question, enumerated options, context, decision policy) and returns the overlord's ruling text. Call this when you are blocked on a choice the user would normally make, in a harness that has adopted only this companion server.
Output schemas not documented. LLMs cannot infer what fields to expect from each tool's response, forcing them to guess downstream field names and risk broken tool chains.
Descriptions lack WHEN/WHY context. 'Escalate a blocking decision' tells WHAT but not WHEN to call it vs alternatives, or what the LLM should do with the returned ruling text.
No error handling guidance. If escalate_decision fails or classify_oracle_outcome returns an unexpected classification, the LLM has no recovery path.
Parameter 'story' in classify_oracle_outcome and acceptance_digests is marked 'not used in classification' but still required. Clarify whether it should be optional or remove it entirely.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | <=2025-11-25 | v2 |
classify_oracle_outcome description mentions specific enum values ('passes', 'empty', 'errors', 'fails_correctly') but the output schema does not declare them as an enum constraint, leaving the LLM to infer valid values.