Harness-neutral, evidence-first company research with a completed-only dossier viewer
20 tools with complete schemas and descriptions, but significant quality gaps. Tool names are action-oriented (discover_, get_, create_, start_, append_, commit_, pause_, resume_, list_, export_, launch_, record_, evaluate_, prepare_, import_) which is good. However, descriptions are often domain-specific jargon-heavy ('durable stage boundary', 'leakage-controlled binary calibration cohort', 'sanitized live stage/tool observations') that may confuse LLMs about when to invoke tools. Parameter descriptions are minimal or absent for complex objects like 'request', 'payload', 'receipts'. Output schemas are not documented in the visible code. Error handling and recovery guidance are not evident. The server appears to be a sophisticated research workflow coordinator, but the tool definitions prioritize internal domain semantics over LLM-friendly clarity.
Append sanitized live stage/tool observations without raw prompts, arguments, or credentials.
Validate and checkpoint one completed analytics stage, then return the next stage contract.
Create a durable 26-stage analytics run for execution by the calling harness.
Discover the analytics workflow, tools, and safety boundaries.
Evaluate one strict leakage-controlled binary calibration cohort.
Export a completed research run as a portable bundle.
Return the result, events, and presentation for one completed research run.
Complex object parameters ('request', 'payload', 'receipts') lack nested schema documentation. LLMs cannot infer the structure of these objects, forcing trial-and-error invocation.
Descriptions use domain jargon ('durable stage boundary', 'leakage-controlled binary calibration cohort', 'sanitized live stage/tool observations') without explaining when or why an LLM should invoke the tool. Descriptions should answer: What does it do? When to use it? What does it return?
No documented output schemas visible in source code. LLMs cannot plan downstream tool calls or extract required fields without knowing what each tool returns.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Return supported, optional, and intentionally unavailable features.
Return redacted, read-only durable-state health for this server runtime.
Return the presentation-safe summary for a completed research run.
Return registered forecasts, append-only outcomes, and deterministic scorecards.
Return the durable control record for one analytics run.
Validate and atomically publish completed research plus analytics sidecars.
Launch the completed-only Research Dossier Viewer on loopback.
Return a paginated list of completed research runs.
Pause a running workflow at its next durable stage boundary.
Return the 26-stage analytics plan and standalone v1 submission schema.
Append a resolved forecast outcome or correction and persist its scorecard.
Resume at the first incomplete stage; interrupted in-flight work is replayed.
Start a prepared durable analytics run and return its first stage contract and context.
No error handling or recovery guidance visible. Tools do not indicate what errors are retryable, user-fixable, or fatal. LLMs cannot self-correct on failure.
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are defined in code but not visible in tool registration. Verify they are being set on all tools via ToolAnnotations.