TypeScript briefing agent that combines model-driven tool use, an MCP server, and a lightweight eval harness.
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. search_briefs has a reasonable description (71 chars) and typed parameters with constraints (minLength, min/max). get_brief has a minimal description (35 chars, below 50-char baseline) and single required parameter. Neither tool documents output schema structure, limiting LLM ability to chain results. No error handling guidance. Parameter descriptions are present but generic. No tool annotations (readOnlyHint, idempotentHint) despite both being read-only operations.
Load a full briefing document by id.
Find relevant briefing documents by topic, technology, or market signal.
Output schemas not documented. LLMs cannot infer what fields search_briefs returns (e.g., does it return id, title, summary, relevance_score?) or what structure get_brief yields. This blocks downstream tool chaining and forces LLMs to guess field names.
get_brief description is 35 characters ('Load a full briefing document by id.'), below the 50-char minimum for LLM clarity. Does not explain when to call it vs search_briefs, what format the id must be, or what the response contains.
No tool annotations despite both tools being read-only. Missing readOnlyHint would help LLMs understand these are safe to retry and do not modify state.
search_briefs 'limit' parameter is optional with no default stated. LLMs may omit it, causing unpredictable result counts. Should document default (e.g., 'defaults to 5') or make it required with a sensible default.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
No error handling guidance. If search_briefs returns zero results or get_brief receives an invalid id, LLMs have no recovery path. Should document: 'If no results, try a broader query' or 'If id not found, call search_briefs first.'