Permission-aware MCP server and agent harness with role-scoped tools, whole-plan validation, deterministic evals, and a React playground.
Strong foundation with 15 well-structured tools, comprehensive input schemas using Zod, and clear descriptions. Tool names follow verb_noun convention (list, get, create, update, resolve). All tools have descriptions (avg ~120 chars, within 10-1024 baseline). Input schemas are complete with type definitions and enums for constrained values (status, priority). Output schemas are documented via Zod. Role-based access control is properly gated. Key gaps: parameter descriptions are sparse (many lack inline docs explaining purpose/format), output schemas lack descriptions for response fields, and error handling lacks recovery guidance. No examples of actionable error messages in code review.
Add an internal note to a case. Supervisor only. Internal notes are never shown to customers.
Set the case assignee. Supervisor only. Resolved cases cannot be reassigned.
Open a new case on a contact. Starts as open, unassigned.
Fetch one case. Supervisors see internal notes. Other roles receive hiddenInternalNoteCount instead of the note bodies.
List ops cases, optionally filtered by status or contact.
Close a case with a resolution code and customer-safe summary. Supervisor only.
Update case fields including tags. Assignment and resolve are separate tools.
Parameter descriptions missing or minimal. 'query' in contacts.list has description, but most enum parameters (status, priority, resolution_code) lack inline guidance on valid values or use cases. LLMs cannot infer when to use 'p1' vs 'p2' without explicit context.
Output schemas documented via Zod but lack field-level descriptions. Response objects (e.g., contact, case, task) have no inline docs explaining what each field means or when it's populated. LLMs must infer structure from field names alone.
Error handling lacks recovery guidance. Code shows try/catch with fromUnknown(error), but no evidence of actionable error messages. LLMs receive raw errors without hints on retry eligibility, user-fixable issues, or next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2025-06-18+ | v2 |
Fetch one contact by id.
Search Harborline contacts by name, email, company, or id.
Pause or reactivate a contact. Supervisor only. Operators cannot freeze an account from the tool catalog.
Mark a task as done. Completed tasks cannot be reopened.
Create a follow-up task on a case.
Fetch one task by id.
List follow-up tasks, optionally filtered by status or case.
Return the connected Harborline principal, role, and the tool names this role may call.
Irreversible operations (cases.resolve, tasks.complete) lack confirmation or dry-run support. Agents can permanently close cases or mark tasks done without a safety gate. No evidence of confirmation_before_execute pattern.
List tools (contacts.list, cases.list, tasks.list) lack pagination parameters (limit, offset, cursor). No evidence of page size constraints or total count in responses. Large result sets could exhaust context windows.