E2E test suite for nexus-agents MCP tools: orchestrate, research_catalog_review, registry_import
Three tools with partial schema definitions and moderate descriptions. orchestrate has a well-structured input schema with type constraints (maxIterations 1-50, timeout 1000-600000ms) and a 95-char description. research_catalog_review and registry_import have enum-constrained parameters and reasonable descriptions (80-90 chars), but output schemas are not visible in source code, only inferred from mock responses. No tool descriptions explain when to use them vs alternatives, prerequisites, or error recovery paths. Parameter descriptions are present but generic (e.g., 'Optional context data' lacks detail on expected structure). No tool declares permissions, audit requirements, or idempotency guarantees. Error handling is absent from tool definitions, only mock responses show error cases. Overall: definitions are functional but lack LLM-optimization depth.
Orchestrate agent execution with task decomposition, subtask tracking, and configurable iteration/timeout limits
Import AI model entries into the registry with dry-run support and validation warnings
Manage research catalog references with list, approve, dismiss, and flush actions
Output schemas not documented in tool definitions. Only mock responses visible; actual response structure must be inferred from fixtures, not from tool registration.
Parameter descriptions lack actionable constraints. 'Optional context data (record of unknown values)' does not explain what keys/types are expected. 'Reference identifier (optional, required for approve/dismiss)' is contradictory.
No error handling guidance in tool descriptions. Mocks show timeout, error, and failed states, but tool definitions do not explain recovery paths or when to retry.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Tool names lack clarity on composition. 'research_catalog_review' combines review + management (list/approve/dismiss/flush). Should split into separate tools: list_catalog_references, approve_reference, dismiss_reference, flush_catalog.
No idempotency or permission declarations. registry_import with dryRun=false persists data, but tool definition does not state if repeated calls are safe or if they require authorization.