Integration-readiness harness for AI agents: discovers MCP tools, risk-classifies them, runs safe probes, and produces a citation-backed readiness report.
Preflight is a demo/testing harness for MCP servers, not a production tool server itself. The 14 bundled demo tools have complete schemas and descriptions, but quality varies. Most tools follow verb_noun naming (list_tickets, get_customer, create_ticket). Descriptions are present and adequate (50-200 chars typical). Schemas include proper types and enums. However, tool #9 (ambiguous_operation) lacks a description entirely, and several tools lack parameter descriptions. Error handling is minimal, no recovery guidance or actionable error messages visible in schemas. The server demonstrates good schema structure but lacks production-grade error handling and some parameter documentation.
Create a new support ticket. Set dry_run to true to validate the request without persisting anything.
Permanently delete a project and all of its data. This cannot be undone and there is no dry-run mode.
Fetch a single customer record by identifier. Read-only; does not modify anything.
Return the nightly reliability report. Read-only. This tool is intentionally broken in the demo environment so that error telemetry has a real failure to record.
Return a report. Read-only, but deliberately slower than any probe timeout.
Return current service health. Read-only status probe with no side effects.
ambiguous_operation lacks description entirely (0 chars). Tool purpose is completely opaque to LLMs.
No error handling guidance in any tool schema. Errors do not indicate whether they are retryable, user-fixable, or fatal. LLMs cannot determine recovery strategy.
Destructive tools (delete_project, revoke_api_key) lack confirmation/dry-run parameters. Agents cannot validate intent before irreversible operations.
get_flaky_report and get_slow_report are intentionally broken for demo purposes. In production, tools must be reliable or clearly document failure modes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Return aggregate ticket counts by status and priority. Read-only; no side effects.
Return the current list of projects. Read-only; has no side effects.
Return support tickets, optionally filtered by status. Read-only.
Revoke an API key immediately. Existing integrations using it will stop working.
Search the document library by free-text query. Read-only; returns matches only.
Send an email notification to a customer contact. There is no preview mode: calling this tool always delivers the message.
Update a customer record. Set validate_only to true to check the change without applying it.
Output schemas not documented in tool definitions. LLMs cannot plan downstream calls or extract required fields without knowing response structure.