Loading ToolBench
Agent Interface Compiler — converts OpenAPI specs into MCP servers and multi-agent interface formats (Claude.md, Agents.md, Cursor Rules, llms.txt, Gemini.md, A2A, CLI, Skills)
Agentify exposes 3 MCP tools that function as wrappers around a CLI compiler. All tools are defined in src/cli.ts with Commander.js. While schemas are visible in the command options, they lack proper JSON Schema type specifications and comprehensive parameter documentation. Tool descriptions are present but generic. Error handling is minimal, failures result in process.exit(1) with basic error messages rather than structured recovery guidance. The tools operate on file system state (WRITE risk on transform and self-describe) without documented safeguards or permission gates. Overall structure suggests proof-of-concept tooling rather than production-grade agent interfaces.
Parse an OpenAPI spec and output the IR as JSON to stdout
Output Agentify's own agent interface files (agentify.md skill, CLAUDE.md, AGENTS.md)
Transform an OpenAPI spec into agent interfaces
Input schemas lack formal JSON Schema type and format declarations. Commander.js option definitions (e.g., string, boolean, array) are visible but not exposed as proper JSON Schema with minLength, maxLength, enum, pattern, or default constraints. LLMs cannot validate input constraints from option objects alone.
Parameter descriptions are minimal or absent. 'output' param: 'Output directory' (24 chars), lacks guidance on path format, whether relative/absolute, or creation behavior. 'format' param in transform: no description of what each format produces or when to use each. Rubric baseline for param annotation: 72 chars average; these fall short.
Tool descriptions are generic and lack action guidance. 'transform' (78 chars): 'Transform an OpenAPI spec into agent interfaces' does not explain WHEN to call it vs parse, WHAT happens on success, or HOW to interpret the output. Rubric baseline tool description: 194 chars average. These are 1/2 to 1/3 that length.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | 2026-07-28+ | v2 |
No output schema documentation. Tools return EmitterResult[], but the structure (success/failure fields, file paths, error details, warnings) is not documented in tool descriptions. LLMs cannot plan downstream calls or extract specific fields without a documented schema.
Error handling offers no recovery guidance. src/cli.ts lines with parseSpinner.fail() and process.exit(1) produce generic messages like 'Failed to parse OpenAPI spec' without suggesting next steps (e.g., 'Check that the spec is valid JSON/YAML' or 'Use parse tool to debug the spec'). Agents receive no actionable direction.
transform tool combines multiple distinct operations: parsing + generating multiple output formats (mcp, claude.md, agents.md, etc.). This violates single-responsibility principle. Agents cannot independently choose to generate only claude.md without also generating mcp. Should split into separate tools or use an action enum.
No permission gates or safeguards on destructive operations. transform writes to output directory; self-describe also writes. No documented checks for user authority, no confirmation step, no undo capability. If an agent is compromised, it can overwrite file systems without constraint.
Ambiguous parameter names and missing type guidance. 'input' param in transform and parse: description says 'Path or URL to OpenAPI specification' but does not specify format validation, size limits, or error handling for invalid URLs. 'name' param in transform: is this the project name in generated code, or the output directory name? Unclear.
Tool names begin with verbs but lack specificity. 'transform' is vague, does it mean convert format, generate code, translate, or refactor? More explicit names like 'compile_openapi_to_mcp' or 'generate_agent_interfaces' would disambiguate intent. 'self-describe' is jargon; 'export_agentify_schemas' would be clearer.