A workflow orchestration and execution engine that supports LLM-powered graphs, API tools, MCP (Model Context Protocol) integration, and multi-step automation
Brockley AI exposes only 2 introspection tools (_list_api_tools, _describe_api_tool) that are meta-tools for discovering other API tools dynamically. Both tools have adequate descriptions (60-90 chars) and properly typed input schemas with required fields. However, the actual API tools being discovered are NOT visible in the provided source code, only the introspection mechanism is shown. This means we cannot evaluate the quality of the tools that agents will actually use. The two visible tools themselves are narrow in scope (read-only discovery), lack output schema documentation, and lack error handling guidance. Since only introspection tools are visible and the real tool definitions are dynamic/hidden, the overall definition quality score reflects only these two meta-tools, which are functional but minimal.
Describe a specific endpoint of an API tool, including its full input schema, HTTP method, path, and description.
List the available endpoints for a given API tool. Returns an array of endpoint names that can be passed to _describe_api_tool for full schema details.
Real tool definitions are dynamically discovered and not visible in source code. Cannot evaluate the quality of actual agent-facing tools, only introspection mechanism is visible.
No output schema documentation for either introspection tool. LLMs cannot predict what _list_api_tools or _describe_api_tool return, forcing them to guess at response structure.
No error handling guidance. If api_tool_id is invalid or endpoint does not exist, no recovery instructions provided to the LLM.
Tool descriptions lack context on WHEN to use them. No guidance on whether to call _list_api_tools before _describe_api_tool, or how they fit into agent workflows.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2025-06-18+ | v2 |
Parameter descriptions are minimal (single phrase). No format constraints, examples, or validation rules documented for api_tool_id or endpoint parameters.