A comprehensive data pipeline and AI agent orchestration platform with RAG capabilities, assistant agents, and MCP server integration for knowledge retrieval
DataPilotFlow exposes 7 tools via fastmcp/HTTP with complete input schemas and descriptions. Naming follows verb_noun convention (create_tool, list_tools, get_tool, update_tool, delete_tool, generate_instructions, generate_assistant_instructions). All tools have descriptions (range 50 - 150 chars) and typed parameters with descriptions. However, output schemas are not documented in the source code provided, critical for LLM planning. Error handling guidance is absent. Parameter descriptions lack constraint details (e.g., valid enum values for tool_type, limits on array sizes). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (WRITE, DESTRUCTIVE). Composition is sound, tools are single-responsibility and chainable (create_tool → list_tools → get_tool → update_tool → delete_tool). Security: no visible secrets in parameters, but no audit logging or permission gates documented.
Create a new tool (prompt-based or MCP remote)
Delete a tool by ID
Generate complete assistant instructions based on persona, personality, and agent type using an LLM
Generate tool orchestration instructions using an LLM based on selected tools
Retrieve a specific tool by ID
List all tools for the current user with optional filtering
Update an existing tool
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., tool_id returned by create_tool, fields in list_tools response). Breaks tool chaining.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite explicit risk levels. delete_tool marked DESTRUCTIVE but no annotation to warn LLM. update_tool marked WRITE but no hint. Agents cannot reason about retry safety.
Parameter descriptions lack constraint details. 'tool_type' accepts 'prompt_based' or 'mcp_remote' but description does not enumerate valid values. 'tags' is an array but no min/max length specified. LLMs may pass invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
No error handling guidance. If create_tool fails (e.g., duplicate name, invalid tool_type), no recovery hint provided. LLM cannot self-correct or suggest alternatives.
generate_instructions and generate_assistant_instructions accept LLM provider/model parameters but no validation or error guidance if provider is unavailable. No timeout or fallback documented.