A multi-agent AI system with intelligent routing, tool integration, and chat capabilities built on FastAPI and LangChain
FastAgent exposes 5 tools with basic schemas and descriptions, but significant gaps limit production readiness. Tool names lack consistent verb prefixes (map_search, route_planning, trending_search are acceptable; mcp_tool is generic). Descriptions exist but are minimal (10-50 chars) and lack context on when/why to use each tool. Parameter descriptions are present but sparse. database_query has good input validation (SQL injection checks, SELECT-only enforcement) but no documented output schema. mcp_tool is a meta-tool that proxies to other MCP servers, unusual and risky. Error handling is basic (try/catch with string returns). No tool annotations (readOnlyHint, destructiveHint). Overall: functional but below production baseline.
Execute SQL SELECT queries against a database with parameter support and result limiting
Search for location information by place name
Execute MCP tools by name with parameters
Plan a route between two locations with distance and duration estimates
Search GitHub trending open source projects with language and time range filtering
mcp_tool is a generic meta-tool that proxies to arbitrary MCP servers. This violates single-responsibility principle and creates security/composition risks. LLMs cannot reason about what mcp_tool does without knowing the target server's capabilities.
Tool descriptions are too brief (10-50 chars). Baseline for A+ tools is 194 chars. map_search ('Search for location information by place name') and route_planning ('Plan a route between two locations...') lack context on when to use them vs alternatives, prerequisites, or return structure.
No output schemas documented. database_query returns formatted JSON rows but LLMs cannot see the structure. map_search and route_planning return free-text strings instead of structured objects. This forces LLMs to parse unstructured output, wasting tokens and increasing errors.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | <=2025-11-25 | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are marked READ_ONLY or WRITE in metadata but this is not exposed in MCP tool definitions. LLMs cannot determine which tools are safe to retry.
Error handling returns raw exception strings ('Error executing query: ...') instead of actionable recovery guidance. Per pattern:recovery-guide, errors should tell the LLM what to do next (retry, ask user, or fail).