Multi-agent document analysis system with CrewAI + MCP (Model Context Protocol) for intelligent document processing, combining filesystem and database MCP servers with FastAPI backend
Server has 12 tools with basic schemas and descriptions, but significant quality gaps prevent production use. Tool naming is mostly verb-based (query_, get_, store_, read_, search_, extract_, analyze_, generate_) but lacks clarity in distinguishing overlapping operations (three 'read_document' tools with identical names). Descriptions are present but generic (10-50 chars), below the 50-200 char LLM-optimized baseline. Parameter descriptions are sparse or missing entirely. Output schemas are undocumented, no guidance on what fields agents should expect. Error handling is absent; no recovery guidance or actionable error messages. Security concerns: no evidence of input validation, SQL injection protection, or secret injection patterns. Composition issues: duplicate tool names (read_document appears 3 times), unclear tool boundaries (query_database vs query_documents), and no pagination support despite database queries.
Analyze the sentiment of text (positive/negative/neutral)
Extract top keywords from text with frequency and relevance scores
Generate an extractive summary of text
Get analysis results for a document
Get a specific document by ID
Query the document database for metadata and previous analyses
Query documents by type or status
Read the contents of a document file
Duplicate tool names: 'read_document' appears 3 times (database_server.py, filesystem_server.py x2). LLMs cannot distinguish between them; tool selection becomes ambiguous and error-prone.
Parameter descriptions missing or trivial. 'uri' in read_document has minimal guidance; 'query' in query_database lacks context. LLMs cannot infer parameter meaning from names alone (e.g., is 'status' HTTP status or document status?).
Output schemas undocumented. Tools return JSON strings (e.g., 'return json.dumps([dict(r) for r in rows])') but agents have no schema to parse results. Missing field documentation forces LLMs to guess structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
Read a document's content from the filesystem via MCP
Search documents containing a keyword
Store analysis results
Store a new document in the database
No pagination support. query_documents accepts 'limit' but no offset/cursor. Large result sets will blow context windows; no guidance on handling >100 results.
No error handling or recovery guidance. SQL errors, missing documents, and invalid inputs return raw strings ('Document not found', 'No analysis found') with no actionable next steps for the agent.