A multi-agent research system that combines web search and Odoo ERP data to generate comprehensive research reports with self-reflection quality evaluation loops
Server has 5 tools with basic schemas and descriptions, but significant gaps prevent higher scores. All tools have descriptions (10-80 chars, mostly adequate) and input schemas with types. However, output schemas are completely undocumented, LLMs cannot plan downstream calls or extract structured data. Error handling returns JSON errors but lacks recovery guidance. No tool annotations (readOnlyHint, idempotentHint). Parameter descriptions are minimal (e.g., 'Product name filter (optional)' lacks format/constraint details). Tools return JSON strings rather than structured objects, forcing LLM parsing. No pagination guidance despite tools accepting 'limit' parameters. Naming is verb-noun compliant but generic ('query_*', 'web_*'). Security: credentials injected via environment variables (good), but no audit logging or permission gates visible.
Query customer details from Odoo ERP.
Query inventory stock levels from Odoo ERP.
Query sales orders and revenue metrics from Odoo ERP.
Fetch and extract readable plain text from a web URL.
Search the web for information given a search query.
Output schemas completely undocumented. Tools return JSON strings (e.g., query_sales returns JSON array of {order_name, date, customer, total_amount}) but LLMs have no schema to plan downstream calls or validate structure. Breaks tool chaining.
Parameter descriptions lack format/constraint details. 'Product name filter (optional)' does not specify case sensitivity, partial match behavior, or max length. 'Search query string' does not document max length or special character handling. Invites invalid LLM input.
No error recovery guidance. Functions return JSON errors like {"error": "Odoo server not connected..."} but do not tell LLM what to do next (retry? check config? call different tool?). Breaks agent error handling.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
No tool annotations. Tools lack readOnlyHint, idempotentHint, or destructiveHint. LLMs cannot determine if tools are safe to retry or have side effects. All 5 tools are read-only but this is not declared.
Pagination not documented. query_* tools accept 'limit' parameter but no mention of offset, cursor, or total_count in response. Large result sets will blow context window without guidance.