FastAPI + RAG + MCP + RBAC demo service for donor analytics with campaign performance tracking, donor retention analysis, and RAG-based document querying
Three tools with basic schemas and minimal descriptions. All tools have input schemas with type definitions, but descriptions are generic (10-50 chars) and lack context for LLM selection. No parameter descriptions beyond the bare minimum. No output schemas documented. Error handling is minimal (ValueError for empty inputs only). Tools lack guidance on when to use them vs alternatives, dependencies, or recovery paths. No tool annotations (readOnlyHint, etc.). Naming is reasonable (verb_noun pattern) but descriptions fail to explain WHAT, WHEN, or WHY.
Ask questions using RAG over internal docs.
Get campaign performance for a client.
Get donor retention for a client.
Descriptions are too short (10-50 chars) and lack context. 'Get campaign performance for a client' does not explain WHEN to use this vs donor_retention, what fields are returned, or what 'performance' means (revenue? engagement? ROI?).
No output schemas documented. LLMs cannot plan downstream calls or extract fields. campaign_performance returns {client_id, impressions, clicks, donations, revenue} but this is inferred from code, not declared in the tool definition.
Parameter 'client_id' and 'question' lack descriptions. Rubric requires every parameter to have a non-empty description explaining what it controls. 'client_id' could mean account ID, organization ID, or donor ID, the description must clarify.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All three tools are read-only but this is not declared in the schema. Agents cannot distinguish safe tools from destructive ones without explicit hints.
Error handling is minimal. ValueError for empty inputs only. No recovery guidance (e.g., 'If client not found, try search_clients() first'). No categorization of errors as retryable vs user-fixable. Agents cannot self-correct.