An AI-powered assistant backend with FastAPI providing chat, authentication, Gmail integration, and Google Drive integration capabilities
Jarvis exhibits significant gaps in definition quality. While 6 tools are registered with basic descriptions and most have input schemas, the quality is inconsistent. Tool names mostly follow verb_noun convention, but descriptions are often generic (e.g., 'List resources from Google Drive' lacks context on when to use it). Parameter descriptions exist but are minimal and do not explain constraints, formats, or expected values. Output schemas are undocumented, no evidence of return type declarations. The 'list_tools' tool is a self-referential anti-pattern. Error handling is absent from visible code. The security posture is weak: no visible credential injection patterns, no permission gates, no audit trails. Only 1 of 6 tools (send_email) explicitly declares side effects.
Get weather information for a location
List resources from Google Drive
List available tools
Read a resource from Google Drive by URI
Search for files in Google Drive
Send an email using Gmail API
Missing output schema documentation. No tool describes what it returns, field types, or pagination structure. LLMs cannot plan downstream calls or extract required data.
'list_tools' is a self-referential anti-pattern. It duplicates the MCP discovery protocol and provides no value to agents. Remove it.
Parameter descriptions are minimal and lack constraint information. E.g., 'cursor' has no explanation of format, when pagination is needed, or what happens at the end. 'query' in search does not explain syntax, max length, or special characters.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 47 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Tool descriptions lack WHEN/WHY guidance. 'Read a resource from Google Drive by URI' does not explain when to use this vs search, or what a 'gdrive://' URI format contains. LLMs cannot decide which tool to call.
No error handling guidance in tool definitions or visible code. Errors are not categorized as retryable, user-fixable, or fatal. LLMs receive no recovery path.
No visible credential/secret injection patterns. If Google API credentials are passed as tool parameters (not shown in schema but likely in implementation), they would be logged and exposed.
send_email accepts 'to' as an array but no description explains valid email format, validation, or behavior on invalid addresses. No mention of dry-run or confirmation for irreversible operations.
get_weather_info uses oneOf for location parameter (string or [lat, lon array]). No description explains how the LLM chooses between them, what format 'city name' accepts, or what happens on ambiguous city names.
No pagination limits documented. list_resources accepts a cursor but no description of page size, max results, or result caps. Agents could request thousands of results and exhaust context.