An intelligent agentic system that integrates Gmail, GitHub, and OpenAI to process natural language queries and execute actions across multiple services
CustomMCP has 9 tools with basic naming and descriptions, but critical gaps in schema documentation, parameter descriptions, and error handling. Tool names follow verb_noun convention (search_gmail, get_email_content), which is good. However, input schemas are inferred from the provided metadata rather than visible in actual code, no explicit JSON Schema definitions found in src/services/agent.py or src/api/mcp.py. Descriptions are present but generic (e.g., 'Search for emails in Gmail' is 26 chars, below the 50-200 char LLM-optimized baseline). Parameter descriptions exist but lack constraints: 'max_results' has no min/max bounds, 'state' in get_repo_issues lacks an enum, 'path' in get_repo_contents lacks format guidance. No output schemas documented. Error handling and recovery guidance absent. The process_query tool is particularly problematic, it conflates multiple concerns (understanding, routing, execution) into one tool, violating single-responsibility principle.
Get the full content of an email
Get security alerts for a repository
Get contents of a specific path in a repository
Get issues for a repository
Get the file structure of a repository
Get the status of all integrated services (Gmail, GitHub, OpenAI)
Process a natural language query to the MCP system. The system will use OpenAI to understand the query, decide which services to use (Gmail, GitHub, etc.), execute the necessary actions, and return a comprehensive response
No explicit JSON Schema definitions visible in source code. Input schemas inferred from metadata only; cannot verify actual parameter types, constraints, or required fields.
Parameter descriptions lack constraints. 'max_results' has no min/max bounds (baseline: 1 - 100). 'state' in get_repo_issues lacks enum declaration (should be: open|closed|all). 'path' in get_repo_contents lacks format guidance. LLMs cannot infer valid ranges and will pass invalid values.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 35 | <=2025-11-25 | v2 |
Search for repositories on GitHub
Search for emails in Gmail
Tool descriptions are generic and under 50 characters (baseline: 50 - 200 chars for LLM optimization). E.g., 'Search for emails in Gmail' (26 chars) does not explain WHEN to use it vs other tools, what it returns, or dependencies. Agents cannot distinguish this from similar tools.
process_query violates single-responsibility principle. It combines understanding (OpenAI), routing (service selection), and execution (calling Gmail/GitHub) into one tool. This should be split: process_query should only route; individual service tools should execute. Agents cannot compose or retry individual steps.
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., email_id from search_gmail to pass to get_email_content). Responses are unstructured, forcing agents to parse free-text output.