Loading ToolBench
An agentic RAG (Retrieval-Augmented Generation) platform with document ingestion, vector search, chat, podcast generation, and Gmail integration
This is a Next.js application that exposes server actions as tools, NOT a proper MCP server. The codebase shows a RAG system with document ingestion, embedding, and content generation features, but lacks foundational MCP protocol compliance. Tools are defined as Next.js server actions in app/actions.ts without explicit MCP schema registration, tool metadata structures, or protocol transport. The tool definitions visible in the evaluation are descriptions extracted from source comments, not from formal MCP tool registrations. Critical issues: (1) No input parameter schemas are visible in the source code, tools like generateEmbedding, connectGmail, and others lack documented JSON Schema input/output specifications. (2) Descriptions are present but minimal (50-150 chars), lacking WHEN to use, prerequisites, or expected output format. (3) Parameters like 'documentIds' in generatePodcast and generateStory lack type constraints, ranges, or validation rules. (4) No explicit error handling guidance is visible, tools do not document retryability, user-fixable vs fatal errors, or recovery paths. (5) No output schemas are documented; it's unclear what shape these tools return. (6) OAuth flow (connectGmail → handleOAuthCallback → ingestGmail) lacks coordination hints and error recovery patterns. (7) The server appears to be a Next.js application, not an MCP transport implementation, transport layer is unknown.
Call an MCP tool by name with provided arguments
Initiate Gmail OAuth connection flow by redirecting to Google authorization URL
Generate vector embeddings for text content using the Supabase edge function 'embed' which uses Xenova transformers with the gte-small model
Generate a podcast script from selected documents and synthesize audio using text-to-speech, then upload to storage
Generate a first-person narrative story script from selected documents and synthesize audio, then upload to storage
Handle OAuth callback from Google, exchange authorization code for tokens, and store in integrations table
Fetch emails from Gmail, create document records, and vectorize email content for RAG
No input parameter schemas visible in source code. Tools lack documented JSON Schema with type definitions, constraints (min/max, enums), or parameter descriptions. Hardening rule: if no input schema is visible, schema score must be 0.
No output/return schemas documented. It is unclear what data structure, fields, or types each tool returns. This forces LLMs to guess at downstream field names and breaks tool chaining (e.g., generateEmbedding must return 'embedding' or 'vector', but this is not specified).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 26 | 2026-07-28+ | v2 |
List available MCP tools
Missing parameter descriptions for documented inputs. 'documentIds' in generatePodcast and generateStory lack description of what each ID refers to (Supabase document IDs?), whether duplicates are allowed, or constraints on array length.
OAuth flow coordination lacking. connectGmail, handleOAuthCallback, and ingestGmail form a three-step flow, but there is no documented error recovery, state management, or timeout handling. If handleOAuthCallback fails, there is no guidance on recovery.
Ambiguous tool naming. 'callTool' is generic and vague, it does not clearly signal what happens (does it execute, introspect, validate, or proxy to another MCP server?). Per naming rules, names should start with specific action verbs (execute_, invoke_, proxy_).
No error handling guidance. Tools do not document what errors can occur, whether they are retryable, or what the LLM should do next. E.g., generateEmbedding calling Supabase edge function could fail with network timeout, auth error, or model error, none are mentioned.
Transport layer unknown. Source code shows a Next.js application (package.json with next@16.1.1), but it is unclear whether this implements MCP via HTTP/SSE transport, STDIO, or something else. @modelcontextprotocol/sdk is imported but not used in the visible code.
Parameter type ambiguity. 'code' in handleOAuthCallback lacks format specification (is it a base64 string? Plain alphanumeric?). 'args' in callTool is generic 'object' with no schema for what structure is expected.