A Next.js chat application that integrates MCP (Model Context Protocol) tools with OpenAI's language models via the Vercel AI SDK
This server has critical definition gaps. Both tools lack proper parameter descriptions, the 'location' and 'temperature' parameters have only type info, no guidance on format or constraints. Tool descriptions are minimal (18-27 chars, well below the 50-200 char LLM-optimized baseline). No output schemas are documented, so LLMs cannot predict what fields to expect. No error handling guidance. The naming is acceptable (verb_noun pattern: 'weather', 'convertFahrenheitToCelsius'), but the overall definition quality is far below production standards.
Convert a temperature in fahrenheit to celsius
Get the weather in a location (fahrenheit)
Parameter descriptions missing or trivial. 'location' and 'temperature' have no guidance on format, constraints, or examples. LLMs cannot infer whether 'location' accepts 'New York', 'NYC', coordinates, or zip codes.
Tool descriptions are too short (18-27 chars). Baseline for LLM-optimized descriptions is 50-200 chars. Current descriptions lack context on when to use each tool or what they return.
No output schemas documented. LLMs cannot predict return structure (e.g., does weather return {temp, condition, humidity} or just a string?). This forces guessing and downstream errors.
No error handling guidance. If weather API fails or location is invalid, LLMs have no recovery path. Error responses should guide the agent on retry vs. user correction.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 26 | 2026-07-28+ | v2 |
Transport mechanism unknown. Source shows Next.js API route (app/api/chat/route.ts) but no explicit MCP server registration, transport binding, or protocol handshake visible. Cannot verify HTTP/SSE/STDIO compliance.