A LangGraph-based chatbot system with tool integration, featuring an MCP calculator server and various chatbot implementations (in-memory, database-backed, with tools). Includes sequential, conditional, iterative, and parallel workflow examples.
This server has 8 tools with basic schemas and descriptions, but significant quality gaps. Tool names lack consistent verb prefixes (calculator, add, subtract vs get_stock_price). Descriptions are present but minimal (10-50 chars for most arithmetic tools, below the 50-200 char LLM-optimized baseline). Parameter descriptions are generic ('First number', 'Second number'). No output schemas documented. Error handling is minimal, divide() mentions ValueError but doesn't guide recovery. The calculator tool duplicates functionality of add/subtract/multiply/divide/power/modulo, violating single-responsibility. Stock price tool embeds API key in code (security risk). No pagination, rate limiting, or result limits documented.
Return a + b.
Perform a basic arithmetic operation on two numbers. Supported operations: add, sub, mul, div
Return a / b. Raises ValueError if b is zero.
Fetch latest stock price for a given symbol (e.g. 'AAPL', 'TSLA') using Alpha Vantage with API key in the URL.
Return a % b.
Return a * b.
Return base ** exponent.
API key embedded in source code (get_stock_price). Credentials must never appear as tool parameters or hardcoded in code. Use server-side secret injection via environment variables.
Duplicate tool functionality: calculator() performs add/sub/mul/div via operation parameter, but separate add/subtract/multiply/divide tools also exist. This violates single-responsibility and forces LLMs to choose between redundant tools, wasting reasoning cycles.
Minimal parameter descriptions. 'First number' and 'Second number' lack context about valid ranges, precision, or constraints. LLMs cannot infer whether negative numbers, decimals, or very large values are acceptable.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
Return a - b.
No output schemas documented. Callers cannot predict response structure (e.g., does divide() return {result: float} or {result: float, remainder: float}?). This forces LLMs to guess and wastes tokens on parsing.
Error handling lacks recovery guidance. divide() mentions 'Raises ValueError if b is zero' but doesn't tell the LLM what to do next. Should return: 'Division by zero. Provide a non-zero divisor and retry.'