Loading ToolBench
An AI-powered product shopping assistant chatbot with LangGraph-based agentic workflows, MongoDB integration, and Razorpay payment processing. Handles product search, recommendations, inquiries, and checkout via MCP connectors.
Server has 13 tools with basic descriptions but critical gaps in schema documentation, parameter typing, and error handling. Most tools lack visible input schema definitions in the source code, only parameter names and descriptions are provided without formal JSON Schema type declarations. Descriptions are present but generic (avg ~120 chars), lacking LLM-optimized guidance on WHEN to use each tool vs. alternatives. No output schemas documented. Error handling is minimal, no recovery guidance, no categorization of retryable vs. fatal errors. Tool naming is reasonable (verb_noun pattern) but composition issues exist: classify_intent, info_gatherer, query_generator, query_executor, and reranker form a rigid pipeline that cannot be composed flexibly by an agent. Destructive tool (delete_history) lacks confirmation pattern. No tool annotations (readOnlyHint/destructiveHint) visible in code.
Main chat endpoint. Accepts user message, routes through LangGraph agent pipeline (intent classification → info gathering → query generation/execution → reranking or inquiry resolution), streams response tokens via Server-Sent Events, saves messages to MongoDB, and returns final response with top products.
Classify the user's latest message into one of three categories: 'buy' (product purchase/search), 'inquiry' (product information questions), or 'general' (non-product related). Uses LLM with deterministic fallback based on keyword patterns.
Delete all chat history and checkpoints for a thread from MongoDB (checkpoints, checkpoint_writes, Chat_History collections).
Generate a short 2-3 word chat title from user message using LLM with fast fallback. Used for dynamic conversation naming in UI.
Retrieve chat message history for a thread from MongoDB Chat_History, with fallback to LangGraph checkpoint state. Returns deduplicated messages and top products.
Input schemas lack formal JSON Schema type declarations. Parameters are described but not formally typed (no 'type', 'format', 'enum', 'minimum', 'maximum' constraints visible in code). This forces LLMs to guess parameter types and valid ranges.
No output schemas documented. Tools return data but LLMs cannot predict field names, types, or structure. E.g., query_executor returns 'raw_products' but no schema for product object fields (id, name, price, rating, etc.).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | <=2025-11-25 | v2 |
Retrieve all conversation thread IDs from MongoDB checkpoints and Chat_History collections. Returns list of thread identifiers for session management.
Extract purchase preferences (brand, budget, rating, category) from user input using keyword-based extraction and LLM. Merges current preferences with existing state, prioritizing current user input as primary authority.
Handle product information queries and general chat. Generates MongoDB query for product-related questions, executes search, applies secondary filtering (budget, rating), and formulates detailed answer using LLM with deterministic fallback.
Return a friendly message redirecting non-product queries back to shopping assistance, or acknowledge order/payout updates. Handles system transaction notifications (checkout, payment failed, payout).
Execute MongoDB query against Products collection via MCP connector. Retrieves raw product documents matching the generated filter, with fallback to broad regex search and catalog defaults if no results found.
Generate a MongoDB find() filter document based on extracted user preferences (category, brand, budget, rating). Uses LLM to create case-insensitive regex queries for product catalog search.
Analyze candidate products against user requirements and select top 3 most relevant products. Prioritizes exact category/title match, brand match, rating, and price within budget. Returns selected indices and friendly introduction text.
Save a single chat message (user or assistant) to MongoDB Chat_History collection with deduplication logic to prevent consecutive duplicate messages.
Rigid pipeline composition. classify_intent → info_gatherer → query_generator → query_executor → reranker forms a fixed sequence. Agents cannot skip steps, retry individual stages, or compose tools flexibly. This violates single-responsibility principle.
delete_history is destructive but lacks confirmation pattern. No dry-run, no confirmation step, no undo tool. Agents can permanently delete conversation history without safeguard.
Error handling is minimal. No recovery guidance, no error categorization (retryable vs. fatal), no actionable error messages. E.g., if query_executor fails, LLM has no guidance on whether to retry, call a different tool, or ask the user.