Loading ToolBench
Multi-service MCP server providing database queries, ML predictions, plan optimization, RAG document retrieval, and chat history management for a commission management system
This server exhibits significant quality gaps across naming, descriptions, schema completeness, and error handling. Of 20 tools, most have adequate names but critically weak or missing descriptions, incomplete input schemas, and no visible output schema documentation. Tool definitions appear inferred from code rather than explicitly registered with full schemas. Error handling is minimal. The server conflates multiple responsibilities (database queries, ML predictions, NLP, chat management, RAG) under 20 tools without clear composition. STDIO transport caps protocol readiness at 50 maximum.
Clear chat history from Redis for a user session
Clear the RAG vector store cache
Execute SQL query with security validation
Extract ML-related parameters from user message (percentage change, plan ID, etc.)
Extract plan structure from natural language text using NLP
Retrieve chat history from Redis for a user session
Load full schema info for all tables in PostgreSQL public schema
Generic/overloaded tool name 'handle_ml_request' violates verb_noun clarity pattern. Does it predict, optimize, or route? LLMs will conflate it with other ML tools. Should be split: 'route_ml_request' (dispatcher) or removed entirely with direct predict/optimize tools.
Intent detection tools (is_plan_creation_intent, is_database_query_intent, is_ml_prediction_intent, is_ml_optimization_intent) have vague names starting with 'is_' (boolean predicates) rather than action verbs. Descriptions are minimal (40-45 chars). These tools should either be merged into a single 'classify_user_intent' dispatcher or removed entirely if intent routing happens server-side. Current design fragments intent logic across 4 tools.
Duplicate tool definition: 'get_database_schema' appears twice (tools #1 and #19). Confuses LLM tool selection. Remove one; retain only the fully documented version.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 38 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 29 | - | v1 |
Get complete database schema
Main handler for ML predictions and optimizations - routes to appropriate ML function
Determine if user wants to query the database
Detect if user is requesting ML plan optimization recommendations
Detect if user is requesting ML commission impact predictions
Determine if user message indicates intention to create a new commission plan
Look up plan_id by plan name with case-insensitive partial matching
Predict commission impact of a percentage change using Monte Carlo simulation
Execute natural language query using MCP + Claude to translate to SQL
Generate AI-powered recommendations to optimize commission plans
Retrieve relevant context from RAG vector store for document-based queries
Save message to Redis chat session storage
Enhanced SQL validation for security - only allows SELECT statements
Tool descriptions are sparse (40-65 chars on average, well below 194-char baseline). Examples: 'Retrieve chat history from Redis for a user session' (55 chars) lacks context on when to call vs other retrieval tools. 'Determine if user message indicates intention to create a new commission plan' is 75 chars but provides no clarity on format of response (boolean, confidence score, reasoning?). Most descriptions need 2-3x expansion with WHEN/WHY/WHAT guidance.
No visible output schema documentation for any tool. Source code shows execute_sql returns {success, error, data, row_count} but this is not declared in tool definitions. process_natural_language_query, predict_commission_impact, recommend_plan_optimizations, retrieve_context all lack output schemas. LLM cannot predict response structure, forcing blind downstream chaining.
Multi-tenant parameters (org_id, client_id) appear in 8+ tools but are often not enforced at tool boundary. If org_id is nullable or optional, no validation shown. Code snippet does not reveal permission checks. Risk: agents can query arbitrary organizations if params are not validated.
SQL execution (execute_sql_query, validate_sql) relies on external sql_security module with fallback that only checks 'SELECT' prefix. No evidence of parameterized query support, prepared statements, or protection against injection via column names, function calls, or UNION-based attacks. Descriptions do not state security assumptions.
Intent detection tools (is_plan_creation_intent, is_database_query_intent) have no specified output schema. Do they return boolean, {intent: boolean, confidence: number}, or {intent, reason}? Ambiguity forces LLM to guess, risking routing failures.
No error handling guidance visible in tool definitions. If execute_sql_query fails, does it return {success: false, error: '...'} or raise exception? If process_natural_language_query hits an LLM timeout, what does the agent see? Descriptions lack recovery hints (pattern:recovery-guide). Tool implementations in source fragment only.
Chat session tools (save_message, get_chat_history, clear_chat_history) require user_id and session_id but no validation shown. If session_id is user-supplied string, agents can read/clear arbitrary sessions. No scoping or permission checks visible.
ML parameter extraction (extract_ml_params) and intent detection are split across 6 tools, fragmenting intent routing logic. This violates composition principle: common workflows should resolve in few calls. Consider a single 'classify_and_extract_request' tool that returns intent class + extracted params in one call, or move intent logic server-side to tool dispatcher.