Loading ToolBench
Enterprise-Grade MCP Orchestration for Modern Development
AIStack-MCP exhibits significant quality gaps across naming, descriptions, schema clarity, and error handling. While tool names follow verb_noun convention (search_code_semantic, analyze_patterns), descriptions are verbose and lack clarity about when to use each tool versus alternatives. Critical issue: Tools 1-3 appear in mcp_intelligence_server.py but Tool 4-6 are duplicates in python_agent/mcp_production_server.py with nearly identical names and functionality, creating ambiguity. Parameter schemas are documented but lack detailed constraints (enums, patterns, ranges). No visible error handling patterns or recovery guidance. Implementation files show TODO placeholders (code_rag_tools.py, rag_tools.py) indicating incomplete functionality. The server purports to handle code analysis and semantic search but actual implementation is skeletal.
Multi-layer impact analysis using LOCAL agents. Industry Pattern (Nov 2024): - Call graph analysis (local) - File dependency analysis (local) - Semantic impact via Qdrant (local) - Compressed summary via Ollama (local)
Analyze codebase patterns using LOCAL Ollama LLM. Industry Pattern (Nov 2024): - Local LLM reads and analyzes code (FREE) - Vector search finds relevant examples (FREE) - Returns compressed summary (200 tokens vs 4000)
Analyze code patterns in current workspace using local LLM. Finds examples via semantic search, analyzes with Ollama. Returns compressed summary (~200 tokens).
Prepare optimized context for code generation. Reads file, finds patterns, extracts key sections. Returns compressed context (~400 tokens vs full file).
Semantic code search using LOCAL Qdrant vector database. Industry Pattern (Nov 2024): - Vector search happens locally (FREE, instant) - Returns compressed snippets (not full files) - 90% token reduction vs reading files directly
Duplicate tool definitions across two files. semantic_search (mcp_intelligence_server.py) and search_code_semantic (python_agent/mcp_production_server.py) serve identical purposes with nearly identical schemas. This creates LLM confusion about which to call and violates single-responsibility principle. Same duplication pattern for analyze_patterns/analyze_code_patterns.
Parameter schemas present but lack actionable constraints. 'min_score' (0-1) has no description of what scores mean or when to adjust. 'pattern_type' in analyze_patterns accepts arbitrary strings but no enum of valid options (error_handling, async, dependency_injection mentioned in docs but not in schema). LLMs cannot validate input without explicit enums or patterns.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 39 | - | v1 |
Semantic code search in current workspace using Qdrant. Searches code by meaning, not just text matching. Returns compressed snippets (not full files).
Tool descriptions lack clarity on differentiation and usage context. 'Semantic code search' vs 'Semantic code search using LOCAL Qdrant', both describe the same thing. No guidance on when analyze_patterns vs analyze_code_patterns should be selected. Descriptions focus on implementation details (Qdrant, Ollama, token counts) rather than user-facing intent.
No documented output schemas for any tool. Tool descriptions claim to return 'compressed snippets', 'compressed summary (~200 tokens)', 'compressed context (~400 tokens)' but actual JSON response structure is not defined. LLMs cannot plan downstream calls without knowing what fields to expect.
No error handling or recovery guidance visible in tool definitions. What happens if Qdrant is not running? If pattern_type is invalid? If workspace is not indexed? Descriptions mention 'auto_index' default but no error message if auto-indexing fails.
Implementation is incomplete. code_rag_tools.py and rag_tools.py contain TODO placeholders and return empty results or boolean success without actual functionality. 'search_code()' and 'upsert_code_snippets()' return empty lists and stub True values. The server cannot deliver on its stated capabilities.
Parameter descriptions are generic or missing detail. 'max_results: Number of results to return', what is the range? Is there a performance penalty? 'pattern_type: Pattern to analyze', what are valid values? 'task: What needs to be done', how specific should this be? LLMs cannot optimize parameters without constraints.
Tool names contain marketing language ('LOCAL', 'Industry Pattern (Nov 2024)') in descriptions rather than focusing on function. 'search_code_semantic' vs 'semantic_search', inconsistent naming across identical tools. LLMs parse names before descriptions; unclear names invite wrong tool selection.