Loading ToolBench
SDLC Best Practices MCP Server with hybrid search architecture using BM25 lexical search, vector semantic search, and cross-encoder reranking for documentation retrieval
This server has three tools with basic schemas and descriptions, but significant gaps in definition quality limit production readiness. Tool naming is adequate but not ideal (prefixes are present but verbs could be clearer). Descriptions exist but are verbose and lack actionable detail for LLM decision-making. Parameter schemas are present and typed, but lack constraints, bounds, and dependency documentation. Output schemas are not documented. Error handling is minimal, error messages are generic and do not guide recovery. The server demonstrates awareness of security (path validation in pqsoft_read_docs) but lacks comprehensive input validation, parameter constraints, and clear permission boundaries. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present despite the Risk field claiming 'READ_ONLY'. Overall, these tools are functional but would fail code review by a principal tool engineer due to incomplete descriptions, missing output documentation, and lack of LLM-specific guidance.
Read specific line range from a documentation file.
Get related documentation based on content similarity.
Search SDLC documentation using hybrid retrieval + reranking. Pipeline: 1. BM25 search for keyword matches (top 30) 2. Vector search for semantic matches (top 30) 3. Union and deduplicate candidates 4. Rerank with cross-encoder for final ordering
Output schemas are completely undocumented for all three tools. LLMs cannot plan downstream tool calls or extract required fields without trial and error.
Parameter constraints (min/max, enums, patterns) are missing from schemas. Only 'limit' has a constraint mentioned in the description text, but LLMs may ignore text-only constraints. JSON Schema should enforce 'start_line >= 1', 'end_line >= start_line', 'limit >= 1 and <= 50'.
Tool descriptions lack LLM-specific guidance. For pqsoft_recommend_docs, the description does not clarify: (1) the difference between it and pqsoft_search_docs, (2) whether the input 'title' must be an exact match or a search query, (3) what happens if no similar docs exist. Without this, LLMs conflate the tools or misuse them.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 41 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 35 | - | v1 |
Error handling is generic and does not guide recovery. Code raises ValueError with messages like 'Invalid path' or 'Line range out of bounds', but LLMs receive these as exceptions without guidance on next steps. No attempt to offer alternatives (e.g., 'Available docs are...') or suggest a corrective action.
Tool annotations are declared in the 'Risk' field as 'READ_ONLY' but are NOT implemented in the MCP protocol layer. The fastmcp @mcp.tool() decorator shows no readOnlyHint parameter. LLMs cannot detect that these tools are safe to retry or that they do not mutate state.
Redundant naming convention: all tools are prefixed with 'pqsoft_', which adds no descriptive value. In a search_docs context, the system name is already known. Shorter names like 'search_docs', 'read_docs', 'recommend_docs' would be clearer.
Parameter descriptions are minimal (often under 50 chars) and lack format/range guidance. E.g., 'The search query string' does not specify: max length, special character handling, whether regex is supported, or how multiple keywords are combined (AND vs OR).
pqsoft_read_docs validates paths server-side but does not expose validation rules to the LLM. The description should state: 'Must be a relative path to a .md file. Paths must not contain ../, /, or start with a dot.'