Loading ToolBench
Production-ready NestJS template with MCP server, RAG pipeline, and comprehensive developer tooling. Exposes course management tools via the Model Context Protocol.
Strong tool naming and parameter schemas with clear descriptions. All 6 tools follow verb_noun convention (list_, search_, get_, semantic_search, find_similar_). Input schemas are well-structured with JSON Schema types, descriptions, and constraints (UUID patterns, min/max bounds, defaults). However, output schemas are not explicitly documented in the source code provided, the rubric requires documented return types for A-level tools, and their absence limits the score. Tool descriptions are solid (100-200 char range, explaining purpose and filtering behavior), though they could be more explicit about return structure. Security considerations are addressed via enrollment filtering and role-based access (admins vs users), but no explicit documentation of permission scopes. Error handling strategy is not visible in the source.
Find lessons with similar content to a given lesson. Uses semantic similarity of lesson embeddings. Results are filtered to enrolled courses.
Get detailed information about a specific course, including modules and lessons. Requires enrollment (except for admins).
Get detailed content of a specific lesson. Requires enrollment in the course containing this lesson (except for admins).
List available courses. Returns all courses for admins, or only enrolled courses for regular users.
Search for courses by title or description. Results are filtered based on user enrollment.
Perform semantic search across lesson content using embeddings. Returns relevant lesson chunks ranked by similarity. Results are filtered to enrolled courses only.
Output schemas not documented in source code. While input schemas are explicit and well-formed, return types (field names, structure, types) for all 6 tools are not visible. This prevents LLMs from understanding what fields to expect and breaks the tool-chaining pattern, downstream tools cannot be planned if the LLM doesn't know what the current tool returns.
No explicit error handling or recovery guidance visible in tool definitions. Descriptions state 'Requires enrollment' but do not specify what error the LLM should expect (403 Forbidden, 404 Not Found) or what action to take if enrollment is lacking. Per pattern:recovery-guide, error responses should tell the LLM what to do next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 61 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Permission scopes not declared. Tools implement role-based filtering (admins see all courses, users see only enrolled) but do not expose scope declarations (e.g., 'read:courses', 'read:lessons') in the tool metadata. Per pattern:scope-declaration, each tool should declare what permissions it requires.
courseId parameter descriptions lack explicit format guidance. The UUID pattern is declared in JSON Schema ('^[0-9a-f]{8}-...$') but the description 'UUID of the course to retrieve' does not mention the expected format explicitly for the LLM. Per pattern:constrained-input, constraints should be readable in the description text, not just in schema.
semantic_search minSimilarity parameter defaults to 0.7 but the description does not explain what this threshold means in practical terms (e.g., 'Results below 0.7 are typically irrelevant') or when to adjust it. This invites suboptimal LLM choices if the default produces poor results.