Loading ToolBench
Educational content search server with role-based access control (RBAC), semantic search via Qdrant vector database, and OAuth 2.1 authentication via Scalekit. Designed for HTL Leonding educational institution.
Server has 3 tools with clear RBAC separation (search_content_student, search_content_teacher, health_check). All tools have descriptions and input schemas with types. However, descriptions lack depth on WHEN to use each tool and WHY they differ. Parameter descriptions are minimal (e.g., 'Search query in natural language' lacks format/constraint guidance). Output schemas are not documented, LLMs cannot predict response structure. Error handling is absent from visible code. Tool annotations present but underutilized. Naming is verb-forward and clear, but composition could be tighter (health_check is utility, not domain-specific).
Check server health and connectivity. Returns basic server information and status.
Search educational content with STUDENT access level. Provides access to public content and student-level educational materials. NOT accessible: teacher-internal documents, administrative content, exam solutions (teacher-only).
Search educational content with TEACHER access level. Provides full access including teacher-internal documents, administrative content, and exam solutions.
Output schemas not documented. LLMs cannot predict response structure (fields, types, pagination). search_content_* tools return results but no schema visible for 'results' array, 'title', 'text', 'source' fields, or total count.
Parameter descriptions lack actionable constraints. 'limit' param says '1-20, default: 10' but no description of what happens at boundaries or why 20 is the cap. 'query' lacks format guidance (German/English noted but no length limits, special char handling, or examples of good queries).
Tool descriptions do not explain WHEN to use each tool or WHY they differ. Both search_content_* tools have identical signatures and similar descriptions. LLMs cannot infer that student tool filters by access_level without explicit guidance like 'Use this if you are a student; teacher version grants access to exam solutions and admin docs.'
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | <=2025-11-25 | v2 |
No error handling guidance visible. Code shows ToolError import but no examples of error messages, recovery steps, or categorization (retryable vs user-fixable vs fatal). LLMs will not know how to respond to 'No results found' or 'Query too broad'.
health_check tool lacks clear purpose in educational context. Description says 'Check server health and connectivity' but does not explain when an LLM should call it (e.g., before search, on startup, on error recovery). Empty input schema is correct but description should guide usage.