Loading ToolBench
Model Context Protocol server for ICTExam. Lets AI assistants read exams, gradebooks and item analysis, and (when writes are enabled) parse a question paper with AI and publish exams.
Strong foundation with 9 clearly defined tools, all with descriptions and input schemas. Tool naming follows verb_noun convention (list_, get_, publish_, unpublish_, parse_) consistently. Descriptions are well-crafted (avg 180 chars) and explain WHAT each tool does and WHEN to use it. Input schemas use Zod validation with type annotations and parameter descriptions. However, several critical gaps prevent a higher score: (1) output schemas are NOT documented, tools return JSON but the response structure is never specified, forcing LLMs to infer what fields to expect; (2) error handling is minimal, generic try/catch with toolError() wrapper provides no recovery guidance; (3) no pagination declared on list tools despite potential for large result sets; (4) parameter constraints (e.g., file types for parse_question_paper) are described in text but not enforced as enums; (5) idempotency and retry semantics not documented for write operations. The parse_question_paper tool's output truncates question text to 120 chars, which is pragmatic but undocumented. All tools correctly avoid exposing secrets in parameters.
Get a single exam paper by id: its settings, timing, share links and questions.
Get the teacher gradebook: per-exam totals and student marks. Optionally filter by class_id or subject_id.
Per-question item analysis for one exam: difficulty, p-value, average mark and how many responses were full vs zero credit.
List the classes (course groups) defined on ICTExam.
List exam papers (assessments) on ICTExam: id, title, status (draft/published), class and subject. Start here to find an exam's id. Optionally filter by class_id or subject_id.
List the subjects on ICTExam, each tied to a class.
Output schemas not documented. Tools return JSON objects but response structure (field names, types, nesting) is never specified in tool definitions. LLMs must infer response shape, risking incorrect downstream chaining and field extraction errors.
Error handling provides no recovery guidance. Generic try/catch blocks call toolError(e) without categorizing errors (retryable, user-fixable, fatal) or providing actionable next steps. LLMs receive raw error messages with no context on whether to retry, call a different tool, or ask the user.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 72 | 2026-07-28+ | v2 |
Upload a local PDF or DOCX question paper and have ICTExam extract the questions with AI: returns each question's type, text, options, correct answer and marks, plus any mark scheme. This spends AI credit on the server.
Publish an exam paper by id so students can take it. Returns the student and teacher share URLs. Use ictexam_unpublish_exam to reverse it.
Set an exam paper back to draft by id, so students can no longer access it.
No pagination declared for list tools. ictexam_list_exams, ictexam_list_classes, ictexam_list_subjects, and ictexam_gradebook may return unbounded result sets. No limit, offset/cursor, or total_count documented. Large result sets will blow context windows without pagination controls.
Parameter constraints not enforced via enums. ictexam_parse_question_paper's document_type is a free-form string with description 'Parser mode, default mcq_structured', should be an enum (mcq_structured, essay, mixed, etc.) to prevent hallucinated values. file_path accepts any string with no format validation.
No idempotency or retry semantics documented. Write tools (publish_exam, unpublish_exam, parse_question_paper) do not declare whether they are idempotent, safe to retry, or have side effects on repeated calls. Agents may retry ambiguous failures and cause duplicate or conflicting state changes.
No confirmation or dry-run for irreversible operations. Publish and unpublish modify exam visibility immediately. Should support a --dry-run flag or explicit confirmation step to prevent accidental publication/unpublication by agents.
Permission gates not declared. Tools that modify exams (publish, unpublish, parse) do not specify what permissions they require (write:exam, admin:parse). LLMs cannot determine if they have authorization or plan least-privilege workflows.
parse_question_paper silently truncates question text to 120 chars without warning. Tool description states 'returns each question's type, text, options...' but implementation returns truncated text. LLMs may not realize they are working with incomplete data.