Loading ToolBench
University of Sydney Canvas and Ed Discussion Integration. MCP server supporting both Canvas REST API and Ed Discussion API.
The server demonstrates solid foundational quality with 16 well-named tools covering Canvas, Ed Discussion, and Gradescope. All tools have descriptions (100-200 chars average), and input parameters are fully typed with Pydantic models and enum constraints. However, there are notable gaps: output schemas are not explicitly documented in the tool definitions, error handling is minimal (no recovery guidance in descriptions), and many tools lack context about when to use them versus similar alternatives. The code shows good security practices (secret injection via env vars), but the tooling lacks modern MCP patterns like tool annotations (readOnlyHint/destructiveHint), structured error responses, and MRTR for confirmation workflows on destructive operations.
Create a new thread in Ed Discussion.
Get detailed information about a specific Canvas course including name, description, enrollment state, dates, and syllabus.
Get discussion threads from an Ed Discussion course.
Get grades from Canvas. Returns grades for a specific course or all courses if course_id is not provided.
Get assignments from Gradescope.
Get submission status for a Gradescope assignment.
Get a single Ed Discussion thread with all its comments (answers, follow-ups).
Output schemas are not documented in tool definitions. LLMs cannot infer what fields they should expect in responses, forcing them to guess or request clarification. Responses should document return structure (e.g., 'Returns: {course_id, name, enrollment_state, created_at, syllabus_url}').
Destructive/write tools (create_thread, reply_to_thread, toggle_thread_action) lack tool annotations (destructiveHint=true) and confirmation workflow guidance. No mention of recovery steps or dry-run capability. LLMs may irreversibly create posts or modify discussions without explicit confirmation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Fetch the unit outline (syllabus) from a Canvas course's External Tool tab (USYD-specific). Fetches the HTML and extracts key information like learning outcomes, resources, and links.
Get announcements from a Canvas course, ordered by newest first.
Get assignments from a Canvas course.
Get list of Canvas courses for the authenticated user. Returns course ID, name, enrollment state, and dates.
Get files from a Canvas course.
Get items within a Canvas module.
Get modules from a Canvas course.
Reply to an Ed Discussion thread (create a comment).
Toggle an action on an Ed Discussion thread (star, pin, lock, endorse, etc.).
Error handling is minimal. Tool descriptions do not include recovery guidance or error categories. For example, create_thread offers no guidance on what happens if course_id is invalid (should the LLM retry? ask the user for a different course? look up the course first?).
List tools lack pagination context in descriptions. Tools like list_courses, list_announcements, list_assignments support limit parameters but do not state maximum result count, explain pagination behavior, or how to fetch next page (offset, cursor, etc.). LLMs may assume all results fit in one call.
Parameter descriptions are generic and lack context. For example, response_format appears in every tool but is described identically without guidance on when to use markdown vs JSON (e.g., 'Use markdown for display to the user, JSON for structured downstream processing'). LLMs cannot infer the right choice.
toggle_thread_action combines 8 distinct operations (star/unstar, pin/unpin, lock/unlock, endorse/unendorse) under one tool. LLMs must reason about which action to invoke; this conflates concerns and makes tool selection harder. Consider splitting into separate tools (star_thread, pin_thread, lock_thread, endorse_answer) for clearer intent.
Some tool descriptions are under 80 characters and lack context. For example, get_thread: 'Get a single Ed Discussion thread with all its comments' (59 chars) does not explain when to use it vs get_discussion_threads or whether it returns all or a limited number of comments.
Canvas and Ed Discussion tools are mixed without clear prefixing. Tool names like list_courses and get_discussion_threads don't signal which service they talk to. Consider prefixing: canvas_list_courses, ed_get_discussion_threads, gradescope_get_assignments for clarity when many tools are present.