Single tool with basic schema and description. Tool name follows verb_noun pattern (get_user_courses). Description is present (73 chars) but lacks context on when to use it vs alternatives, prerequisites, or what the return structure contains. Input schema uses Zod with type definitions and descriptions for all three parameters, which is good. However, output schema is not documented, LLMs cannot predict the response structure. Error handling returns plain text errors without recovery guidance. No tool annotations (readOnlyHint, etc.). Overall: functional but below production baseline (194 char avg description, 100% A+ tools document output).
Get courses for a specific user with optional grading count
Output schema not documented. LLMs cannot predict response structure (formatted courses with id, name, code, state, needsGradingCount, enrollmentState fields). Agents cannot plan downstream operations or extract data reliably.
Tool description lacks context on when to use it, prerequisites (Canvas API token required), and what data is returned. At 73 chars, it's below the 194-char baseline for A+ tools. Should explain: 'Retrieves courses for a Canvas user, optionally including grading counts. Requires CANVAS_BASE_URL and CANVAS_ACCESS_TOKEN environment variables. Returns array of course objects with id, name, code, workflow state, and optional grading count.'
Error handling returns plain text error messages without recovery guidance. Per pattern:recovery-guide, errors should tell LLMs what to do next. E.g., 'Failed to fetch user courses: 401 Unauthorized' should become 'Authentication failed. Verify CANVAS_ACCESS_TOKEN is valid and has user:read scope. Check Canvas instance URL.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
No tool annotations. Tool is read-only (no side effects) but lacks readOnlyHint annotation. This prevents agents from understanding retry safety and idempotency guarantees.
Parameter 'enrollment_state' accepts free-form strings with no enum constraint. Description says '(active, invited, etc)' but LLMs may hallucinate invalid values like 'pending' or 'archived'. Should declare enum: ['active', 'invited', 'completed', 'rejected'].