Loading ToolBench
Access and manage your Shiori study data — assignments, grades, notes, flashcards, grade predictions, and study plans.
The Shiori MCP server has 6 tools with reasonable naming and parameter structure, but critical gaps in documentation, schema completeness, and error handling. Tool names follow verb_noun convention (get_*, add_*) which is positive. However, parameter descriptions are sparse or missing context, output schemas are not documented, and there is no evidence of error recovery guidance. The server appears to be a thin wrapper around a backend API without agent-optimized definitions. Most tools are READ_ONLY, reducing error surface, but the single WRITE tool (add_assignment) lacks validation detail and confirmation patterns. Overall quality is below median (baseline ~45-55); this server would require significant rework before production use with agents.
Add a new assignment to Shiori.
List your upcoming assignments. Optionally filter by course or status.
List your flashcard decks and review status.
Show your current GPA and grades per course.
List and read your course notes.
Get a full summary of your current academic status — assignments, grades, habits, and upcoming events.
No documented output schemas for any tool. LLMs cannot plan downstream calls or extract required fields when return types are invisible. Critical for agent reasoning.
Descriptions lack specificity about WHEN to use each tool and what distinguishes them. E.g., get_study_summary vs get_assignments, unclear which to call first. Descriptions should be 50 - 200 chars and answer 'what does it do?' + 'when to use it?'
add_assignment is the only WRITE tool but lacks a confirmation/dry-run pattern. Agents can make mistakes, irreversible operations should support a confirm step.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
Parameter descriptions are minimal. E.g., 'course' in get_assignments is 'Filter by course name (partial match)', but what happens if no course matches? What format is expected (exact, substring, regex)? Descriptions must include constraints and examples.
No error handling guidance in any tool description. If get_assignments fails because the backend is offline, what should the agent do? Retry? Call a diagnostic tool? Error responses must include recovery steps.
get_assignments and get_notes accept optional 'course' parameter but do not document how partial matching works or what to return if multiple courses match. Parameter relationships and edge cases must be explicit.
add_assignment dueDate format is documented as 'YYYY-MM-DD' but no validation detail (e.g., must be in future, cannot be more than 1 year out). LLMs frequently pass invalid dates, constraints must be explicit and validated server-side.
get_study_summary returns a 'full summary' but no documentation of what fields are included or limits (e.g., is it capped at 50 assignments? All notes?). Large responses dilute signal, must document limits.