Loading ToolBench
FastMCP server exposing SugarLearning data to AI agents, including learning modules, items, user backlog, progress tracking, and semantic search via Qdrant.
The server defines 15 tools with reasonable naming and descriptions, but has systematic gaps in schema completeness, parameter documentation, and error guidance. Tool names follow verb_noun pattern well (list_modules, get_item, complete_item, save_note, search_learning). Descriptions range from adequate to good, averaging ~150 chars. However, input schemas are minimally described, most parameters lack detail on constraints, valid ranges, or allowed values. Error handling is basic: some tools return error dicts but without recovery guidance. Output schemas are not formally documented. The server is functional for read operations but weak on write safety (complete_item, save_note lack confirmation/dry-run patterns). STDIO transport caps overall protocol readiness at 50, which significantly limits the overall score.
Mark a learning item as complete. Automatically determines whether to request approval or auto-complete. Args: item_id: The learning item ID (e.g. from search or backlog). comment: Optional comment to include with the completion.
Get the learning backlog for a user, showing all assigned modules and items with completion status. Args: user_id: User identifier. Defaults to configured user.
Get a learning item's full details and content. Args: item_id: The SugarLearning item ID. user_alias: Optional user alias. Defaults to the current user ("me").
Get company leaderboard rankings showing user progress, points, and badges. Args: group_id: Group ID to filter by, or 'all' for everyone. limit: Maximum number of users to return (default 20).
Get detailed information about a specific learning module by its ID.
Parameter descriptions lack constraint details. 'module_id' is typed as integer but no description states valid range, what happens with invalid IDs, or whether the ID is internal or API-facing. 'count', 'limit', 'user_id' similarly under-documented. LLMs cannot infer constraints from type alone.
Write operations (complete_item, save_note) lack confirmation/dry-run patterns. These are state-modifying tools that agents might call incorrectly. No explicit 'Are you sure?' step or ability to preview changes before committing. Violates confirmation-request pattern for irreversible actions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 50 | - | v1 |
Get all groups assigned to a specific module.
Get all learning items within a specific module.
Get the module list (employee/public view) - simpler than admin modules.
Get all users assigned to a specific module, including their progress percentage and completion status.
Get recent change diffs showing what changed in SugarLearning (new modules, new items, user assignment changes). Args: count: Number of recent diffs to return (default 5).
Get badges earned by a user, showing which modules they completed. Args: user_alias: User's alias (e.g. 'jk'). Defaults to current user.
Get a user's profile including badges, companies, and personal info. Args: user_alias: User's alias (e.g. 'jk'). Defaults to current user.
List all learning modules in SugarLearning with their metadata (name, manager, item counts, user counts).
Save or update a private note on a learning item. Args: item_id: The learning item ID. content: The note text. note_format: 'markdown' or 'html' (default: markdown).
Semantic search across all learning modules and items using Qdrant vector search. Requires Qdrant to be running and indexed (run 'sl index' first). Falls back to listing all modules if Qdrant is unavailable. Args: query: Natural language search query. limit: Maximum number of results to return.
Error handling returns generic error dicts (e.g. {"error": "Item {item_id} not found in your backlog."}) without recovery guidance. LLM does not know whether to retry, call a different tool, or ask the user. Per pattern, errors should categorize as retryable/user-fixable/fatal and suggest next steps.
Output schemas are not formally documented. Tools return dicts/lists but the structure, field names, and types are implicit. 'get_leaderboard' returns list[dict], what fields does each dict contain? 'percentageOfPointEarned' is mentioned in code but not in documentation. Downstream tools cannot reliably chain on outputs.
'get_module_list' has a vague name and minimal description ('simpler than admin modules'). Unclear what this retrieves vs 'list_modules' or 'get_backlog'. Ambiguous tool names cause LLMs to conflate or misselect. Recommendation: rename to 'list_modules_employee_view' and clarify when to use vs list_modules.
Parameters accept user_alias and user_id (sometimes optional, sometimes required) but distinction is unclear. 'get_item' has optional user_alias; 'get_backlog' has optional user_id. Are these the same concept? Do they accept email, display name, or only internal aliases? Inconsistent naming across tools.
'note_format' parameter in save_note defaults to 'markdown' but accepts only 'markdown' or 'html'. No enum constraint visible in schema. Description says "'markdown' or 'html'" but parameter definition does not enforce this. LLM may pass invalid values like 'text' or 'rst'.
'search_learning' has undocumented fallback behavior. If Qdrant is unavailable, it silently lists all modules and filters by text search. This could return hundreds of items if no limit is enforced. No documentation of fallback behavior in description. LLM unaware it may get unranked, generic results.
'get_leaderboard' filters out users with 0% progress silently ('Filter out 0% users by default'). This silent filtering is not documented in the tool description. LLM expects all available results, may be surprised by filtered output.