Loading ToolBench
Udemy MCP Server for instructor API
Three tools with complete Zod schemas and basic descriptions. All tools are READ_ONLY and follow verb_noun naming (get_*). However, descriptions are minimal (10-40 chars), lack context on WHEN to use each tool, and parameter descriptions are generic. No output schema documentation. Error handling returns isError flag but lacks recovery guidance. Schemas are present and typed, but descriptions fall short of the 50-200 char LLM-optimized baseline for A-grade tools.
Get a list of questions for a course
Get a list of reviews for courses
Get a list of courses taught by the instructor
Tool descriptions are too brief (10-40 chars). LLMs cannot determine WHEN to use each tool or how they differ. Descriptions should be 50-200 chars and answer: What does it do? When should I call it instead of a similar tool? What does it return?
No output schema documentation. Callers cannot predict response structure. Document what fields are returned (e.g., 'Returns array of question objects with id, title, body, created, num_replies, is_featured, etc.').
Error responses lack recovery guidance. When makeUdemyRequest fails, tool returns 'Failed to fetch questions' with no hint on what to do next (retry? check auth? try different params?). Errors should guide the LLM toward recovery.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Parameter descriptions are generic and lack constraints. 'Page number (default: 1)' does not specify valid range (e.g., 1-1000). 'Number of questions per page (default: 100)' does not state min/max bounds. LLMs may pass invalid values.
No pagination limit enforcement in tool descriptions. Tools accept page_size but do not document max results per call or total result cap. Large result sets risk context window exhaustion.