AI-powered adaptive diagnostic learning engine for CS students with personalized study plan generation and Notion integration
GapForge exposes 4 tools via FastMCP with HTTP transport. All tools have descriptions (10-200 chars) and input schemas with typed parameters. However, schemas lack depth: no enums for constrained inputs (e.g., 'difficulty' accepts free-form strings instead of enum [beginner, basic, intermediate, advanced]), no output schemas documented, and parameter descriptions are minimal (avg 40 chars vs baseline 72). Tool naming is verb-forward (create_, get_) but descriptions do not explain WHEN to use each tool or dependencies between them. Error handling is absent, no recovery guidance, no actionable error messages. Security: no mention of secret injection, rate limiting, or permission gates. Composition: tools are single-purpose but lack chaining metadata (e.g., create_study_roadmap needs topic_id from get_prerequisite_chain, but no guidance on this flow).
Creates a structured study roadmap page in Notion.
Evaluate a submitted student answer against the known correct answer.
Retrieve the prerequisite topic chain for a given topic ID. Returns an ordered chain from the root concept down to the current topic, with a depth limit of 10 to prevent infinite recursion and ensure performance.
Fetch a list of questions for a specific subject and difficulty level. This function utilizes the exclude_ids parameter to ensure that it never returns questions that the student has already seen, protecting the integrity of the diagnostic assessment.
No output schemas documented. LLMs cannot plan downstream calls or extract required fields (e.g., what does get_prerequisite_chain return? Is it an array? Does it include topic_id for chaining to create_study_roadmap?).
Parameter 'difficulty' in get_questions accepts free-form strings instead of enum [beginner, basic, intermediate, advanced]. LLMs will hallucinate invalid values.
No error handling or recovery guidance. If get_questions returns empty (no questions match criteria), the tool provides no actionable next step for the LLM.
Parameter descriptions are minimal (avg 40 chars). E.g., 'exclude_ids' is 'Optional list of question UUIDs already seen by this student', lacks format guidance (is it a JSON array? CSV? How many IDs max?).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | <=2025-11-25 | v2 |
Tool composition not documented. create_study_roadmap requires 'root_cause_topic' (string), but get_prerequisite_chain returns a chain, no guidance on how to extract the root from the chain or whether they use the same topic ID format.