Loading ToolBench
A Python implementation of the MEM ontology MCP server for querying and navigating German curriculum (Lehrplan) data stored in a SPARQL triple store
The MEM Ontology server has 8 well-named tools with generally clear descriptions and documented input schemas. All tools use action verbs (sparql_query, list_*, find_*, get_*, search). Descriptions are in the 60-250 character range, meeting the baseline (avg 194 chars). Input parameters are typed with descriptions. However, OUTPUT schemas are entirely undocumented, the server returns results but provides no schema specification for what fields agents should expect. Error handling and recovery guidance are minimal. Tool descriptions lack dependency hints and context on when to call each tool vs alternatives. The domain coverage is complete for the MEM ontology use case, but token efficiency could improve by capping result sets.
Find curricula (Lehrpläne) by Bundesland, optionally filtered by Schulfach, Schulart, or Jahrgangsstufe. Use state codes/names. For Schulfach and Schulart, use the German name as shown by the list tools.
Get the direct children of a specific node in the Lehrplan hierarchy (via 'hat Teil'). Use this to drill down into a specific branch after using get_lehrplan_tree.
Get the hierarchical structure (parent-child via 'hat Teil') of a specific Lehrplan. Use a Lehrplan URI obtained from find_lehrplaene. The depth parameter controls how many levels deep the tree goes (default 2, max 10). Use get_children to drill deeper into specific nodes.
List all German federal states (Bundesländer) available in the ontology with their codes and URIs.
List all school types (Schularten) for a Bundesland. Accepts a state code (BY, SN, RP, ...) or name (Bayern, Sachsen, ...).
List all school subjects (Schulfächer) for a Bundesland. Accepts a state code (BY, SN, RP, ...) or name (Bayern, Sachsen, ...).
Output schemas completely undocumented. No return type specifications visible for any tool. LLMs cannot determine what fields to expect in responses, forcing them to guess structure and breaking downstream tool chaining.
No error handling or recovery guidance. Descriptions lack actionable error messages, retry hints, or alternative tools to call on failure. E.g., if search() returns no results, should the LLM try list_* instead? Undocumented.
sparql_query description mentions 'available graphs: <graph_list>' but does not populate the list. LLMs cannot determine which graphs to use in FROM clauses without this critical context.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 65 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 56 | - | v1 |
Full-text search across all Lehrplan nodes by keyword. Uses prefix matching (e.g. 'Fisch' also finds 'Fische'). Returns matching nodes with their parent Lehrplan for context. Optionally filter by Bundesland and/or Schulfach.
Execute a SPARQL query against the MEM ontology triple store. PREFIX lp: <https://w3id.org/lehrplan/ontology/> is available. You MUST include FROM clauses for the graphs you need. Available graphs: <graph_list>
Result set limits not documented or enforced. Tools like find_lehrplaene, list_schulfaecher, and search may return unbounded lists, risking context window explosion. No pagination parameters visible.
Missing dependency and sequencing hints. Descriptions do not explain that get_lehrplan_tree requires output from find_lehrplaene (lehrplan_uri), or that get_children is a drill-down after get_lehrplan_tree. Forces LLM to discover usage patterns via trial.
Parameter 'bundesland' overloaded across multiple tools (list_schulfaecher, list_schularten, find_lehrplaene, search) with flexible input (code, name, URI), but no validation or error message documented if input is invalid.