Loading ToolBench
Tracks documented and undocumented Microsoft Graph API schema changes across Entra ID, Entra ID Governance, Identity & Access Management, Entra Agent ID, and Information Protection — with an interactive MCP Apps schema visualiser.
Graph Atlas MCP has well-structured tool definitions with consistent naming (verb_noun pattern: search_, get_, visualize_, expand_) and comprehensive descriptions (avg 180 chars, within 10-1024 baseline). All 12 tools have input schemas with type definitions and enums for constrained parameters (endpoint, object_kind, view, filter_by). However, several tools lack output schema documentation, and some parameter descriptions are generic. Error handling is minimal, no recovery guidance or actionable error messages visible in source. Tools 10-12 are inferred from app-ui bridge code rather than explicit server registration, capping their scores at 50. No tool annotations (readOnlyHint, destructiveHint) despite all being read-only operations.
Expand a node in the schema graph to show its relationships (navigation properties, inheritance, permissions, roles).
Get detailed information about a specific schema change, including before/after snapshots and impact analysis.
Retrieve the change timeline for a specific entity type, showing all recorded modifications over time.
Retrieve the complete change history for a specific entity type, property, permission, or role across all snapshots.
Retrieve permission and role context for a Microsoft Graph entity type, including which permissions touch it and which roles grant those permissions.
Retrieve the most recent Microsoft Graph schema changes detected, optionally filtered by entity type, property, permission, or role.
About this server — running version, its changelog entry, data freshness (latest snapshots and newest detected changes), and how releases work: the snapshot DATABASE is re-released daily as calendar-tagged GitHub releases (e.g. v2026.08.07) when changes are detected, while the SERVER code ships on npm with semver versions (e.g. v0.1.0). Call for "what version is this?", "what's new in the latest Graph Atlas release?", or to check whether the local data is current.
Tools 10-12 (expand_schema_node, search_schema_objects, get_node_timeline) are inferred from app-ui/atlas-app/bridge.ts rather than explicitly registered in server code. Tool definitions are not directly visible in src/tools/ directory, making schema and error handling verification impossible.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., change_id from search_changes to pass to get_change_detail). Violates pattern:tool and mxe:include-chaining-ids.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all 12 tools being read-only operations. Agents cannot infer safety properties without explicit hints. Misses current spec pattern (2026-07-28).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2025-06-18+ | v2 |
Retrieve a summary of a specific Microsoft Graph API snapshot, including entity type count, property count, and enum count.
Generate a comprehensive report of schema changes over a time period, with statistics and categorization.
Search for Microsoft Graph schema changes by entity type, property, permission, or role name across snapshots and change history.
Search for entity types, permissions, or roles in the Microsoft Graph schema by name or description.
Launch the interactive schema visualizer with an initial view of entity types, permissions, or roles and their relationships.
Parameter descriptions are generic or missing context. E.g., 'filter_by' in get_recent_changes lacks guidance on when to use each enum value (entity_type vs property vs permission vs role). Violates pattern:tool-description.
No error handling or recovery guidance visible in source. Tools return raw API responses without actionable error messages. E.g., if change_id is invalid, no guidance on how to find valid IDs. Violates pattern:recovery-guide.