Loading ToolBench
MCP server for interacting with Apple's App Store Connect API
The server has 25 tools with consistent naming, reasonable descriptions, and input schemas. Naming follows verb_noun conventions (list_, get_, create_, delete_, search_). Descriptions are present and action-oriented but often lack strategic guidance on when/why to use each tool. Input schemas are well-typed with parameter descriptions, but output schemas are not documented in the tool definitions themselves. Error handling is minimal, no recovery guidance, retryability classification, or actionable error messages visible. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite having a clear write/destructive risk taxonomy. The server handles complex domain concepts (App Store Connect APIs) but does not expose helper tools to resolve human-friendly names (email, username) into IDs, all tools require system IDs.
[Analytics/Requests] List analytics report requests for an app. Default limit is 50 to prevent response size issues. Increase limit up to 200 for more results.
[TestFlight] Get detailed information about a specific crash submission.
[TestFlight] Get the raw crash log text for a specific crash submission.
[TestFlight] List crash submissions from beta testers. Default limit is 25 due to large response size. Max 200. Use pagination metadata for additional pages.
[TestFlight] Search crash submissions with advanced filtering. Default limit is 25 due to large response size. Max 200. Use pagination metadata for additional pages.
[Analytics/Data] Download analytics report data to a TSV file. This tool fetches all segments for a report instance and saves the data to a TSV file. The file can then be analyzed using other tools. Args: instance_id: The analytics report instance ID output_path: Optional path for the output file. If not provided, saves to temp directory Returns: Dict containing: - status: "success", "no_data", or "error" - file_path: Path to the downloaded TSV file - file_size_mb: File size in megabytes - segment_count: Number of segments downloaded - row_count: Number of data rows (excluding header) - message: Error message if status is "error" or "no_data"
Destructive operations (users_delete, user_invitations_delete) lack confirmation/dry-run patterns and error guidance. No mention of retryability, blast radius, or recovery steps.
Write/mutation tools (report_requests_create, users_modify, user_invitations_create) have vague parameter descriptions. 'request_data' and 'user_data' are too generic, no indication of required fields, valid keys, or constraints. LLMs cannot know what to pass without reading the API docs.
No tool annotations present (readOnlyHint, destructiveHint, idempotentHint). Server has explicit risk taxonomy (READ_ONLY, WRITE, DESTRUCTIVE, REVERSIBLE) but does not expose it to clients via MCP tool metadata. Clients cannot easily determine which tools are safe to retry or have side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 69 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
[Analytics/Reports] Get detailed information about a specific analytics report instance.
[Analytics/Segments] List segments for a specific analytics report instance. Default limit is 100 (lightweight data). Max 200. Use pagination metadata for additional pages.
[Analytics/Requests] Create a new analytics report request.
[Analytics/Requests] Get detailed information about a specific analytics report request.
[Analytics/Reports] List reports for a specific analytics report request. Default limit is 50 to prevent response size issues. Increase limit up to 200 for more results.
[Analytics/Segments] Get detailed information about a specific analytics report segment.
[Analytics/Reports] Get detailed information about a specific analytics report.
[Analytics/Reports] List instances for a specific analytics report. Default limit is 100 (lightweight data). Max 200. Use pagination metadata for additional pages.
[App] Get detailed information about a specific customer review.
[App] List customer reviews for an app.
[App] Search customer reviews with advanced filtering.
[Users/Invitations] Create a new user invitation. Args: invitation_data: Dictionary containing the invitation details include: Optional list of related resources to include visible_apps_limit: Optional limit for visibleApps (1-50) Returns: Created user invitation information (expires in ~3 days)
[Users/Invitations] Cancel/delete a user invitation. Args: invitation_id: The ID of the invitation to cancel Returns: Confirmation of cancellation
[Users/Invitations] Get detailed information about a specific user invitation.
[Users/Invitations] List all user invitations. Default limit is 50. Max 200. Use pagination metadata for additional pages. Supported sort: email, -email, lastName, -lastName
[Users/Management] Remove a user account. Args: user_id: The ID of the user to delete Returns: Confirmation of deletion
[Users/Management] Get detailed information about a specific user.
[Users/Management] List all users in the organization. Default limit is 50. Max 200. Use pagination metadata for additional pages.
[Users/Management] Modify a user account. Args: user_id: The ID of the user to modify user_data: Dictionary containing the fields to update include: Optional list of related resources to include Returns: Updated user information
Output schemas are not documented in tool definitions. LLMs do not know what fields to expect from responses, making downstream tool chaining and result parsing error-prone. E.g., report_instances_download_data returns 'status', 'file_path', 'file_size_mb', 'segment_count', 'row_count', 'message' but this is only visible in the docstring, not in a machine-readable schema.
All tools require system IDs (app_id, request_id, user_id, etc.). No helper tools to resolve human-friendly names (email, username) into IDs. Agents must either know IDs in advance or waste lookup calls. Design does not match chat data model (users say 'John', not 'U12345').
Parameter 'include' appears in many tools (analytics, reviews, crashes, users) with description 'Related resources to include in response' but no enum of valid values or guidance on when to use. LLMs cannot know what's available to request without API knowledge.
Error handling is not visible in tool definitions. No guidance on which errors are retryable, how to recover from 'not found', or what to do on rate limits. Source code may handle this, but it is not declared to clients.
Search tools (reviews_search, crashes_search) accept arrays of filter values (rating, territory, device_platform, etc.) but do not indicate whether multiple values are AND or OR. LLMs cannot compose correct queries.