MCP server for Portkey admin operations, enabling management of users, workspaces, configurations, virtual keys, and analytics
Server has 8 tools with mostly complete schemas (Zod-based) and reasonable descriptions. Naming follows verb_noun convention well (list_*, get_*, invite_*). However, critical issues undermine quality: (1) Output schemas are NOT documented, tools return JSON.stringify'd text without declaring expected fields, making it impossible for LLMs to plan downstream calls or extract data reliably. (2) Error handling is minimal and non-actionable, catch-all error messages like 'Error fetching workspaces: {message}' provide no recovery guidance. (3) No pagination documentation despite tools accepting page_size/current_page parameters, unclear if tools return total counts, next_cursor, or other pagination signals. (4) Missing parameter constraints, e.g., get_user_stats has numeric filters but no explicit min/max bounds. (5) Sensitive operations (invite_user) lack dry-run/confirmation patterns. (6) No audit logging or permission gates visible. Tools are well-named and most input schemas are present (via Zod), but the lack of output schema documentation and weak error handling are significant gaps for production use.
Retrieve detailed cost analytics data over time, including total costs and averages per request
Retrieve detailed analytics data about user activity within a specified time range, including request counts and costs
Retrieve detailed information about a specific workspace, including its configuration, metadata, and user access details
Invite a new user to your Portkey organization with specific workspace access and API key permissions
List all users in your Portkey organization, including their roles and account details
Retrieve all configurations in your Portkey organization, including their status and workspace associations
Output schemas are completely undocumented. All tools return JSON.stringify'd text without declaring expected fields. LLMs cannot plan downstream calls, extract specific data, or understand what references are available for chaining.
Error handling is non-actionable. All errors caught in try-catch blocks return generic messages like 'Error fetching X: {error.message}' with no guidance on what to do next, whether to retry, or which alternative tools to call.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 51 | - | v1 |
Retrieve all virtual keys in your Portkey organization, including their usage limits, rate limits, and status
Retrieve all workspaces in your Portkey organization, including their configurations and metadata
Pagination behavior is unclear. list_all_users and list_configs accept no pagination parameters; list_workspaces and others accept page_size/current_page but tools do not document whether they return total counts, next_cursor, or how many items are actually returned when limits are exceeded.
Numeric parameters lack explicit bounds. get_user_stats and get_cost_analytics accept total_units_min/max and cost_min/max with no documented minimum or maximum allowable values. Unbounded numbers allow LLMs to pass absurd values (negative, zero, millions) that break API calls.
invite_user (WRITE operation) has no dry-run, confirmation, or idempotent pattern. An agent mistake invites real users without recovery path. No confirmation step documented.
No documentation of which tools are read-only vs destructive. LLMs need to know side effects upfront. Tool descriptions do not explicitly state whether they modify state.
list_configs description is vague ('Retrieve all configurations'). Does not explain what configurations are, what structure they have, or when to call this vs other list tools.
No visible permission gating or audit logging. Code does not show permission checks before executing sensitive operations (invite_user, list_all_users). No audit trail of who called what and when.