Loading ToolBench
An MCP server for using the Supabase API
Supabase MCP server has structured tool definitions with Zod schemas converted to JSON Schema, but exhibits critical definition quality gaps across the rubric. Tool names follow verb_noun convention (list_, get_, create_, delete_) which is excellent, but descriptions are minimal (avg ~45 chars, below 194-char baseline) and lack actionable context. Critically, NO input parameter descriptions are visible in the tool registration, the Zod schemas define types but the corresponding descriptions for 'ref', 'name', 'organization_id', etc. are not present in the source code shown. Output schemas are completely undocumented. Error handling is minimal (basic Zod validation + generic 'Supabase API error'). Security concern: destructive tools (delete_project) have no confirmation or permission gating. The server exposes the Supabase API verbatim with minimal abstraction, accepts opaque 'ref' IDs rather than human-friendly project names, forces separate lookup calls, and returns entire API responses without result limiting or pagination.
Create a new organization
Create a new Supabase project
Delete a Supabase project
Get details of a specific organization
Get details of a specific Supabase project
Get API keys for a specific Supabase project
List all organizations
Parameter descriptions completely missing. Input schemas are registered with Zod types only, no descriptions visible for 'ref', 'name', 'organization_id', 'region', 'db_pass', 'plan', 'billing_email', 'slug'. LLM cannot infer whether 'ref' means 'project reference ID' or 'project name', what format it expects, or valid value ranges. This violates pattern:tool-description which requires descriptions for every parameter.
Tool descriptions are too terse (avg 45 chars vs 194-char baseline). 'List all Supabase projects' and 'Get details of a specific Supabase project' lack WHEN to use guidance, what the response contains, or dependencies. Descriptions do not state whether tools modify state. When should the LLM call it? What does it return?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 28 | - | v1 |
List all Supabase projects
Output schemas undocumented. Tool handlers return raw API responses (Project[], Organization[], ProjectApiKey[]) with no schema documentation visible to LLM. LLM cannot know what fields to extract or which fields downstream tools require. LLMs need to know what fields to expect so they can plan downstream tool calls.' Also no pagination, no result limiting, list_projects and list_organizations may return hundreds of items, bloating context.
API key required via environment variable (SUPABASE_API_KEY), but no secure management pattern documented. Server does not validate API key existence at startup clearly, will crash at runtime if missing. Per pattern:secret-injection, credentials should never appear in tool params (correct here), but server should fail gracefully and log clearly if auth fails.
Destructive operation delete_project has no confirmation step, permission gate, or dry-run option. An LLM can irrevocably delete a Supabase project with a single tool call. Per pattern:confirmation-request, irreversible operations should support confirmation before execute. No audit trail or logging of who called delete and when.
Tools accept opaque system IDs only (ref, slug) instead of human-friendly identifiers (project name, organization name). Per mxe:natural-identifiers, tools should accept usernames, emails, and display names alongside IDs. Requiring opaque IDs forces extra lookup calls and mismatches the chat data model. Example: delete_project expects 'ref' but user says 'delete my project called staging', agent must first search by name.
Error handling is minimal and unhelpful. catch block returns generic 'Supabase API error: {statusText}' with no actionable recovery guidance. Per pattern:recovery-guide, errors must tell LLM what to do next. If a project is not found, should LLM try a search? If auth fails, is the API key wrong? Zod validation errors are logged but not formatted for LLM clarity.
create_project requires 'db_pass' as a string parameter. Per pattern:secret-injection, passwords should NEVER be tool parameters, they will be logged in agent traces and appear in prompt history. This is a critical security leak. Database password must be either generated server-side, passed via secure out-of-band channel, or injected from vault after project creation.
No rate limiting, timeout configuration, or retry guidance. An LLM in a retry loop could hammer Supabase API thousands of times per minute. Per pattern scope-declaration and rate limiting best practices, server should implement per-agent rate limits and explicit timeout settings.
get_project_api_keys returns sensitive API keys (api_key field) directly in response. Per pattern:secret-injection, 'Tool responses must not include tokens, session IDs, or internal secrets. Anything in the response enters the LLM context and could be echoed to the user or logged.' Returning api_key in plain text risks leaking credentials to the LLM or end user.