Loading ToolBench
The server registers 4 tools with basic descriptions and parameter schemas, but critical gaps prevent higher scoring. Tool names are moderately clear (verb_noun pattern present) but descriptions are minimal and lack LLM-optimization guidance. Parameter schemas exist but lack comprehensive descriptions for all fields. No output schemas documented. Error handling is present in code but not surfaced in tool definitions. Security concerns around credential handling in parameters and destructive operations without confirmation. The 'where' parameter accepts untyped 'object', and 'data' accepts 'any' type, violating schema rigor baselines.
Manage Supabase authentication (create user, sign in, sign out, etc.)
Insert, update or delete data
Select data from a table
Upload or download files from Supabase Storage
Credentials exposed as tool parameters: 'password' in auth tool and 'projectUrl' in all tools violate secret-injection pattern. Secrets in parameters are logged in agent traces and prompt history.
Destructive operations (mutate delete action, auth signout) lack confirmation or dry-run step. Agents make mistakes, a delete without confirmation risks data loss.
Parameter 'where' in select/mutate tools accepts untyped 'object' with vague description 'Filter conditions'. LLMs cannot infer valid filter syntax (eq, neq, gt, lt, etc.), invites hallucinated filter keys.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Parameter 'data' in mutate and storage tools accepts type 'any'. This violates schema rigor, LLMs cannot validate input shape. For mutate, should be object with known keys; for storage, should be string (base64) with length bounds.
No output schemas documented in tool definitions. LLMs cannot predict what fields the response contains or plan downstream tool calls (select returns 'data' and 'count', but mutate returns unknown structure).
Tool descriptions are too brief (17-65 chars) and lack LLM-guidance. 'Manage Supabase authentication' does not explain when to call auth vs select, what happens on error, or password format requirements. Baseline is 50-200 chars with actionable guidance.
Tool names are not consistently verb_noun. 'storage' and 'auth' are nouns, not actions, LLMs infer less clearly. Should be 'upload_file', 'download_file', 'manage_auth' or split into 'create_user', 'signin_user', etc.
mutate tool combines insert, update, delete into one operation. Should split into create_row, update_row, delete_row so agents can compose independently and error handling is clear.
No numeric bounds on 'limit' parameter. Unbounded limit can cause memory exhaustion or timeout. Should restrict to 1 - 1000 with default 100 (currently defaults to 100 in code but not declared in schema).
storage tool combines upload and download into one operation enum. Should split into 'upload_file' and 'download_file' tools with distinct parameters (upload needs base64 data, download needs path only).