Loading ToolBench
Server provides 4 well-structured tools for Supabase CRUD operations with reasonable descriptions and input schemas. However, several definition quality gaps prevent a higher score: (1) Tool names are adequately clear (read_table_rows, create_table_records, update_table_records, delete_table_records) but generic, they lack domain specificity that would clarify when to use each. (2) Descriptions are present and moderate length (126-184 chars), explaining WHAT each tool does and providing context, but they lack explicit WHEN to use guidance and do not mention prerequisites or dependencies. (3) Input schemas are defined with proper JSON types (string, object, integer, boolean) and include descriptions for all parameters, but lack critical constraints: no enums where applicable (filters lack validation guidance), numeric bounds missing (limit parameter unbounded), and no validation rules for table_name or column names. (4) Parameter descriptions are generic, 'filters' is described as 'Dictionary of column-value pairs' but doesn't explain the query syntax (e.g., 'Supabase uses .eq() for equality; pass {"status": "active"}'). (5) Output schemas are undocumented, tools return dicts or lists but the response structure (especially for read_table_rows' List[Dict] or create_table_records' Dict with 'data', 'count', 'status' keys) is not explicitly documented. (6) Error handling is absent, no guidance on what to do if a table doesn't exist, if a filter is malformed, or if a query times out. (7) Security: credentials are correctly injected via environment (SUPABASE_URL, SUPABASE_SERVICE_KEY), not as parameters; however, no rate limiting, audit logging, or permission checks are visible. (8) Composition: tools are appropriately single-responsibility and idempotent (read twice = same result; update with same filters twice = same result), but no batch variants exist despite agents often needing multi-record operations.
Create one or multiple records in a Supabase table. Use this tool to insert new data into a specific table in the Supabase database. You can insert a single record or multiple records at once.
Delete records from a Supabase table that match the specified filters. Use this tool to remove data from a specific table in the Supabase database. You provide filter conditions to identify which records to delete.
Read rows from a Supabase table with optional filtering, ordering, and limiting. Use this tool to query data from a specific table in the Supabase database. You can select specific columns, filter rows based on conditions, limit the number of results, and order the results.
Update records in a Supabase table that match the specified filters. Use this tool to modify existing data in a specific table in the Supabase database. You provide the new values and filter conditions to identify which records to update.
No output schema documentation. Tools return complex dicts (e.g., {"data": [...], "count": 0, "status": "success"}) but the response structure is not explicitly documented. LLMs cannot plan downstream tool calls or extract fields without knowing what to expect.
Parameter constraints and validation rules missing. 'filters' parameter lacks guidance on syntax (e.g., does it use Supabase .eq() or raw SQL?). 'limit' parameter is unbounded (no min/max). 'table_name' and 'columns' accept any string without validation hints. LLMs will guess at correct syntax and pass invalid table names.
No error handling or recovery guidance. Tools have no try-catch or exception documentation. LLMs have no guidance on what to do if table not found, filter malformed, or query times out. Raw errors (if any) will confuse the agent.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 46 | - | v1 |
Descriptions lack WHEN-to-use guidance and prerequisites. Each tool explains WHAT it does but not WHEN an agent should select it. E.g., read_table_rows doesn't explain 'Use this after learning available tables' or 'Combine with filters to find specific records.' This forces LLMs to infer intent.
No confirmation or dry-run for destructive operations. delete_table_records modifies state irreversibly but has no confirmation step or dry-run capability. An agent error deletes data silently. Pattern suggests confirmation-request for destructive tools.
Tools lack result limits and pagination guidance. read_table_rows accepts 'limit' but no guidance on default or max. Returning unbounded lists can exhaust context windows. No mention of pagination for large result sets.
No audit logging or permission checks visible. Tools operate on arbitrary tables with no logging of who called what, when, or with which parameters. No permission gates to prevent unauthorized access. Compliance and incident response are impossible.