Loading ToolBench
A basic Dedalus MCP server exposing GitHub and Supabase tools via the Dedalus MCP framework with credential encryption and JIT token exchange.
This server demonstrates solid definition quality with consistent naming, present descriptions, and well-structured schemas across 20 tools. Strengths: all tools follow verb_noun naming convention (gh_*, db_*), all have descriptions (avg ~80 chars), all parameters have types and descriptions, and output structures are generally clear. Weaknesses: descriptions are somewhat generic and lack WHEN/WHY context that LLMs need for tool selection; no enum constraints on string parameters that accept only a few values (e.g., gh_list_issues 'state' param); error handling guidance is absent; no documentation of output schemas or response structures; parameter descriptions lack format/range constraints. The GitHub and Supabase tools are functional but formulaic, they read like API documentation rather than LLM-optimized descriptions.
Delete rows from a Supabase table matching the specified filters
Get a single row by primary key from a Supabase table
Insert one or more rows into a Supabase table
Call a Supabase RPC (stored procedure/function)
Select rows from a Supabase table with optional filters, ordering, and pagination
Update rows in a Supabase table matching the specified filters
String parameters that accept only a fixed set of values lack enum constraints. Examples: gh_list_issues/gh_list_prs 'state' param accepts only 'open', 'closed', or 'all' but is defined as plain string with description. db_select, db_insert, db_update filters use free-form PostgREST strings. LLMs will hallucinate invalid values (e.g., 'state': 'pending') and waste retries.
Output schemas are not documented. The source code provides parameter input schemas but does not specify what fields are returned by each tool. LLMs cannot plan downstream calls or extract required data without knowing the response structure. Critical for chaining tools (e.g., gh_get_repo returns fields, then gh_put_file needs a ref, does the response include 'ref'?).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 67 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 61 | - | v1 |
Upsert rows into a Supabase table (insert or update on conflict)
Delete a file from a GitHub repository
Get file contents from a GitHub repository
Get a specific issue by number
Get a specific pull request by number
Get details for a specific GitHub repository
List issues in a GitHub repository
List pull requests in a GitHub repository
List repositories for the authenticated user
List GitHub Actions workflow runs
List GitHub Actions workflows in a repository
Create or update a file in a GitHub repository
Get the authenticated GitHub user's profile
Smoke test ping (no enclave dispatch required)
Descriptions are generic API documentation style, not LLM-optimized. They state WHAT the tool does but lack WHEN/WHY context. Example: 'List repositories for the authenticated user' (67 chars) does not explain when to use gh_list_repos vs gh_get_repo, or that it requires pagination. LLMs need: action + context + differentiation from similar tools.
No error handling guidance. Tools are defined with input schemas but provide no recovery instructions or error classification. If gh_get_repo returns 404, what should the LLM do? Try gh_list_repos? Check the owner/repo names? Silent failures lead to agent hallucination.
Parameter descriptions lack format/range/constraint documentation. Example: gh_get_file 'ref' param is described as 'Git ref (branch, tag, commit SHA)' but does not clarify: is 'main' valid? 'refs/heads/main'? Can it be a 40-char SHA or must it be shorter? db_select 'limit' and 'offset' are integers but no min/max are stated. LLMs guess and send invalid values.
gh_put_file and gh_delete_file are destructive operations (WRITE/DESTRUCTIVE risk) with no confirmation step or dry-run option. Agents can accidentally overwrite or delete files. No idempotency guarantees stated.
List tools (gh_list_*, db_select) do not document pagination limits or default page size. If an agent calls gh_list_repos without 'per_page', how many repos are returned? Is there a max limit? Can the LLM request 10,000 items? No guidance risks context window exhaustion or API throttling.
db_rpc 'params' is defined as generic 'object' type with no structure or examples. LLMs cannot infer what fields belong in 'params' for a given RPC function. This tool requires out-of-band knowledge of the stored procedure signature.