Loading ToolBench
MCP server for semantic SQL exploration with access control, query caching, async execution, and semantic concept mapping across PostgreSQL tables
Server has 16 well-named tools with consistent verb_noun patterns (query, list, describe, suggest, compare, cache). Descriptions are present and substantive (avg ~120 chars), explaining WHAT and WHEN. Input schemas are visible with type declarations and defaults. However, output schemas are undocumented, LLMs cannot predict response structure. Error handling is minimal; most tools lack recovery guidance. Security is strong (RLS enforcement, permission gates on cache_invalidate), but parameter descriptions could be more prescriptive about constraints and formats. No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications.
Clear the entire query cache. Directors and admins only. Use after schema changes or when stale results are suspected.
Show query cache statistics: total entries, live vs expired, and total cache hits. Use this to demonstrate caching effectiveness during a demo or performance review.
Show how a concept maps across teams — e.g. 'money' maps to deal_value, spend, amount depending on which team's table you're looking at. Returns aggregates from each accessible table.
Return every table+column tagged with a given concept, filtered to tables you can access.
Return columns and indexes for a table.
Get the Postgres query plan with your RLS context applied. Set analyze=True to run EXPLAIN ANALYZE.
Output schemas are not documented. LLMs cannot predict response structure (e.g., what fields does query() return? Is it {rows, columns, row_count} or {data, metadata}?). This forces agents to guess and risks parsing errors.
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are missing despite clear risk classifications. cache_invalidate is marked WRITE but has no destructiveHint. query_async and get_task_result lack idempotentHint despite being safe to retry.
Error handling lacks recovery guidance. Tools call external services (semantic_mapper, access_control) but do not document what to do if they fail. E.g., suggest_join and list_concepts will raise urllib exceptions with no actionable message for the LLM.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Retrieve the status and result of a background query submitted with query_async. Status values: pending → running → done | error. When status is 'done', the result field contains columns, rows, and row_count.
List all semantic concepts (customer, geography, money, etc.).
Return all user-facing schemas in the database.
List tables visible to the current user.
Run a read-only SQL query. RLS is enforced at the database level based on your role.
Run a query in the background and return a task_id immediately. Use for long-running queries (>5s). Call get_task_result(task_id) to fetch results.
Show recent query history from the audit log. Analysts and managers see only their own queries. Directors and admins see all users' queries.
Ask the Semantic Mapper which columns to JOIN on between two tables.
Return the table description and per-column comments. Use to understand column semantics and cross-table mappings.
Return the current user's identity, role, team, and accessible tables.
Parameter descriptions lack prescriptive constraints. 'sql' parameter in query() and explain() has no mention of read-only enforcement, query timeout, or forbidden keywords. LLMs may attempt DDL or DML despite the tool claiming read-only.
Pagination is missing from list_* tools. list_schemas, list_tables, list_concepts, and concept_columns return unbounded lists. If a database has 1000+ tables or concepts, the response will blow the context window.