Loading ToolBench
AI-ready metadata governance and semantic layer platform with MCP servers for database connectivity, metadata cataloging, and data product management
RunContext provides 14 tools across database and semantic layer operations. Overall definition quality is mixed: tool names follow verb_noun conventions (db_list_schemas, db_query, search, explain), but parameter schemas and descriptions have significant gaps. All tools are marked READ_ONLY, which is good for safety. However, the server lacks input validation details, output schema documentation, and error recovery guidance. Most tools have descriptions (10-12 tools visible), but descriptions are often brief (20-100 chars) and lack context for LLM decision-making. Parameter descriptions exist but many lack type clarity and constraints. No tool exhibits destructive operations, which is positive for a data access layer. The semantic layer tools (search, explain, validate, tier, golden_query, guardrails) are particularly under-documented relative to the database tools.
Describe a table's columns including data types, nullability, and primary keys
List all schemas (databases/namespaces) available in the connected database
List all tables and views with row counts, types, and schema information
Execute a read-only SQL query. Only SELECT statements are allowed. Automatically enforces row limits and timeouts.
List foreign key relationships. Optionally filter by a specific table.
Sample rows from a table (max 100 rows). Read-only, with automatic row limit guardrails.
Get detailed explanation of a specific model, field, metric, or entity in the semantic layer
Semantic layer tools (search, explain, validate, tier, golden_query, guardrails, get_product) have vague descriptions under 50 characters. LLM cannot distinguish when to call 'search' vs 'explain' vs 'validate' without better contextual guidance. E.g., 'search' description lacks what it searches across (models, fields, metrics all mentioned but not distinguished).
No output schemas documented for any tool. The rubric requires all tools to document what fields they return so LLMs can plan downstream calls and extract the right data. E.g., db_list_tables returns 'row counts, types, and schema information' but the exact JSON structure is not specified. This breaks composition (pattern:tool-chain).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 62 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 29 | - | v1 |
Get detailed metadata for a specific data product
Get the golden query (canonical SQL) for a model or metric
List and describe all data guardrails and access policies defined in the semantic layer
List all data products available in the catalog
Search the semantic layer for models, fields, metrics, and entities by keyword
Calculate and report the Bronze/Silver/Gold tier for a model based on metadata completeness and quality
Validate semantic metadata against OSI standards and return detailed validation results
Result limits documented in descriptions (db_sample_values: max 100, db_query: max 1000) but no pagination parameters (page, offset, limit, next_cursor) visible in input schemas. For tools returning lists, pagination is critical to avoid context window exhaustion (pattern:paginated-result). Schema shows no support for limit control on list_products or db_list_tables.
Error handling guidance is missing. db_query description states 'Automatically enforces row limits and timeouts' but does not explain what happens when a timeout occurs, what error message the LLM will receive, or what to do next. No recovery hints provided (pattern:recovery-guide).
Parameter descriptions lack format specifications and constraints. db_query 'sql' parameter description is 'SQL query to execute (SELECT only)' but does not specify: maximum query length, forbidden keywords beyond SELECT (subqueries? CTEs?), or example format. db_describe_table 'table' param says 'e.g. "users" or "public.users"' which is an example value in description, LLMs may reuse these literally (anti-pattern mxe:response-field-naming).
Semantic layer tools have unclear scope boundaries. 'search' description says 'Search the semantic layer for models, fields, metrics, and entities' but does not explain: Are these categories mutually exclusive? Can a search for 'user' return both a User model AND a user_id field? Should the query be a regex, free-text, or exact match? This ambiguity forces LLM guessing.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) visible in schema. All tools are READ_ONLY and idempotent, which should be declared explicitly in the tool definition for LLM clarity (pattern:tool, Spec Alignment 2026-07-28).