Loading ToolBench
A Java-based MCP (Model Context Protocol) server for data integration and metadata management. Provides tools for searching tables/terms, executing SQL, updating metadata, and managing custom parameterized SQL tools.
DatI MCP Server defines 7 tools with clear verb-based naming and basic structure. All tools have descriptions and input schemas are visible. However, parameter descriptions are minimal (most are 1-3 words), output schemas are not documented, and error handling patterns are absent. Tools follow a logical data discovery → analysis → enrichment workflow appropriate for data exploration, but lack the depth and precision required for production-grade agent integration. Parameter validation, response chaining, and recovery guidance are missing. The server is well-structured for a domain-specific tool (metadata exploration), but falls short of A-grade quality due to incomplete descriptions and undocumented output contracts.
Execute an SQL query or statement against a data source and return result rows. Use this tool only when no specialized or custom analysis tool is available for your task.
Get detailed column schemas (names, types, comments, and sample values) for specified tables (up to 20 tables).
List all available tables (with schema, name, and description; overview only, no columns) and business terms under the service scope. Use this tool when exploring the catalog or when keyword search returns no matches.
Search relevant database tables, column names, sample values, and business terms by keywords. Always use this tool first to discover the right tables and terms instead of guessing tables or running exploratory SQL.
Enrich column descriptions (e.g. business meanings, enum mappings) or aliases after analysis to improve future queries.
Output schemas not documented for any tool. LLMs cannot plan downstream calls or extract required data without knowing what fields to expect. e.g., search_tables_and_terms and get_table_schema do not document what fields they return (table_id? table_name? column_info? sample_values?)
Parameter descriptions are minimal or trivial (typically 1 - 3 words: 'Search keywords', 'Table identifiers', 'SQL query or statement'). Lack context for when/how to use parameters. e.g., 'keywords' in search_tables_and_terms should explain: What constitutes a keyword? Can it include table prefixes? Does it support regex or only exact/substring match? Are there performance limits?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Enrich table description or aliases after analysis to improve future queries and search accuracy.
Create or update a business glossary term (definition, calculation rules, or aliases) under a subject after analysis.
No error handling or recovery guidance. No indication of retryable vs fatal errors, or what the LLM should do if a tool fails. e.g., what happens if execute_sql times out? Returns a connection error? Hits a permission wall? Should the agent retry, try a simpler query, or ask for help?
No idempotency guarantees for metadata update tools. update_table_metadata and update_column_metadata do not declare whether repeated calls with identical inputs are idempotent. Agents may retry on transient failures and cause unintended double-updates.
execute_sql tool lacks result pagination and size limits. No indication of max rows returned, or whether paginated results are supported. Large result sets can exhaust context windows.
Parameter names lack type suffixes for ambiguous identifiers. 'tableId', 'columnId', 'subjectId' are opaque, unclear whether they accept human-readable names, UUIDs, numeric IDs, or schema-qualified names (schema.table). No guidance on format.
Tool descriptions lack 'when to use' guidance. e.g., list_tables_and_terms says 'Use when exploring the catalog or keyword search returns no matches', but execute_sql does not clarify when to use it vs search-based tools, what performance costs apply, or what SQL dialects are supported.
No explicit permission or scope declaration. execute_sql is marked WRITE (risky) but no indication of what scopes or roles are required. No audit trail guidance.