Loading ToolBench
Conversational governance agent for OpenMetadata. Standalone Python project using data-ai-sdk + LangGraph + OpenAI.
Server has 6 well-structured tools with complete JSON schemas and descriptions. All tools follow verb_noun naming (search_, get_, create_, assign_). Descriptions are present and actionable (avg ~120 chars). Parameters include type definitions, descriptions, and constraints (min/max for numeric fields, minLength for strings). However, output schemas are not documented, LLMs cannot predict response structure. Error handling guidance is absent. No tool annotations (readOnlyHint/destructiveHint) despite clear risk stratification (3 READ_ONLY, 3 WRITE). Parameter descriptions lack dependency hints and format guidance. Missing idempotency guarantees for write operations.
Assign a glossary term to an entity (table, column, dashboard, etc.) for semantic tagging and governance.
Create a new glossary in OpenMetadata with optional owners and reviewers for governance workflows.
Create a new term within a glossary, optionally nested under a parent term, with owners and reviewers.
Retrieve full entity details including metadata, owners, tags, and columns for a specific OpenMetadata entity.
Retrieve data lineage for an entity, including upstream and downstream dependencies with configurable traversal depth.
Natural language or keyword search query across OpenMetadata entities with optional filtering by entity type, pagination, and aggregations.
Output schemas not documented. LLMs cannot predict response structure, fields, or types. Breaks downstream tool chaining and forces agents to guess at field names.
No tool annotations despite clear risk stratification. create_glossary, create_glossary_term, and assign_glossary_term are WRITE operations but lack destructiveHint. READ_ONLY tools lack readOnlyHint. Agents cannot distinguish safe from risky calls.
No error handling guidance. Tools lack recovery hints (e.g., 'If entity not found, try search_metadata first'). Agents cannot self-correct on failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Parameter descriptions lack format guidance and dependency hints. E.g., 'fqn' (fully qualified name) format is unexplained. 'owners' and 'reviewers' arrays lack guidance on valid name formats (username vs email vs display name).
No idempotency guarantees documented for write operations. Agents retrying create_glossary or assign_glossary_term risk duplicate records. Missing confirmation/dry-run pattern for destructive operations.