Loading ToolBench
Agentic BI toolkit for Looker and BigQuery — MCP server, LookML governance analysis, and BI engineering skills
18 tools with consistent verb-noun naming (strata_*). All have descriptions (avg ~120 chars, within baseline 34-392). Input schemas present for all tools with proper type definitions. However, output schemas are NOT documented, critical gap per pattern:tool. Parameter descriptions are present but terse (avg ~40 chars vs baseline 72). Error handling is minimal: most tools return {"error": "..."} without recovery guidance. No tool annotations (readOnlyHint/destructiveHint). One WRITE tool (strata_render_chart) lacks confirmation pattern. Composition is sound, each tool has single responsibility. Missing: pagination for list tools, per-item success/failure for batch operations, and structured error classification.
List available Vega-Lite chart templates (name, mark).
Get the current Conductor workflow status (active_slice, next_steps, latest_handoff).
Get the list of dead code items from L1 metadata.
Get explore dependencies including base view, joins, resolution chain, and field count.
Search for fields matching a query string across all views (query, kind, matches, count). Returns up to 50 matches.
Get the downstream impact of a physical table (physical_table, views, explores, fields).
Get the current IR (Intermediate Representation) status including repo path, build time, node counts, edge count, and cache path.
Output schemas not documented. LLMs cannot infer what fields to expect from responses, forcing them to guess at downstream tool chaining and risking context loss.
Error handling lacks recovery guidance. Tools return bare {"error": "..."} without suggesting next steps (e.g., 'Try search_users() first'). Agents cannot self-correct.
No tool annotations (readOnlyHint/destructiveHint). strata_render_chart is WRITE but unmarked. Agents cannot distinguish safe from destructive operations.
List tools (strata_list_orphans, strata_dead_code_register, strata_pdt_costs, strata_schema_drift) lack pagination (limit, offset, next_cursor). Large result sets will blow context windows.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
List orphaned views, explores, or fields (kind: all, view, explore, field).
List available skills (name, domain, mode, complexity, trigger).
Classify a ticket anchor and return a full navigator brief in one call. anchor: BQ table, field name, view, explore, or .lkml filename. Returns the views/explores/fields it touches with source_file:source_line citations, plus an inferred change type and what-to-touch list when ticket is set.
Get the PDT (Persistent Derived Table) ledger with cost information.
Query a specific field in a view to get its SQL, type, tags, source file, and resolution chain.
Render a chart using a Vega-Lite template and JSON data (path, status).
Get schema drift items detected between LookML and BigQuery.
Get the full SKILL.md content for a specific skill.
Get a summary of L1 usage data including explore count, total queries, dead code count, PDT counts, and schema drift count.
Determine blast radius for changes (impacted_views, impacted_explores, impacted_fields).
List views and their resolved physical tables (model_filter, views, count). Pass model= to restrict to views reachable from a specific model's explores.
Parameter descriptions are terse (avg ~40 chars vs baseline 72). Many lack format/constraint hints. E.g., 'kind' in strata_list_orphans should state 'one of: all, view, explore, field' in description, not just schema.