Loading ToolBench
Semantic database monitoring and AI-powered data querying for Beacon projects. Provides tools for natural language questions, SQL generation, validation, documentation, and data exploration.
Beacon MCP has 8 tools with mixed quality. Strengths: all tools have descriptions (10 - 200+ chars), most are action-verb named (get_, ask, search, etc.), and the server implements read-only safety gates. Weaknesses: 3 of 8 tools (query, get_documentation, search) have minimal descriptions (<50 chars, some generic); input schemas are visible for only 5 of 8 tools in the provided code; no output schemas documented; error handling guidance is absent; no tool annotations (readOnlyHint/destructiveHint). The 'ask' and 'dry_run' tools are well-described and have complete schemas, but the majority lack depth. Average tool score: 62.
Ask a natural language question about your data or project. For data queries, Beacon auto-detects the right data source(s), generates SQL, and executes it. For conceptual questions (e.g., 'how do notifications work?'), it answers from project documentation and knowledge base.
Validate a SQL query through all of Beacon's safety gates — the read-only gate (regex guardrail + AST validation), schema column check, and a provider dry-run (EXPLAIN) — without executing it. Returns the exact SQL that would run (with the row limit applied) and any issues found. If the data source has no extracted schema metadata yet, the schema gate reports an advisory issue and the verdict is invalid (the column check could not be performed); the provider dry-run still runs. Engines without a provider dry-run strategy (e.g. SQLite) report that gate as skipped with an advisory issue, so a query that could not be validated is never reported as valid. Use before query.
Record whether a previous `ask` answer was correct. Pass the signal_id from that ask response. A 'correct' verdict is saved as a verified example that improves future answers. Only include a note when the user explicitly asked to record one.
Get an overview of the project: its data sources, schemas, tables, quality scores, and documentation status. This is the starting point for understanding what data is available.
Three tools (query, get_documentation, search) have minimal descriptions (<20 chars) and no visible input schemas in source code.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract structured data without knowing response field names and types.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. The 'feedback' tool is WRITE but lacks destructiveHint; read-only tools lack readOnlyHint. Agents cannot distinguish safe from risky operations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2025-06-18+ | v2 |
Get Documentation
Get the grounding context Beacon uses for SQL generation, scoped to your question: table schemas with real sample values, verified join paths, human-verified example queries, learned patterns, and the business glossary. Use it to write your own SQL, then validate with dry_run and execute with query.
Run Read-Only Query
Search Catalog
Error handling lacks recovery guidance. No tool description explains what to do if a call fails (e.g., 'If dry_run reports invalid SQL, call get_query_context to refine your question').
Parameter descriptions for 'project_id' are identical across tools ('Optional. Specify project if your API key has access to multiple projects.') but lack guidance on when it is required vs optional, or how to discover available project IDs.