Loading ToolBench
Custom MCP Server on Databricks Apps
Server has 4 tools with explicit registration via @mcp_server.tool decorator in server/tools.py. All tools have descriptions and basic schemas. However, significant gaps exist: tool naming lacks consistency in verb patterns, parameter descriptions are minimal, output schemas are not formally documented, and error handling is weak. The 'add_numbers' tool (sampled in query_remote.py) suggests tools may be registered in multiple files, creating verification uncertainty. Overall definition quality is fair but below production baseline.
Add two numbers together. This tool performs basic arithmetic addition of two numeric values. It accepts both integers and floating-point numbers. Useful for: - Mathematical calculations - Summing values - Basic arithmetic operations
Get information about the current authenticated user. This tool retrieves details about the user who is currently authenticated with the MCP server. When deployed as a Databricks App, this returns information about the end user making the request. When running locally, it returns information about the developer's Databricks identity. Useful for: - Personalizing responses based on the user - Authorization checks - Audit logging - User-specific operations
Check the health of the MCP server and Databricks connection. This is a simple diagnostic tool that confirms the server is running properly. It's useful for: - Monitoring and health checks - Testing the MCP connection - Verifying the server is responsive
Trigger a Databricks job run. This tool starts a new run of an existing Databricks job. It uses app-level authentication (service principal) to trigger the job. The job must exist and the app must have permission to run it. Useful for: - Automating job execution - Triggering data pipelines - Running scheduled tasks on-demand - Integrating workflows with AI assistants
Output schemas not formally documented. Tools return dicts with fields (e.g., 'status', 'message', 'result', 'run_id'), but no JSON Schema specification is visible for response structures. LLMs cannot infer response field types or plan downstream tool calls.
Parameter descriptions are minimal and lack constraint details. 'a' and 'b' in add_numbers only say 'The first/second number to add', no mention of range, precision, negative value handling. 'job_id' in trigger_job_run lacks guidance on where to find job IDs (e.g., via UI path or API listing).
Error handling is minimal. get_current_user catches exceptions and returns {'error': str(e), ...}, which includes raw exception text, opaque to LLMs. No guidance on recovery steps, no classification (retryable vs. user-fixable), no suggestion to try alternatives.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | - | v1 |
tool 'trigger_job_run' is destructive (WRITE risk) but lacks confirmation/dry-run mechanism. Agents can trigger production jobs without safeguards. No mention of rollback or undo capability if the job fails or produces unintended side effects.
Tool registration location is ambiguous. server/tools.py defines tools via @mcp_server.tool decorator, but scripts/dev/query_remote.py also references 'add_numbers' and 'trigger_job_run'. Unclear if these are duplicates, if query_remote.py is a test harness, or if tools are loaded from multiple sources. Cannot definitively verify the single source of truth.
No pagination or result limit documented. If get_current_user or other tools return lists in future iterations, no guidance on handling large result sets, pagination cursors, or token budget management.
No tool annotations visible (readOnlyHint, destructiveHint, idempotentHint). The rubric explicitly requires these for protocol alignment. 'health' and 'get_current_user' should be marked read-only; 'trigger_job_run' should be marked destructive.