Loading ToolBench
MCP server for VibeKit — deploy apps, manage hosting, chat with AI agents, and run coding tasks from Claude Desktop, Cursor, and other MCP clients.
VibeKit MCP has 27 tools with consistent naming (verb_noun pattern: vibekit_list_apps, vibekit_create_app, etc.) and all tools have non-empty descriptions. Input schemas are defined for all tools with required parameters clearly marked. However, significant gaps exist: (1) Parameter descriptions are minimal or absent for most tools, many parameters like 'appId', 'template', 'subdomain' lack context explaining what values are valid, format constraints, or when to use each parameter. (2) Output schemas are not documented, tools return JSON but callers cannot predict field structure, types, or what IDs/references are available for chaining. (3) Error handling is implicit in the apiRequest helper but not surfaced in tool documentation, no guidance on retry semantics, what errors mean, or recovery steps. (4) Tool descriptions average ~50 chars, which is below the 194-char baseline and often lack prerequisites, when to use the tool, or what it returns. Example: 'vibekit_chat' is described as 'Chat with an app's AI agent to modify its code' (57 chars), it omits that this is async, may timeout, or that users should check vibekit_agent_status first. (5) No security annotations visible (permissions, scopes, audit requirements). Tools managing databases (vibekit_db_query), deleting apps (vibekit_delete_app), and submitting tasks (vibekit_submit_task) lack confirmation patterns or scope declarations. The tools.json file is referenced but not provided in source code review, tool definitions may have richer metadata there, but cannot be verified. Conservative estimate assumes definitions in tools.json match the source listings.
Get the conversation history with an app's AI agent
Get the status of an app's AI agent
Get environment variables for an app
Get application logs
Chat with an app's AI agent to modify its code
Create a new hosted app from a template
Get database status for an app
Execute a SQL query on an app's database
Output schemas not documented. Tools return JSON objects but callers cannot predict field structure, types, or what IDs/references are available for tool chaining. Example: vibekit_list_apps returns 'data' but what fields does each app object contain? Is there a 'appId' field, 'status', 'url', 'created_at'? Without this, LLMs cannot plan follow-up calls (e.g., passing appId to vibekit_get_app).
Parameter descriptions missing or incomplete. Parameters like 'appId', 'template', 'subdomain', 'repo', 'branch' lack context about format constraints, valid values, or when/how to obtain them. Example: vibekit_create_app accepts 'template' (string) but no description explains what template names are valid, callers must call vibekit_list_templates first without knowing to do so. Another example: 'lines' in vibekit_app_logs says 'Number of log lines to retrieve (default 100)' but omits min/max bounds or whether 0 means unlimited.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | 2025-06-18+ | v2 |
Get the database schema for an app
Query a specific table in an app's database
Delete an app (destructive operation)
Deploy a GitHub repository
Enable Postgres database for an app
Get details of a specific app
Get the status and details of a submitted task
List all hosted apps
List deployment history for an app
List available app templates
Get the status of QA tests for an app
Redeploy an existing app
Restart an app
Rollback to a previous deployment
Run quality assurance tests on an app
Set environment variables for an app
Start a stopped app
Stop a running app
Submit an autonomous coding task to be executed and deployed
Tool descriptions are too brief (average ~50 chars vs. 194-char baseline). Many descriptions lack prerequisites, context on when to use the tool, or what it returns. Example: vibekit_chat is 'Chat with an app's AI agent to modify its code' (57 chars), this omits that the operation is async, may timeout waiting for the turn to complete (CHAT_MAX_WAIT_MS=110s), and callers should check vibekit_agent_status afterward. Another: vibekit_set_env is just 'Set environment variables for an app' (35 chars), it doesn't explain when to call this vs. vibekit_app_env, or whether partial updates merge with existing vars or replace them entirely.
No error handling guidance in tool definitions. The apiRequest helper logs errors but tools do not document what errors are retryable, what mean a user must fix input, or recovery steps. Example: vibekit_delete_app may fail with 'Insufficient permissions' or 'App not found', the tool should describe both cases and hint at how to resolve. Currently, LLMs receive raw error strings with no classification.
No confirmation or dry-run for destructive operations. vibekit_delete_app directly deletes with no confirmation step or idempotency hint. vibekit_db_query executes arbitrary SQL (DDL/DML) on a production database with no dry-run or confirmation. vibekit_submit_task auto-deploys by default (deploy=true) with no confirmation. These tools should support a 'confirm' parameter or Multi-Round-Trip Request pattern to ask the user before executing.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). The spec supports annotations to signal to clients which tools are safe, which are destructive, and which are idempotent. None are present in source code. This forces LLMs to infer intent from the description alone, increasing the risk of unintended side effects. Example: vibekit_restart_app should be marked idempotent; vibekit_delete_app should be marked destructive.
API key exposed as environment variable without documented guidance. VIBEKIT_API_KEY is read from process.env but tool docs do not explain where keys come from (Telegram bot /apikey command is mentioned in SERVER_INSTRUCTIONS but not in individual tool docs). If an LLM logs tool invocations or a client caches responses, the key could be leaked. Documentation should advise users to rotate keys and clarify that keys should never appear in tool parameters.
Missing pagination guidance for list tools. vibekit_list_apps, vibekit_list_templates, vibekit_list_deploys return lists but input schemas have no 'limit', 'page', 'offset', or 'cursor' parameters documented. If an account has 500 apps, vibekit_list_apps may return all of them, bloating the response and exhausting context. Conversely, if pagination is required but the tool doesn't document it, LLMs will call the tool repeatedly without knowing when to stop.