Loading ToolBench
Server has 11 tools with basic schemas and descriptions, but significant gaps limit production readiness. All tools have descriptions (10-194 chars, within baseline range) and input schemas are present and properly typed using Zod. However, parameter descriptions are minimal or absent for most tools, only 'uuid' and 'tag' parameters have descriptions; 'force', 'settings' subfields, and many others lack guidance. Output schemas are not documented anywhere. Error handling is minimal: coolifyApiCall throws generic JSON errors without recovery guidance or categorization. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Tool composition is reasonable (each does one thing), but several tools lack context about when to use them vs. similar tools (e.g., list-resources vs. list-applications vs. list-services). No pagination support documented for list operations, risking context window exhaustion. Naming is verb-first and clear, but descriptions could better explain prerequisites and expected return structures.
Deploy an application or service using a tag or UUID. This allows you to deploy new versions or updates to your applications.
Retrieve detailed information about a specific application using its UUID. This includes the application's status, configuration, and deployment details.
Fetch a list of all applications currently managed by Coolify. This provides an overview of all deployed applications.
Fetch a list of all databases managed by Coolify. This provides an overview of all database instances.
Retrieve a list of all running deployments in Coolify. This includes details about the deployment status and history.
Retrieve a comprehensive list of all resources managed by Coolify. This includes applications, services, databases, and deployments.
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., does list-applications return application_id, app_uuid, or id?). Baseline: 100% of A+ tools document return types.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. start-application, stop-application, restart-application, deploy, and update-application are destructive but unmarked. Agents cannot distinguish safe retries from irreversible operations.
Parameter descriptions missing or incomplete. 'force' in deploy has description, but 'settings' object in update-application lacks guidance on which fields are required, what values are valid, or what happens if conflicting settings are passed. Baseline: 100% of A+ tool params have descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
Retrieve a list of all services managed by Coolify. This includes external services and microservices.
Restart a specific application using its UUID. This stops and then starts the application, applying any configuration changes.
Start a specific application using its UUID. This initiates the application and makes it available for use.
Stop a specific application using its UUID. This halts the application and makes it unavailable.
Update the settings of a specific application, such as health check configurations. This allows you to modify the application's behavior and monitoring settings.
Error handling is generic and non-actionable. coolifyApiCall throws JSON-stringified errors without recovery guidance. E.g., 'Coolify API error: 404' tells the LLM nothing about whether to retry, call a discovery tool, or ask the user. Baseline: errors must guide next steps.
No pagination support documented for list operations. list-resources, list-applications, list-services, list-databases, list-deployments do not accept limit/offset/page parameters. Large result sets risk context window exhaustion. Baseline: tools returning lists should support pagination.