Loading ToolBench
MCP server for managing local services, workers, and MCP servers in a self-hosted application platform. Provides tools for app discovery, lifecycle management, configuration, registration, and activity logging.
Collabhost MCP server demonstrates solid tool design with consistent naming (verb_noun pattern), comprehensive parameter descriptions, and well-structured schemas. All 6 tools follow action-verb conventions (get_, list_, update_). Descriptions are substantive (100-250 chars), explaining WHAT, WHEN, and prerequisites. Input schemas include type definitions and descriptions for all parameters. However, output schemas are not explicitly documented in the visible code, and error handling guidance is minimal. The server lacks tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications. Parameter descriptions are strong but could benefit from explicit format/constraint documentation (e.g., enum values for 'status' filter in list_apps).
Returns detailed information about a specific application including identity, status, process info, route URL, capabilities, technology probe data (detected runtime, frameworks, dependencies), and 'writableDataPath' (a platform-provisioned absolute directory inside Collabhost's writable data root -- point any app that writes to disk at this path rather than its install directory). Use this to inspect an app's current state before taking action. Probe data is cached and refreshed on app start and config changes.
Returns the schema-driven settings for an application, organized by capability. Each section includes the capability name, the schema defining valid fields, and the current effective values (defaults merged with overrides). Use this before update_settings to understand what configuration the app supports. A setting marked 'requiresRestart' means the app must be restarted after changing it.
Returns system hostname, Collabhost version, and uptime in seconds. Use this to identify the host and verify the platform version. This tool does not require any apps to be registered. Format the uptime locally if a human-readable form is needed.
Lists all registered applications with their current status, type, and route URL. Use this to discover app slugs before calling other app-specific tools. The 'status' filter narrows results to apps in a specific state.
Output schemas not documented. Tool descriptions claim return fields (e.g., 'status, type, route URL' for list_apps; 'writableDataPath' for get_app) but no explicit schema definition is visible in source. LLMs cannot plan downstream tool calls without knowing return structure.
Tool annotations missing. Despite clear risk classifications (READ_ONLY vs WRITE), tools lack readOnlyHint and destructiveHint annotations. This prevents agents from reasoning about safety and idempotency.
Enum constraints not formalized. list_apps 'status' parameter description lists valid values ('running, stopped, crashed, backoff, fatal') as prose rather than JSON Schema enum. LLMs may hallucinate invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
Lists recent activity events (state changes, operator actions, system events). Use to understand what happened to an app or who performed an action. Filter by app slug, event type, or category. Returns newest events first.
Saves capability override settings for an application. Only provided fields are changed -- omitted fields retain their current values. Settings must conform to the schema from get_settings. Some settings require an app restart to take effect (check the 'requiresRestart' flag in get_settings). Use get_settings first to see the schema and current values.
Error handling guidance absent. No visible error responses or recovery hints in tool definitions. If a tool fails (e.g., app not found, invalid settings), LLMs have no guidance on next steps.
Pagination not visible. list_apps and list_events accept 'limit' but no 'offset', 'cursor', or 'total' fields documented. Large result sets could exhaust context without pagination support.