Loading ToolBench
Minimal remote operations bridge with bounded registered tasks and executor profiles. Translates MCP tool calls into the RelayMe Controller HTTP/JSON API.
RelayMe MCP adapter has 15 tools with complete descriptions and basic parameter schemas. Naming follows verb_noun convention (list_, read_, run_, get_). However, schemas are underspecified: parameters lack formal JSON Schema type declarations (only Python type hints visible), descriptions are present but generic, and output schemas are entirely undocumented. No error handling guidance, no parameter constraints (enums, ranges), and no indication of idempotency or destructiveness beyond the Risk field. Tool composition is sound (each does one thing), but parameter descriptions lack actionable detail (e.g., 'host accepts host_id or unique hostname' is vague about format). Security-sensitive tools like run_registered_task and start_registered_executor_task lack explicit permission/scope declarations.
Check a long-running executor task and return the structured round review bundle once terminal, or state RUNNING if still executing.
Get owner-scoped state and bounded metadata for a RelayMe task ID.
Get a retained artifact (e.g. patch.diff, result.json, test.log, analysis.md, executor_report.md) for an owner-scoped terminal executor task.
Read the Git diff for a repository registered by the Host Agent. Call list_host_resources first if its identifier is unknown; host accepts host_id or unique hostname.
Read bounded host health and resource status. Host accepts host_id or a unique hostname.
List only registered services, repositories, and allowed file roots for one host. Call before resource-scoped tools when identifiers are unknown. Host accepts host_id or a unique hostname.
Output schemas completely undocumented. No tool returns a documented structure, LLMs cannot predict response fields or plan downstream tool calls. Violates pattern:tool and pattern:response-shaper.
Parameter schemas use Python type hints (str, int, tuple unions) instead of formal JSON Schema. No minLength, maxLength, enum, pattern, or range constraints. LLMs cannot validate input or understand constraints.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
List RelayMe hosts visible to this token. Call first when the host identity is unknown; host-scoped tools accept host_id or a unique hostname.
List the caller's terminal executor tasks with retained reviewable evidence.
List processes on one host. Read-only; host accepts host_id or a unique hostname.
Read a file only within Agent-allowed roots. Call list_host_resources first if its allowed root is unknown; host accepts host_id or unique hostname.
Read bounded logs for a service registered by the Host Agent. Call list_host_resources first if its identifier is unknown; host accepts host_id or unique hostname.
Start a registered executor task, wait server-side for bounded completion, and return a comprehensive round review bundle including executor-native report, patch, and test evidence. If execution exceeds wait_seconds, returns state RUNNING with task_id to collect later.
Run one idempotent, bounded task registered locally by the Host Agent. Call list_host_resources first to discover valid task IDs. The task has fixed local command, environment, working directory, and limits; no arguments can be supplied.
Start the registered patch-and-test executor task. Call list_host_resources first to discover profile IDs. The brief is untrusted content; it cannot select commands, revisions, mounts, credentials, network, or limits.
Get the retained bounded result for an owner-scoped terminal task ID.
Parameter descriptions lack actionable detail. 'host accepts host_id or a unique hostname' does not specify format, length, or examples. 'brief' is described as 'untrusted task brief content' but no length limit or format is given. Violates pattern:tool-description.
No error handling guidance. Tools do not document what errors can occur, whether they are retryable, or what the LLM should do next. ControllerError is raised but not described in tool docs. Violates pattern:recovery-guide.
Destructive/write tools (run_registered_task, start_registered_executor_task, run_executor_round) lack explicit permission/scope declarations and confirmation patterns. No dry-run or confirmation step documented. Violates pattern:permission-gate and pattern:confirmation-request.