Loading ToolBench
Repo-native HTTP workflows for developers, scripts, and coding agents via the runmark CLI and runmark mcp stdio server.
Runmark MCP server has 18 well-named tools with consistent verb_noun patterns (list_, describe_, run_, get_, etc.). All tools have descriptions (avg ~80 chars, within baseline 34-392 range). Input schemas are present and typed for all tools. However, descriptions are terse and lack LLM-optimized guidance on WHEN to use each tool, dependencies between tools, and recovery paths. Output schemas are not documented in the source. Error handling is not visible, no guidance on retryability, user-fixable vs fatal errors, or actionable recovery messages. Parameter descriptions are minimal (e.g., 'Optional project root directory path' repeated 18 times). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk classifications in the metadata. Missing pagination guidance for list_* tools and no documented limits on result sizes.
Persist the latest captured response body as the declared snapshot for one step
Cancel an in-progress or paused session
Clean up persisted runtime state and session artifacts
Compile and materialize one request for inspection without sending HTTP
Compile one run and return a simplified view of its executable step graph
Explain variable resolution and precedence for a request or run
Export a complete audit trail of all sessions and their execution details
Retrieve the current state of a persisted session
Output schemas not documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., what does list_requests return? Is there pagination? What fields are in each item?). This forces agents to guess or make exploratory calls.
Tool annotations missing. Destructive tools (clean_runtime, accept_snapshot, resume_session, cancel_session) lack destructiveHint. Read-only tools lack readOnlyHint. Idempotent tools lack idempotentHint. LLMs cannot determine retry safety or side-effect severity without these hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
List all tracked environment definitions in the project
List all tracked request definitions in the project
List all tracked run definitions in the project
List all artifacts captured during a session execution
List all persisted runtime sessions for the project
Read the content of a specific artifact from a session
Resume execution of a paused or failed session from the next step
Start a fresh session for a single request definition and execute it
Start a fresh session for a full run definition and execute all steps
Validate all tracked project files without compiling or executing
Descriptions are terse and lack LLM-optimized guidance. Most descriptions are 40-60 chars (below baseline 194 avg). They state WHAT but not WHEN to use, dependencies, or prerequisites. E.g., 'List all tracked request definitions' does not explain when to call this vs describe_request, or that it should be called first to discover available requests.
Parameter descriptions are generic and repeated. 'Optional project root directory path' appears 18 times verbatim. No guidance on format, constraints, or when to omit. envId parameter lacks explanation of how it affects variable resolution or what happens if omitted.
No error handling guidance. No indication of which errors are retryable, which require user intervention, or what recovery steps to take. E.g., if run_request fails, should the agent retry, call validate_project first, or ask the user? No actionable error messages visible.