Loading ToolBench
An MCP server that executes Ruby code with optional gem dependencies, supporting STDIO, SSE, and Streamable HTTP transports
Single tool 'code_exec_ruby' has a complete input schema with proper types and descriptions, and a detailed tool description. However, the tool exhibits critical issues: (1) no output schema documentation despite a complex return structure (JSON with returncode, stdout, stderr), (2) no error handling guidance or recovery patterns, (3) missing tool annotations for the destructive/irreversible nature flagged in the risk field, (4) parameter 'packages' has nullable=true but no guidance on what happens when null vs empty array. The description is substantive (195 chars, within baseline 34-392) and mentions irreversibility and sandbox behavior, but lacks structured error classification and recovery steps that an LLM would need to handle failures. Tool naming 'code_exec_ruby' is adequately verb-noun oriented but generic, does not indicate scope (local sandbox, temporary dir, gem isolation). The server provides a working definition but falls short of production-grade error handling and output documentation expected at 70+.
Executes a Ruby code snippet with optional gem dependencies. When `use_temp_dir` is True, the code and any installed gems are run in a throwaway temporary directory, and gems are isolated. When False, gems are installed to ~/.gem. The Ruby runtime has access to networking, the filesystem, and standard libraries. A non-zero exit code is an error and should be fixed. Returns: JSON containing: - 'returncode': Exit status of the execution. - 'stdout': Captured standard output. - 'stderr': Captured standard error or install failure messages.
Output schema not documented. Tool description mentions it returns JSON with 'returncode', 'stdout', 'stderr' but no formal schema, types, or field descriptions are visible in code. LLMs cannot reason about output structure or plan downstream operations.
No error handling guidance or recovery patterns. Tool is marked Risk: IRREVERSIBLE but provides no guidance on how to handle non-zero exit codes, gem install failures, or sandbox errors. LLM gets raw stderr with no actionable next steps.
Missing tool annotations. FastMCP framework supports destructiveHint and idempotentHint (tool annotations), the tool should be marked destructive (code execution with side effects) and non-idempotent (running the same code twice may produce different results or side effects). These annotations are absent.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 63 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 67 | - | v1 |
Parameter 'packages' nullability not explained. Schema allows nullable: true, but tool description does not clarify: does null mean no gems, or is it equivalent to empty array []? What happens if an invalid gem name is passed? No validation error guidance.
No pagination or result limiting guidance. Tool description doesn't mention limits on stdout/stderr capture size. If code prints 100 MB of output, what happens? Unbounded text responses risk context window exhaustion.
Gem installation errors conflated with code execution errors. If gem install fails, stderr is returned but the tool description does not clarify whether the code was executed (it wasn't). LLMs may misinterpret 'gem not found' as a runtime error from the Ruby code.