Loading ToolBench
Arduino Lab for MicroPython MCP-compatible REST server — run scripts, manage files and eval Python on any MicroPython board over serial
Server exposes 15 tools with basic schemas and descriptions. All tools have names starting with action verbs (list_, run, exec, eval, upload, download, ls, rm, reset, install_, validate, build, compile, flash, list_). Descriptions are present but brief (10-50 chars for most), lacking context on WHEN to use each tool or what happens on failure. Input schemas are visible and typed, but parameter descriptions are minimal (e.g., 'Device port or auto for default' repeated 12 times). No output schemas documented. Error handling is absent, no guidance on retryability, recovery steps, or actionable error messages. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (DESTRUCTIVE, IRREVERSIBLE). Path traversal guard present (resolveWorkspace), but no input validation on other parameters (e.g., package names, Python code strings). Composition is reasonable, tools are single-purpose, but chaining is weak: no tool returns device IDs or board metadata that downstream tools need.
Build an Arduino sketch
Compile an Arduino sketch
Download a file from the board to workspace
Evaluate a Python expression on the board and return the result
Execute a Python code string on the board
Flash a compiled sketch to an Arduino board
Install a MicroPython package via mip on the board
Descriptions are too brief (10-50 chars) and lack context. Most do not explain WHEN to use the tool, what happens on success/failure, or dependencies. E.g., 'Run a .py file from the workspace on the board' omits that it executes from RAM, not flash (which IS mentioned in the description but buried). LLMs cannot reliably select tools without richer guidance.
Parameter descriptions are generic and repeated. 'Device port or auto for default' appears 12 times verbatim. No guidance on format (e.g., '/dev/ttyACM0' vs 'COM3'), valid ranges, or when to omit. Parameter descriptions should be 50-100 chars and specific to context.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
List connected MicroPython boards via mpremote
List available firmware images for flashing
List files on the board filesystem
Reset the board
Remove a file or directory from the board
Run a .py file from the workspace on the board (executes from RAM, not flash)
Upload a file from workspace to the board
Validate an Arduino sketch
No output schemas documented. Tools return stdout/stderr/exitCode but LLMs do not know the structure. E.g., does 'eval' return a JSON object, a string, or raw Python output? Does 'list_boards' return an array of objects with what fields? Undocumented outputs force LLMs to guess and parse unstructured text.
No tool annotations despite clear risk levels. 'rm' is DESTRUCTIVE, 'flash' is IRREVERSIBLE, 'eval' is READ_ONLY. MCP spec supports readOnlyHint, destructiveHint, and idempotentHint annotations. Without them, agents cannot reason about side effects or retry safety.
No error handling guidance. Tools fail silently (e.g., device not found, invalid Python syntax, file not found). Responses do not indicate retryability, recovery steps, or what the LLM should do next. E.g., 'Device not found' should suggest 'Call list_boards() to see available devices.'