Loading ToolBench
An AI orchestration system that exposes tools for code editing, Arduino/ESP32 firmware management, session memory, and system automation to Gemini Live and autonomous agents.
O.R.I.O.N has 15 tools with generally clear naming (verb_noun pattern: codex_read, arduino_compile, sessions_list) and comprehensive descriptions (avg ~150 chars). However, critical gaps exist: tool annotations (readOnlyHint, destructiveHint) are missing despite clear risk levels documented in metadata; parameter descriptions are present but inconsistent in depth; output schemas are not explicitly documented in the source; error handling guidance is minimal. The codex_* tools (1-6) are well-structured with proper constraints (offset/limit pagination), but arduino_* tools (7-10) lack detailed error recovery paths. Session tools (11-13) and ask_user/use_skill (14-15) have good descriptions but no documented return schemas. Risk classification is visible in metadata but not exposed to LLMs via tool annotations.
Compila un sketch del repo (carpeta bajo arduino/) para ESP32. Devuelve si compiló y cuánta flash usa. NO flashea nada — para eso está arduino_upload.
Escucha el monitor serie de la placa unos segundos y devuelve lo que imprimió (logs del firmware, lecturas de sensores, errores de boot). Opcionalmente manda una línea de texto antes de escuchar (para firmwares con consola de comandos). No usar mientras se está subiendo firmware al mismo puerto.
Estado del entorno Arduino: si arduino-cli está instalado, qué placas (ESP32/Arduino) hay conectadas por USB (puerto, modelo, FQBN) y qué sketches del repo se pueden compilar/subir. Usala antes de compilar o subir para saber puerto y nombres.
Compila y FLASHEA un sketch del repo a la placa conectada en el puerto dado (ej: COM5). Esto reemplaza el firmware que la placa tenga — confirmá con el usuario el sketch y el puerto antes de usarla. Obtené el puerto con arduino_status.
Asks the user a clarification question with multiple-choice options. Use BEFORE executing a task when the request is ambiguous and you need a specific detail: time period, language, depth (fast/deep), source type, output format, scope, target audience, etc. The user picks an option (or types their own if allow_other is true) and the choice comes back as the tool result so you can continue. Best practice: keep it to 2-4 mutually exclusive options. Examples: '¿Qué período histórico te interesa?' with ['Colonial 1532-1821', 'Independencia 1821-1824', 'Toda la conquista']. '¿Búsqueda rápida o profunda?' with ['Rápida (5 fuentes)', 'Profunda (20+ fuentes vía NotebookLM)'].
Tool annotations missing: readOnlyHint, destructiveHint, idempotentHint not exposed to LLMs despite clear risk classification in metadata (READ_ONLY, WRITE, DESTRUCTIVE). LLMs cannot infer safety properties without explicit annotations.
Output schemas not documented in source code. Tool descriptions state what they return (e.g., 'returns matching lines with file paths and line numbers') but formal JSON Schema output types are absent, forcing LLMs to infer structure.
Error handling lacks recovery guidance. arduino_upload description warns 'confirmá con el usuario' but no error classification (retryable vs fatal) or actionable next steps documented. codex_bash silently fails on shell metacharacters without suggesting alternatives.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | <=2025-11-25 | v2 |
Run a shell command scoped to the workspace. Commands are parsed with shlex (no shell metacharacters like &&, |, >, ; allowed — use multiple calls if needed). Useful for git, npm, pip, or other CLI tools. Fails on commands that try to escape the workspace.
Edit a text file by replacing an exact string. The old_string MUST match exactly once in the file (unless replace_all=true). Include enough surrounding context in old_string to make the match unique. This is safer than diffs because it forces you to read the file first. Preserves indentation exactly as-is.
Search for files in the workspace by glob pattern (e.g. '*.py', 'src/**/*.ts'). Returns relative paths. Skips common noise directories (.git, node_modules, __pycache__, etc.). Useful to explore the codebase before reading or editing.
Search for a regex pattern in all workspace files (skipping noise dirs). Returns matching lines with file paths and line numbers. Useful to find usages, TODOs, or specific code patterns.
Read a text file from the current workspace and return its contents with line numbers (starting at 1). Use for source code, config, README, etc. Fails on binary files and on files outside the workspace root. If the file is large, use offset and limit to page through it.
Create a new file (or overwrite an existing one) with the given content. Use codex_edit for surgical changes; use codex_write when the file is new or when the change is a full rewrite. Creates parent directories if needed.
Devuelve los mensajes de una conversación pasada por su id (obtené el id con sessions_list). Útil para recordar qué se habló o retomar un tema anterior.
Lista las conversaciones pasadas guardadas (id, título, fecha, cantidad de mensajes, canal). Usala para ubicar una conversación antes de pedir su historial con session_history.
Busca un texto en TODAS las conversaciones pasadas y devuelve los mensajes que lo mencionan (con fecha y conversación). Usala cuando el usuario pregunte por algo que se habló antes pero no sepas en qué conversación fue.
Loads a SKILL.md recipe as context for the next step. Use when the user request matches one of the installed skills (see catalog in your system prompt). The returned markdown teaches the next agent how to combine existing tools (shell, files, web) to fulfill the task. Skills are NOT executable on their own — after calling use_skill, the next step must be generated_code or another tool that actually runs the instructions.
Parameter descriptions inconsistent in depth. codex_read/codex_edit have detailed constraints (offset 1-indexed, limit max 2000); arduino_compile/arduino_upload lack equivalent detail. ask_user options array structure documented but return type (user's choice) not formally specified.
Pagination not uniformly implemented. sessions_list and session_history accept limit but no cursor/offset for continuation. codex_read uses offset/limit correctly. Inconsistency forces LLMs to reason about different pagination patterns.