Loading ToolBench
A Next.js-based interactive investigation game with declarative forms and WebMCP tool integration. Provides a desktop environment with applications (Mail, Messages, Photos, Browser, Terminal, Files, Evidence, SystemLog) for narrative-driven investigation gameplay.
Orpheus is a Next.js game/investigation UI with 9 tools exposed via a custom LINK console. Tool definitions are scattered across smoke test scripts and React components rather than centralized. Most tools lack formal input schemas, parameters are inferred from smoke test calls (e.g., 'application', 'query', 'command') but no JSON Schema is visible in the source. Descriptions exist but are brief (avg ~80 chars) and lack LLM-optimized guidance on when/why to use each tool. No output schemas documented. Error handling is minimal, tools return success/failure but do not guide recovery. Security concerns: terminal_command accepts arbitrary shell input with no sanitization. Composition is weak: open_application, open_directory, and show_in_document overlap in intent. Tool names are inconsistent in style (verb_noun vs. noun_verb). Overall, this reads as a game prototype, not a production-grade agent toolkit.
Retrieve metadata for a photograph by its id. Returns metadata about the photo.
Inspect and view details of a photograph. Declarative form for photo inspection.
Opens an application on the desktop. Accepts application names: mail, messages, photos, browser, terminal, systemlog, evidence, files.
Navigate to and open a directory in the file manager.
Record one already-discovered item on the shared Evidence board by its id. Ids come from get_case_evidence; invented ids are rejected.
Search the whole workstation for a term the player noticed. Returns how many files, messages, and emails mention it, so you know where to look next.
No formal JSON Schema visible for any tool. Input parameters inferred from smoke test calls only; no type definitions, constraints, or enums declared. Violates pattern:constrained-input and pattern:tool-description.
terminal_command accepts arbitrary shell input with no sanitization. Agents can inject commands, read arbitrary files, or execute destructive operations. No input validation, no allowlist, no sandboxing.
No output schemas documented. LLMs cannot predict what fields to expect from tool responses, forcing them to parse unstructured output or make assumptions about return types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
Navigate to and highlight a specific location in a document. Searches for a query term within the file.
Execute a command in the terminal. Supports filesystem navigation, file inspection, and vault unlocking via the 'unlock' command.
Attempt to unlock the vault with a passphrase. The passphrase is a space-separated sequence of words.
Overlapping tool intent: open_application, open_directory, and show_in_document all navigate/open resources. No clear distinction in naming or descriptions. LLMs will struggle to pick the right tool.
Descriptions are generic and lack LLM-optimized guidance. Most are 40 - 60 chars; none explain WHEN to use the tool, WHAT it returns, or HOW it differs from similar tools. Baseline for A+ tools is 50 - 200 chars with clear intent.