Loading ToolBench
Desktop AI-coding assistant powered by Flowise with MCP server capabilities for file operations, chat management, and LLM proxy
Orcheus AI presents a hybrid desktop+backend architecture with 32 tools spanning IPC handlers, HTTP endpoints, and MCP proxies. While tool names follow verb_noun conventions (auth:sign-in, chats:create, files:read), critical quality gaps severely limit production readiness. Most tools lack input validation guidance, parameter constraints are absent (no enums for role, status fields), and output schemas are undocumented. Descriptions are present but generic (avg ~60 chars, well below the 194-char baseline for A+ tools). No error recovery guidance, no permission declarations, and secrets (FLOWISE_TOKEN, SUPABASE keys) are exposed in environment config without clear injection patterns. The MCP endpoints (/mcp/tools/list, /mcp/tools/call) are thin proxies that delegate to backend handlers, making tool definitions inferred rather than directly visible in MCP registration code.
Reset token usage statistics
Get public configuration for client
Stream-based file generation endpoint with SSE
Get token usage statistics
Health check endpoint returning server status and configuration
Stream-based file generation endpoint with SSE (POST variant)
Predict endpoint - calls Flowise prediction API
MCP endpoint to call a tool
Output schemas undocumented. No tool specifies what fields are returned, forcing LLMs to guess structure and plan downstream calls blindly. Violates pattern:tool and pattern:response-shaper.
Parameter constraints missing. 'role' in messages:save accepts free-form strings (no enum for user|assistant|system). 'status' fields lack enums. LLMs will hallucinate invalid values. Violates pattern:constrained-input.
Secrets exposed in environment config (FLOWISE_TOKEN, SUPABASE_ANON_KEY). No evidence of server-side injection or vault integration. Tokens could leak into agent traces and logs. Violates pattern:secret-injection.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
MCP endpoint to list available tools
OpenAI-compatible chat completions endpoint (LLM proxy)
OpenAI-compatible completions endpoint (LLM proxy)
Get current authenticated user information
Sign in user with email and password
Sign out current user
Sign up new user with email and password
Create a new chat session
Delete a chat session
List all chats for current user
Load messages from a specific chat
Show folder picker dialog
List files in project directory tree
Read file contents
Write content to file
Stream-based file generation from Flowise with progress events
Cancel ongoing stream-based file generation
Get token usage statistics from backend
Call Flowise prediction API for code generation
Save a message to a chat
Write file to project directory with validation
Load application settings
Save application settings
Open project folder in system file explorer
No error recovery guidance. Tools return generic descriptions without actionable next steps. E.g., 'Sign in user' does not explain what to do if credentials fail or user not found. Violates pattern:recovery-guide.
Tool definitions inferred from MCP proxy endpoints (/mcp/tools/list, /mcp/tools/call). Actual tool registration code not visible in source. Per HARD SCORING RULE, inferred tools capped at 50 overall. Cannot verify schema completeness or description quality at registration layer.