Loading ToolBench
Custom minimal AI agent with memory, MCP tools, and Discord integration. Runs on a homelab with persistent memory, bash execution, file I/O, web search/fetch, and delegation to sub-agents.
Luna Agent provides 7 tools with reasonable naming and descriptions, but significant gaps prevent a higher score. All tools have descriptions (10 - 194 chars, within baseline range) and input schemas with parameter types. However, output schemas are completely undocumented, no tool explicitly declares what it returns, forcing LLMs to infer structure. Error handling is minimal: bash tool caps output at 50KB but provides no guidance on recovery; write_file restricts writes to workspace but lacks actionable error messages. Tool descriptions lack dependency hints and prerequisites. No tool annotation hints (readOnlyHint, destructiveHint, idempotentHint) despite clear risk profiles (bash=IRREVERSIBLE, write_file=WRITE). Bash tool accepts command strings without visible input validation, creating injection risk. Overall, the server demonstrates partial patterns compliance but fails on output schema documentation (a critical requirement for agent reasoning) and lacks proper error guidance.
Execute a bash command. Use for system commands, git, package management, etc.
List all available tools (native and MCP-registered).
List files and directories at a path. Relative paths resolve to the workspace directory.
Read a file's contents. Supports offset/limit for large files. Relative paths resolve to the workspace directory.
Fetch a web page and extract content as markdown.
Search the web using DuckDuckGo and return results.
Write content to a file. Creates parent directories if needed. Relative paths resolve to the workspace directory. Writes outside the workspace are not allowed.
NO OUTPUT SCHEMAS DOCUMENTED. All 7 tools lack documented return types. LLMs cannot plan downstream calls or extract required fields (e.g., file paths from list_directory, search result structure from web_search). This violates the pattern:tool and mxe:response-field-naming rules.
BASH INJECTION RISK. bash tool accepts raw command strings with no visible validation or sanitization. Source code shows timeout/cwd params but no filtering against shell metacharacters or command injection attempts. LLMs can be tricked via prompt injection into passing 'rm -rf /' or similar destructive commands.
MISSING TOOL ANNOTATIONS. bash (IRREVERSIBLE), write_file (WRITE), and list_directory/read_file (READ_ONLY) lack readOnlyHint/destructiveHint/idempotentHint annotations. MCP spec requires these to help clients enforce safety policies and plan confirmations.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 31 | - | v1 |
MINIMAL ERROR GUIDANCE. bash tool caps output at 50KB and returns truncated results without recovery hint. write_file and web_fetch lack actionable error messages ('permission denied' needs 'try different path or user' context). No pattern:recovery-guide implementation.
WEB_FETCH/WEB_SEARCH LACK OUTPUT STRUCTURE DOCS. Both accept freeform 'prompt' and 'query' but nowhere document what fields to expect in responses (e.g., title, snippet, url, rank order). LLMs must guess whether results are sorted by relevance or recency.
LIST_DIRECTORY MAX_ENTRIES (500) NOT DOCUMENTED. Parameter descriptions omit that recursive listing is capped at 500 entries and max_depth defaults to 3. Large dirs silently truncate without pagination or overflow hint, violating mxe:enforce-result-limits.
BASH TIMEOUT DEFAULTS POORLY DOCUMENTED. Timeout range (30 - 120s) appears in schema description but not in tool description. LLM has no context on when to lower/raise timeout for different command classes.
WRITE_FILE WORKSPACE RESTRICTION NOT ACTIONABLE. Description says 'Writes outside the workspace are not allowed' but lacks actionable error format. If an LLM tries '/etc/passwd', what exact error is returned? This violates review:actionable-errors.