Loading ToolBench
AI daemon with MCP client support, bot management, conversation management, and skill execution
AIDaemon presents 22 tools with mixed quality. All tools have descriptions and visible schema definitions via Java annotations (@Tool, @ToolParam), which is a strong foundation. However, there are significant gaps: (1) Parameter descriptions are often generic or terse (many under 50 chars), failing to explain WHEN or WHY to use the tool. (2) Output schemas are not explicitly documented, the code shows parameters but returns are inferred from method signatures (e.g., String return for createBot). (3) Several tools combine multiple concerns (e.g., writeToConversation both writes AND notifies participants). (4) Error handling guidance is minimal, errors are caught but not classified or actionable. (5) Enums are not declared for constrained inputs (e.g., cron expressions, image sizes are documented in description only, not as enum constraints). (6) Natural identifiers (participant names, bot names) are used, which is good, but conversationId is opaque. The median tool score is ~58, reflecting decent naming and basic descriptions but weak output documentation and composition issues.
Add a participant to an existing conversation.
Add a Smithery-hosted MCP server to the framework. Provide the Smithery server page URL (e.g. https://smithery.ai/servers/googlecalendar) or the server slug (e.g. googlecalendar). If the server requires OAuth, returns an authorization URL for the user to open; after authorizing, ask the user to reload MCPs.
Cancel a scheduled job by its ID.
Create a new bot with a given name and soul (personality description). The soul defines the bot's character, expertise, and behaviour.
Create a new group conversation with specified participants. Returns the conversation ID — save it to memory for future reference.
Create a new file with the specified content. Shows a diff preview to the user before writing. Only works when shell access is enabled.
Delete an existing file. Shows the current file content to the user before deleting. Only works when shell access is enabled.
Output schemas are not formally documented. Many tools return inferred types (String, array) with no schema specification of fields, types, or structure. LLMs cannot plan downstream calls or extract nested data without knowing response structure.
Constrained inputs declared in descriptions, not as enum/pattern constraints. E.g., generateImage width/height are 'mentioned' as 1024 or 1792 in description text, but not declared as enum. Cron expressions in scheduleJob have examples but no regex pattern validation. LLMs may pass invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 53 | - | v1 |
Edit specific lines of an existing file by providing a list of line-number/newContent pairs. Each pair replaces that line (1-based) with the given content. Shows a diff preview to the user before writing. Only works when shell access is enabled.
Execute a shell command on the host system and return its output. Use this to run CLI tools like git, gh, docker, curl, etc.
Generate an image using DALL-E 3. Returns a markdown image reference to include in the answer. Supported sizes: 1024x1024 (default, square), 1792x1024 (landscape), 1024x1792 (portrait).
List all available participants you can invite to conversations: 'user' (the human) and other bot names.
List all currently scheduled jobs.
List all conversations you are a participant in.
List all files inside a specific skill folder.
List all available skills (top-level folders in the skills directory).
Modify an existing file by replacing its entire content. Shows a diff preview to the user before writing. Only works when shell access is enabled.
Read a file from a skill folder. Path is relative to the skills root, e.g. 'github-cli/instructions.md'.
Retrieve older conversation messages by index range. Provide startIndex (inclusive); optionally endIndex (exclusive). If endIndex is omitted, returns all messages from startIndex to the end. Result is capped by context window. Returns: startIndex, actualEndIndex (exclusive), and the message list.
Save information to persistent memory. Use when the user asks you to remember something.
Schedule a recurring or one-time job. The job will run at the specified frequency and execute the given instruction by sending it to the AI.
Search the Smithery MCP server registry by keyword (e.g. gmail, calendar, notion). Returns one page of results; use page parameter for more. Use addSmitheryMcp with the server URL or slug to add one.
Write a message to a conversation and specify which bot participants to wake up (trigger async response). Use an empty list to just write without triggering anyone. The conversation ID must be one you are a participant in. Save conversation IDs to memory so you can reference them later.
Error handling is minimal, no guidance on recovery, classification, or actionable next steps. E.g., cancelJob (job not found), readSkillFile (file not found), addParticipant (participant not found) show no error responses or suggestions.
Tool composition issues: writeToConversation combines two concerns (write message + notify participants). Should split into separate tools so agent can call each independently. Similarly, editing tools (createFile, modifyFile, editFile) have overlapping responsibilities.
Preconditions not enforced: tools like createFile, modifyFile, deleteFile, editFile mention 'Only works when shell access is enabled' in description, but no input validation or permission check visible in code. If shell access is disabled, what error is returned?
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) visible in Java code. deleteFile and executeCommand are clearly destructive/irreversible but not marked. This requires LLMs to infer risk from descriptions.
Parameter descriptions are often terse (under 50 chars), lacking context on format, range, dependencies, or validation rules. E.g., 'participantName' in addParticipant is just 'Participant to add: user or a bot name', no guidance on case sensitivity or ID vs name resolution.
No idempotency or retry guidance. Tools like createBot, createConversation, scheduleJob do not state whether repeated calls with same input are safe or will create duplicates. Agents may retry on ambiguous failures, causing unintended side effects.