Loading ToolBench
A standalone skills execution engine for LLM agents with MCP server support
SkillEngine defines 5 tools with partial schema coverage. Memory tools (recall_memory, save_memory, explore_memory, add_knowledge) have reasonable descriptions (50-150 chars) and input schemas with type definitions and enums. However, apply_patch has a verbose, instruction-heavy description (500+ chars) that reads like API documentation rather than LLM guidance. Output schemas are undocumented across all tools, LLMs cannot infer what fields to expect. Parameter descriptions exist but lack actionable constraints (e.g., no min/max for limit, no format guidance for path). Error handling is minimal; tools return generic strings like '[Memory unavailable]' without recovery guidance. No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. Composition is reasonable, tools are single-purpose, but chaining is unclear (e.g., does add_knowledge return an ID for later recall?).
Index a local file or directory into the knowledge base so it can be recalled later.
Apply one file change per call using top-level fields: type, path, diff. type must be create_file, update_file, or delete_file. diff is required for create_file and update_file. Diff prefixes: '+' add line, '-' remove line, ' ' unchanged context line. Each diff line must strictly match [+-<space>]line format. '-' and ' ' entries must contain one complete line after the prefix. Multiple [+-<space>]line entries must be separated by '\n'. For update_file, a diff with only '+' lines inserts content at the top of the file; to update existing content, include context using ' ' or '-' lines. Example (top insert): original='b\nc\n', call={"type":"update_file","path":"notes.txt","diff":"+a"}, result='a\nb\nc\n'. Example (targeted update): original='line1\nline2\n', call={"type":"update_file","path":"notes.txt","diff":" line1\n-line2\n+line two"}, result='line1\nline two\n'.
Browse the memory filesystem to discover what memories exist. Useful for understanding what has been remembered.
Search your memories for relevant information. Use this to recall user preferences, past decisions, entity details, or prior context.
Save important information to persistent memory. Use this for user preferences, key decisions, entity details, or patterns worth remembering.
Output schemas undocumented. All 5 tools return strings or untyped responses. LLMs cannot plan downstream calls or extract structured data. recall_memory returns formatted text; add_knowledge returns success/failure string; apply_patch returns patch result, none declare field types or structure.
apply_patch description is 500+ chars of instruction-level detail ('Diff prefixes: + add line, - remove line...') rather than LLM-friendly guidance. Should state: 'Apply a single file mutation (create/update/delete) using unified diff format. One call per file.'
No tool annotations despite clear risk levels. apply_patch is DESTRUCTIVE; save_memory and add_knowledge are WRITE; recall_memory and explore_memory are READ_ONLY. Annotations enable agents to reason about side effects and retry safety.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Error handling returns generic strings ('[Memory unavailable]', '[No memories found]') without recovery guidance. LLMs cannot determine if the error is retryable, user-fixable, or fatal.
Parameter constraints missing. 'limit' has no min/max (baseline: 1 - 100). 'path' in add_knowledge has no format guidance (file vs directory?). 'scope' enum is clear, but 'category' enum in save_memory lacks description of what each category means.