Loading ToolBench
Organizational memory for AI agents - governed read, write, and recall.
HaloMem demonstrates solid tool design with complete schemas, clear descriptions, and proper parameter typing across all 8 tools. Naming follows verb_noun conventions (company_file_list, company_memory_search, etc.). All tools have descriptions (avg ~100 chars) and input schemas with typed parameters. However, output schemas are not documented in the visible code, and error handling lacks recovery guidance. Tool composition is clean (single responsibility per tool), but descriptions could be more LLM-optimized with explicit WHEN/WHY context. No tool annotations (readOnlyHint/destructiveHint) despite clear risk stratification in metadata.
List governed files in the organization's knowledge base with optional filtering by namespace, tag, or search term.
Read the full content of a governed file by ID with optional byte range support.
Revoke or delete a memory entry from the organization's knowledge base.
Retrieve live memory results with real-time ranking and federation across memory sources.
Retrieve the organization's memory index or full manifest with optional filtering by namespace or variant.
Pin or unpin a memory entry to control its visibility and priority in search results.
Output schemas not documented in visible code. Tools return ToolResponse with generic 'content' field; LLMs cannot infer what fields to expect from results (e.g., does company_memory_search return ranked_results with score/id/title, or a flat list?). This forces agents to guess downstream field names.
Tool annotations missing. Metadata identifies risk levels (READ_ONLY, WRITE, DESTRUCTIVE) but tools lack readOnlyHint/destructiveHint/idempotentHint annotations in schema. LLMs cannot determine which tools are safe to retry or have side effects without explicit hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Search organizational memory with full-text query, optional filters, and ranked results.
Write or update a memory entry in the organization's knowledge base.
Error handling lacks recovery guidance. Code shows fail() returns generic JSON error; no actionable next steps. E.g., if company_memory_search returns no results, should agent retry with different query, call company_memory_manifest first, or escalate? Pattern requires 'Try X instead' guidance.
Descriptions lack WHEN/WHY context. E.g., 'Search organizational memory with full-text query' does not explain when to use company_memory_search vs company_memory_live vs company_file_list. LLMs need explicit disambiguation to pick the right tool.
Destructive operations lack confirmation step. company_memory_forget deletes entries; no dry-run or confirmation_required pattern. Agents can irreversibly delete organizational memory without a safety gate.