Loading ToolBench
Local-first, zero-cost memory system with four-layer hierarchy and multi-path recall for AI agents
FlyupMem defines 12 tools with consistent Zod schemas and descriptions. All tools have non-empty descriptions (range 60 - 180 chars, within baseline 34 - 392). All parameters are typed and described. However, output schemas are not documented, responses are JSON-stringified text blobs without declared structure. Error handling is minimal (e.g., flyup_forget returns plain text on not-found, no recovery guidance). Tool names follow verb_noun convention (flyup_learn, flyup_recall, flyup_feedback) but lack clarity on when to use similar tools (e.g., flyup_recall vs flyup_experiences both retrieve memories). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (WRITE, READ_ONLY, REVERSIBLE). Composition is sound, tools are single-purpose and chain-able, but parameter naming could be more explicit (e.g., 'memory_id' is good; 'query' in flyup_recall could specify 'semantic_query' to distinguish from full-text search).
Get detailed Experience by ID with resolved evidence chain (source memories).
Browse Experience layer memories. Returns list of Experiences with pattern metadata.
Give feedback on a memory (positive/negative/neutral). Negative feedback can trigger auto-retirement.
Retire a memory (soft delete). The memory is marked as retired but not removed.
Inspect a single memory by ID: full detail, current activation, related memories, graph edges, and feedback.
Learn a fact from conversation. Extracts knowledge, deduplicates, and stores as an engram.
Run maintenance: batch decay, consolidation, graph updates, and optional REM processing.
Output schemas not documented. Tools return JSON-stringified text without declared field structure. LLMs cannot plan downstream calls or extract specific fields reliably.
Error handling lacks recovery guidance. flyup_forget returns 'Memory not found' with no suggestion to search or list available memories. Errors do not categorize as retryable/user-fixable/fatal.
No tool annotations despite explicit risk levels. Tools marked WRITE (flyup_learn, flyup_maintain, flyup_pack) and REVERSIBLE (flyup_forget) lack destructiveHint/idempotentHint annotations. LLMs cannot infer safety properties.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Export or import a Knowledge Pack (YAML/JSON backup of all memories).
Search memories relevant to a query. Returns formatted context for injection into LLM prompts.
Synthesize Mental Models from Observations using LLM (requires FLYUP_LLM_API_KEY).
Get memory store statistics and health check.
Get recent episodes (conversation history snapshots).
Ambiguous tool selection. flyup_recall, flyup_experiences, and flyup_inspect all retrieve memories but differ in scope/format. Descriptions do not clarify when to use each (e.g., 'Use flyup_recall for semantic search; flyup_experiences for pattern browsing').
Parameter 'query' in flyup_recall lacks specificity. Does it support semantic search, keyword matching, or both? Description should clarify search mode and expected format.