Loading ToolBench
Subcog demonstrates solid definition quality with comprehensive tool coverage (14 tools), explicit input schemas for all tools, and clear descriptions. All tools follow verb_noun naming conventions (capture, recall, consolidate, etc.). Most tools have descriptions in the 70-200 character range, suitable for LLM context. However, there are gaps: output schemas are not explicitly documented in the visible source code, error handling guidance is minimal, and some parameter descriptions lack format/constraint details. The tool set exhibits good composition (single responsibility per tool, chaining-friendly IDs), but parameter validation rules and recovery guidance are underdocumented.
Captures a memory with content, namespace, tags, source, TTL, and domain scope
Consolidates related memories by grouping similar items based on namespace, time window, similarity threshold, and minimum group size
Soft/hard delete capability matching industry patterns. Defaults to soft delete (tombstone) for safety, with explicit hard delete option for permanent removal
Bulk delete memories matching filter criteria. Implements Mem0's delete_all() pattern with dry-run safety
Enriches a memory by generating/improving tags, restructuring content for clarity, and adding inferred context and rationale
Provides CRUD and advanced operations for entities in the knowledge graph. Actions: create, get, list, delete, extract, merge
Direct memory retrieval by ID - a fundamental CRUD operation
Output schemas not explicitly documented. While input schemas are comprehensive, return types and structures are not visible in the tool definitions. LLMs cannot infer what fields to expect in responses, making it hard to chain tools or extract specific data.
Error handling and recovery guidance absent. Tool descriptions do not explain what to do if a call fails. For destructive operations (delete, delete_all, consolidate), there is no confirmation/dry-run explanation in the description itself, only hinted at in parameter docs. LLMs need explicit recovery paths.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 60 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Retrieves a summary memory by ID
Retrieves change history for a memory by querying the event log. Provides audit trail visibility without storing full version snapshots
Lists all memories with optional filtering and pagination. Matches Mem0's get_all() and Zep's list_memories() patterns
Searches for memories by query, filter, namespace, mode, detail level, limit, entity, user_id, and agent_id. When query is omitted or empty, behaves like list and returns all memories matching filter criteria
Reindexes the memory store, optionally scoped to a specific git repository
Restores a tombstoned (soft-deleted) memory. Implements the inverse of soft delete for data recovery
Allows updating content and/or tags of an existing memory. Follows the industry pattern of partial updates
Parameter constraint documentation incomplete. Many parameters lack format/range specifications in descriptions. E.g., 'ttl' accepts duration strings like '7d', '30d', '24h', '60m', or seconds, but the description doesn't explain the format clearly enough for LLMs to generate valid values reliably. 'similarity' is 0.0-1.0 (good), but 'days' and 'limit' lack min/max guidance.
'entities' tool exhibits low cohesion with over-loaded action parameter. The tool combines CRUD (create, get, list, delete) and advanced ops (extract, merge) under a single interface. This forces LLMs to reason about which action to invoke and requires complex conditional parameter validation. Should split into separate tools (create_entity, get_entity, etc.) or at least separate read-only actions from write actions.
Parameter relationships undocumented. 'entities' tool has conditional parameters (e.g., 'store' only applies to 'extract' and 'merge' actions; 'entity_id' required for 'get'/'delete' but not 'create'). These interdependencies are not stated in parameter descriptions, forcing LLMs to infer intent.
Multi-tenant filtering parameters (user_id, agent_id) appear in multiple tools (recall, list, delete_all) but are not consistently described across tools. Some mention 'flexible scoping via metadata', others are terse. Consistent language would improve clarity.