Loading ToolBench
A memory management and cognitive RAG system with bitemporal time-travel queries, hybrid search, and memory branching capabilities
Piyapi exposes 7 tools with basic schemas and descriptions, but critical gaps prevent production use. All tools have descriptions (10-80 chars, mostly adequate) and input schemas with type definitions. However, NO output schemas are documented, LLMs cannot predict what these tools return, breaking downstream composition. Tool names lack clear verb-noun structure (e.g., 'graph.timeTravel' uses camelCase; 'branches.create' is nested). Parameters lack constraints: 'alpha' (0-1 weighting) has no min/max; 'temperature' (0-1) has no bounds; 'limit' has no range. No error handling guidance, no idempotency hints, no permission scopes. The 'ask' tool returns 'response' and 'citations' but this is inferred from example code, not documented in the schema. Branching operations (create, merge, diff) lack conflict resolution or rollback guidance. Overall: functional definitions but missing ~40% of production-grade patterns.
Perform cognitive RAG (Retrieval-Augmented Generation) to answer queries based on stored memories
Create a new memory branch based on an existing branch
Compare differences between two memory branches
Merge one memory branch into another
Query memories as they existed at a specific point in time using bitemporal time-travel
Store a memory with content and optional metadata
Perform hybrid search across stored memories
No output schemas documented. LLMs cannot predict return types, breaking tool chaining and composition. E.g., 'ask' returns {response, citations} but this is inferred from examples, not in schema.
Numeric parameters lack bounds. 'alpha' (0-1), 'temperature' (0-1), 'limit' have no min/max constraints. LLMs may pass invalid values (e.g., limit=-1, alpha=2.5).
Tool names use inconsistent patterns. 'graph.timeTravel' and 'branches.create' use dot notation and camelCase; 'memories.create' and 'search' use flat names. Inconsistency confuses LLM tool selection.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
No error handling guidance. Tools provide no recovery hints (e.g., 'If merge fails due to conflict, call branches.diff() first'). Agents cannot self-correct on failures.
No idempotency or destructiveness hints. 'branches.merge' is destructive but unmarked. 'search' is read-only but not annotated. LLMs cannot reason about retry safety.