Loading ToolBench
Portable, governed memory and context for AI agents with audit, encryption, and MCP integration
OACS exposes 41 tools with basic schemas and descriptions, but quality is inconsistent. Most tools have 1-2 sentence descriptions (avg ~60 chars, well below the 194-char baseline for A+ tools). Parameter descriptions exist but are minimal. No output schemas documented. Error handling is absent from tool definitions. Tools like 'health', 'list_audit', 'verify_audit' have trivial descriptions. Complex tools like 'build', 'grant_capability', 'call_tool' lack detail on when to use them, what they return, or how they chain. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Naming is generally verb-noun compliant, but composition issues exist: 'grant' and 'grant_capability' are near-duplicates; 'observe', 'propose', 'commit' form a multi-step workflow that should be documented as a pattern.
Blur (reduce precision of) a memory record
Build a context capsule from memory, rules, tools, and skills
Call a tool with payload
Commit a proposed memory
Compare benchmark runs
Correct a memory record
Create an actor (human or agent)
Explain a context capsule
Descriptions are uniformly short (avg ~40 chars vs 194-char baseline). Most tools lack context on WHEN to use them, WHAT they return, or HOW they chain. E.g., 'Verify the audit chain integrity' tells an LLM nothing about prerequisites, output format, or recovery steps.
No output schemas documented. Tools return structured data but LLMs cannot plan downstream calls without knowing response fields. E.g., 'get_context' returns a capsule but schema does not specify fields like capsule_id, actor_id, memory_refs, or token_count.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Export a context capsule
Forget (delete) a memory record
Generate synthetic benchmark tasks
Get a specific audit event by ID
Retrieve a context capsule by ID
Grant capability for a context capsule
Grant a capability to an actor
Health check endpoint returning server status
Import a context capsule
Import benchmark task pack from file
Ingest a tool execution result as evidence
Inspect a capability definition
Inspect a rule by ID or name
Inspect a skill by ID
Inspect a tool by ID
List all actors
List all audit events
List all capability definitions
List all rules
List all skills
List all tools
Lock a context capsule
Observe and record a memory from external input
Propose a memory for later commitment
Query memory records
Read a specific memory record
Revoke capability for a context capsule
Check rules for an operation
Run a benchmark test against memory system
Run a skill with payload
Sharpen (increase precision of) a memory record with evidence
Validate a context payload
Verify the audit chain integrity
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. 'forget' is marked DESTRUCTIVE but has no annotation. 'observe' and 'propose' are non-idempotent but unmarked. LLMs cannot reason about retry safety or side effects.
Duplicate/overlapping tools: 'grant' and 'grant_capability' both grant capabilities but with different parameter sets. 'list_*' and 'inspect_*' pairs (e.g., list_skills + inspect_skill) lack pagination guidance. LLMs waste reasoning cycles choosing between near-duplicates.
Multi-step workflows (observe → propose → commit) are not documented as a pattern. No guidance on when to use each step, what state transitions are valid, or error recovery. LLMs may call them out of order or skip steps.