Loading ToolBench
A learning and memory consolidation system for Claude sessions. Captures correction cues from conversations, scans for stale project state, and ingests/queries documents via Pinecone vector embeddings.
Stratos Memory Stack exhibits significant definition quality gaps. Tool names lack action verbs (capture-session, staleness-scan, pinecone_ingest, pinecone_query are inconsistent in style). Descriptions exist but are verbose and domain-specific rather than LLM-optimized. Input schemas are present but lack proper JSON Schema typing for all parameters. The pinecone_ingest and pinecone_query tools have minimal parameter descriptions. No output schemas are documented. Error handling is absent, tools fail silently or with generic messages. Security concerns: API keys exposed in .env files without server-side injection pattern. No permission gates or audit trails.
SessionEnd hook that performs no-ask capture of correction-cue triggers. Scans the session transcript for spoken rules (from now on, next time, remember to, etc.) and appends them to a capture buffer as trigger-candidate lines.
Ingests markdown documents into Pinecone vector database. Chunks text, generates embeddings using OpenAI, and upserts vectors with metadata to the configured Pinecone index.
Queries the Pinecone vector database for documents matching a question or topic. Returns top-k matching chunks with source path and similarity score.
Deterministic decay helper for the Consolidation engine. Scans project STATE.md frontmatter and flags entries that are stale (last_touched older than STALE_DAYS) or carry a paused/archived status. Read-only. Prints a markdown flag list.
Tool names lack consistent action verbs. 'capture-session' and 'staleness-scan' use hyphens and are not verb-noun pairs. LLMs cannot infer intent from names alone.
Input parameter descriptions are missing or minimal. pinecone_ingest 'path' param has description but no type constraints (file vs folder ambiguity). pinecone_query 'top_k' lacks range bounds (1-100?).
No output schemas documented. Tools return unstructured text or implicit JSON. LLMs cannot plan downstream calls or extract structured data reliably.
API keys (PINECONE_API_KEY, OPENAI_API_KEY) exposed in .env files and loaded in tool code. Credentials must use server-side secret injection, not tool parameters or environment variables visible in traces.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 33 | 2026-07-28+ | v2 |
No error handling or recovery guidance. Tools fail silently (e.g., 'exit 0' in bash scripts) or with generic exceptions. LLMs receive no actionable error messages to self-correct.