Loading ToolBench
Persistent memory loop with GOAP research subagent for Hermes Agent. Implements retrieve→validate→apply→write-back loop with semantic retrieval from Qdrant, scope checking, confidence scoring, and orchestrated gating between research staging and operational memory.
This server has 16 tools with schemas present but significant quality gaps. Most tools have descriptions (good), but many descriptions are vague or lack actionable detail. Parameter descriptions are sparse or missing entirely. No error handling guidance. Tools like 'dispatch' and 'promote' combine multiple concerns. Output schemas are not documented. The codebase shows domain complexity (GOAP, memory management, orchestration) but the tool interface does not expose it clearly to LLMs. Naming is mostly verb-noun but some tools are ambiguous (e.g., 'verify' vs 'promote', what's the difference?). No tool annotations (readOnlyHint, destructiveHint). Critical issue: many WRITE tools lack confirmation or dry-run patterns.
Step 6-7: Apply candidate and write back result. Applies a candidate memory fix, verifies the result, and writes back the outcome to Qdrant with appropriate status transition (confirmed/observed/failed).
Build a valid memory payload dict. Constructs a properly formatted payload for Qdrant upsert with all required fields and enum values.
Main entry for GOAP Research Subagent: dispatch a research task. Runs the research loop (research problem, synthesize solution, test if possible) and returns a candidate memory staged for gate review.
Summary of gate decisions. Returns counts of promote/reject/hold decisions made by the orchestration gate.
Get all staged candidates awaiting gate. Returns list of CandidateMemory objects from the GOAP staging queue.
Human explicitly approves a staged candidate. Removes candidate from staging queue and promotes to main collection with human_approved annotation.
No output schemas documented. Tools return complex objects (MemoryRecord, CandidateMemory, GateDecision) but LLMs cannot see the structure. This forces agents to guess field names and types, increasing errors and wasting tokens on clarification.
Ambiguous tool names and overlapping responsibilities. 'verify' (gate decision) vs 'promote' (write to collection), what's the difference? 'apply_and_verify' combines two concerns. 'dispatch' runs research AND stages candidate. LLMs will struggle to choose the right tool.
No error handling guidance. Tools like 'promote' and 'apply_and_verify' are destructive (write to Qdrant) but descriptions do not explain failure modes, recovery steps, or when to retry. No actionable error messages documented.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Remove and return a staged candidate by task_id. Removes a candidate from the staging queue and returns it, or None if not found.
Process all staged candidates through the gate. Iterates through GOAP staging queue, verifies each candidate, and promotes/rejects/holds based on gate decisions.
Promote candidate to main Qdrant collection. Serializes writes with lock, converts staged candidate to MemoryRecord with appropriate status, and removes from staging queue.
Step 2-3: Retrieve and rank candidates by score. Performs semantic retrieval from Qdrant collection using problem embedding, filters out failed/superseded memories, and ranks by similarity * status_weight * freshness_decay.
Scheduled sweep: re-check aging confirmed memories. Finds confirmed memories older than max_age_days, creates sweep tasks for GOAP research subagent to re-verify them.
Step 4: Scope check against current environment. Validates if candidate memory's scope (tool versions, configs, OS) matches the current environment.
Step 5: State the candidate out loud. Formats a candidate memory record with its metadata and scope for presentation to the user.
Find aging confirmed memories and create sweep tasks. Queries Qdrant for confirmed memories older than max_age_days and returns list of ResearchTask objects for GOAP to re-verify.
Validate a memory payload against the Qdrant schema. Checks required fields, enum values, data types, ISO date formats, and returns validation result with errors and warnings.
Orchestration Gate: Verify candidate against current live problem. Returns gate decision (promote/reject/hold) based on confidence, testing status, and optional Hermes test result.
Missing parameter descriptions. 'env' parameter appears in 8+ tools but is never explained. What fields does EnvFacts contain? What happens if a field is missing? LLMs cannot construct valid payloads without this detail.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Agents cannot distinguish safe reads from destructive writes. 'promote', 'apply_and_verify', 'human_approve' are destructive but unmarked. 'retrieve_and_rank', 'scope_check' are read-only but unmarked.