Loading ToolBench
A semantic memory and knowledge management system for project wikis with vector-based search, incremental indexing, and session-aware context restoration.
Server has 8 tools with basic verb-noun naming (memory_*). All tools have descriptions (10-100 chars, mostly adequate). Input schemas are present with type definitions and parameter descriptions. However, output schemas are NOT documented, tools return TUI-rendered strings rather than structured JSON, making it impossible for LLMs to extract data for chaining. No error classification, no recovery guidance, no pagination support. Tool descriptions lack WHEN/WHY context. Parameters lack constraints (enums, ranges). No tool annotations (readOnlyHint/destructiveHint). Composition is reasonable (single responsibility per tool), but output structure defeats downstream tool chaining.
Delete a memory file from storage and index.
Discover wiki files and naming convention for a project.
Read the current project's memory inbox content.
Append a learning block to the project's memory inbox.
List indexed memory files.
Extract, link, and structure concepts from the project's inbox.
Semantic & Hybrid search across indexed wikis.
Output schemas not documented. All tools return TUI-rendered strings (via TUIRenderer.render_panel) instead of structured JSON. LLMs cannot extract data for downstream chaining (e.g., memory_search returns formatted text, not a list of {rel_path, score, title, excerpt} objects). This violates pattern:response-shaper and breaks tool composition.
No tool annotations. memory_delete is destructive but lacks destructiveHint. memory_search, memory_list, memory_inbox_read are read-only but lack readOnlyHint. memory_write and memory_organize modify state but lack idempotentHint or destructiveHint. Agents cannot reason about side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
Write concept file directly and compute relations.
No error classification or recovery guidance. Errors return TUI panels with raw exception messages (e.g., 'Error: {e}'). LLMs cannot determine if errors are retryable, user-fixable, or fatal. No suggestions for next steps (e.g., 'Try search_users() first').
Parameter constraints missing. memory_search accepts k (integer) with no min/max bounds, LLM could pass k=999999. memory_write accepts filename with no validation, could cause path traversal. No enums for variant selection. Descriptions lack format/range specifications.
No pagination support. memory_list and memory_search return unbounded results. memory_list iterates all registry entries; memory_search returns k results but no total count or next_cursor. Large projects could exhaust context window.