Loading ToolBench
Autonomous code evolution engine — observe, plan, execute, verify
This server has 6 tools with basic structure but significant gaps in parameter descriptions and error handling. Tool names follow verb_noun conventions (rag_get_settings, rag_set_settings, rag_diff_files, rag_upsert_chunks, rag_delete_file, rag_search), which is good. However, parameter descriptions are completely absent from all input schemas, the JSON schemas show only types and required fields, with no description text. Tool descriptions are present but generic (10-80 chars), lacking actionable context about when to use each tool or what errors might occur. The rag_upsert_chunks schema includes nested objects with limited type specificity. Error handling is minimal, no recovery guidance, no error classification, no validation examples. The server reads and writes files, manages a knowledge graph, and performs destructive deletes, but lacks permission checks, audit trails, and idempotent operation guarantees.
Remove a file's chunks, entities-mentions, and manifest entry.
Walk configured sources and return added/changed/removed files vs. manifest.
Return the current `.noory/rag/settings.json` contents.
Search the knowledge graph via vector embedding + entity expansion.
Write `.noory/rag/settings.json` with the given object after schema validation.
Upsert pre-chunked text + extracted entities/relations for one file.
All input parameters lack descriptions. JSON schemas show only type/required fields with no description text. LLMs cannot infer what 'source_path', 'chunks', 'entities', 'relations' mean or how to structure them. Violates pattern:tool-description (100% of A+ tools have param descriptions).
Tool descriptions are too generic and lack actionable context. 'Return the current .noory/rag/settings.json contents' (50 chars) and 'Write .noory/rag/settings.json with the given object after schema validation' (76 chars) do not explain WHEN to use these tools vs others, what happens on validation failure, or what dependencies exist. Baselines show A+ tools average 194 chars with clear WHAT/WHEN/WHY guidance.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
rag_upsert_chunks accepts 'chunks' as array of objects with only 'text' required and optional 'meta' object. No schema for meta structure, no description of allowed chunk size, no guidance on entity/relation reference format. This invites malformed input and hallucinated chunk structures.
rag_delete_file is destructive (removes file chunks, entities, manifest entry) but lacks confirmation/dry-run pattern. No error handling guidance on what happens if file does not exist or is in use. Violates pattern:confirmation-request and pattern:recovery-guide.
No permission checks or audit trails visible. Tools write to .noory/rag/settings.json and delete manifest entries without verifying caller identity or logging who made changes. Violates pattern:permission-gate and pattern:audit-trail.
rag_search accepts 'k' (1-64) and 'expand_depth' (0-4) with numeric bounds but NO descriptions. LLM cannot reason about what k controls (result limit? embedding count?) or what expand_depth does without guessing. Violates pattern:tool-description.
No documented output schemas. Tools return responses (rag_search returns 'results', rag_get_settings returns 'settings', etc.) but schema code does not show what fields/structure LLMs should expect. Pattern:tool requires output schema documentation so agents can extract needed data and plan chained calls.
rag_diff_files requires 'source_path' (string) but no description specifies expected format, absolute vs relative path, or what happens if path is invalid. Violates pattern:tool-description (descriptions must state format, range, allowed values).