Loading ToolBench
amp-rs has 13 tools with complete input schemas and descriptions, but quality is inconsistent. Naming follows verb_noun convention well (add_lesson, search_lessons, delete_lesson). However, 4 tools (search_code, index_repo, diff_index, full_reindex) are stubbed with 'not yet fully integrated' errors, making them non-functional. Descriptions are present but brief (avg ~80 chars, below the 194-char baseline). Parameters lack enums for constrained values (e.g., severity accepts free-form strings instead of enum Info|Warning|Error|Critical). Output schemas are not documented, callers cannot predict response structure. Error handling is minimal: most errors are generic strings without recovery guidance. No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. Composition is reasonable (single responsibility per tool), but missing pagination support on list_lessons and search_* tools.
Add a new checkpoint for an agent with working_on description and optional state data
Add a new lesson with title, content, optional tags, and severity level
Delete a lesson by id
Perform differential indexing on a repository, only re-indexing changed files (not yet fully integrated)
Perform full re-indexing of a repository, clearing all indexed data and re-indexing from scratch (not yet fully integrated)
Get aggregated status for an agent based on recent checkpoints
Get recent checkpoints for a specific agent
Four tools (search_code, index_repo, diff_index, full_reindex) are non-functional stubs returning 'not yet fully integrated' errors. These should be removed or completed before production use.
Severity parameter accepts free-form strings instead of enum constraint. Description says 'Optional severity level (Info, Warning, Error, Critical)' but no JSON Schema enum enforces this. LLMs will hallucinate invalid values like 'debug' or 'fatal'.
No output schemas documented. Callers cannot predict response structure. E.g., search_lessons returns array of {lesson, score} but this is inferred from code, not declared in tool definition.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Get overall server status including lesson count, checkpoint count, chunk count, indexed repos, and uptime
Index a repository at the given path (not yet fully integrated)
List all lessons, optionally filtered by severity level
Search checkpoints by semantic similarity to a query, optionally filtered by agent
Search code chunks by semantic similarity (not yet fully integrated)
Search lessons by query with semantic similarity, returns results with relevance scores
No pagination support on list_lessons or search_* tools. Default limits (5-50) are hardcoded; no offset/cursor mechanism. Large result sets will blow context windows.
Error messages are generic strings without recovery guidance. E.g., 'Failed to add lesson: <error>' tells LLM nothing about whether to retry, ask user, or give up. No error classification (retryable vs fatal).