Loading ToolBench
Maintain a device-local spec from coding-session intent, then generate focused unit tests.
Rudder Plugin has 9 tools with mixed quality. All tools have descriptions (10-200 chars, mostly adequate), and input schemas are present with type definitions. However, naming lacks consistency, tools use hyphens (validate-migrations, backup-tests) rather than verb_noun convention. Several tools have vague descriptions ('Gather coding session context') that don't explain WHEN to use them or what they return. Output schemas are not documented anywhere in the source. Error handling is absent, no guidance on retryability, user-fixable vs fatal errors, or recovery paths. Parameter descriptions are present but sparse (e.g., 'Run identifier' lacks context on format or generation). No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels (DESTRUCTIVE for manage-data-delete, WRITE for backup-tests). Composition is reasonable, tools are single-purpose, but chaining is unclear (e.g., context tool returns 'stored prompts' but manage-data-delete consumes them; no explicit field mapping documented).
Create a backup of test files before reset, capturing tracked and untracked changes.
Gather coding session context including changed paths, test metrics, and stored prompts from the local database.
Create a new local Rudder spec for the current branch from a repository source or input file.
Replace the source of an existing local Rudder spec by syncing from the repository source.
Delete all stored prompts from the Rudder database with secure deletion.
Get the status of the Rudder database including prompt count.
Record completion metrics for a Rudder run including test status, coverage, and changes.
Record that a question was asked during a Rudder run.
Tool names use hyphens (validate-migrations, backup-tests) instead of verb_noun convention (validate_migrations, backup_tests). LLMs expect snake_case for parsing intent.
No output schemas documented. Tools return results but LLMs cannot plan downstream calls without knowing field names and types. E.g., context tool returns 'stored prompts' but structure is undocumented.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. manage-data-delete is DESTRUCTIVE but unmarked; backup-tests is WRITE but unmarked. Agents cannot reason about side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Validate committed Drizzle migrations against disposable SQLite databases.
Parameter descriptions are minimal. 'Run identifier' (runId) lacks format, generation method, or uniqueness guarantees. 'Phase of context gathering' (phase enum) lacks explanation of when to use 'start' vs 'refresh'.
No error handling guidance. Tools lack recovery hints (e.g., 'If migration fails, check drizzle folder exists'). No categorization of errors as retryable vs user-fixable vs fatal.