Loading ToolBench
Human-in-the-loop approval, evidence management, forensic command execution for incident response investigations
Valhuntir CLI exposes 28 tools with significant definition gaps. Most tools lack proper input schemas, only 8 of 28 have visible schema definitions with typed parameters. Descriptions vary widely: some are detailed (cmd_approve, cmd_exec), others are minimal (cmd_evidence, cmd_export, cmd_merge). Parameter descriptions are inconsistent; many tools accept parameters without type constraints or validation guidance. No output schemas are documented. Error handling is implicit (no recovery guidance in descriptions). Tool naming follows verb_noun convention (cmd_* prefix), but several tools combine multiple concerns (cmd_review handles audit, findings, timeline, TODOs, IOCs in one tool). Security-sensitive tools (cmd_approve, cmd_exec, require_confirmation) lack explicit permission/scope declarations. Overall, the server reads as a CLI wrapper rather than an agent-optimized tool interface.
Approve staged findings and timeline events. Requires password confirmation via /dev/tty. Supports interactive review with options: approve, edit, note, reject, todo, skip, quit
Review case audit trail and approval records
Back up case data with optional evidence, extractions, and OpenSearch index
Configure Valhuntir settings including examiner identity and approval password
Launch Examiner Portal dashboard for case review and approval
Manage evidence files and integrity verification
Execute forensic command with audit trail and sandbox isolation. Requires --purpose flag explaining why command is being run
12 tools have no input schema (cmd_audit, cmd_dashboard, cmd_portal, cmd_evidence, cmd_lock_evidence, cmd_unlock_evidence, cmd_report, cmd_export, cmd_merge, cmd_update, and others).
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., what does cmd_review return? What fields are in findings?). Violates pattern:tool and pattern:response-shaper.
cmd_review combines 8 distinct concerns (audit, evidence, findings, timeline, TODOs, IOCs, filtering, verification) into one tool. Should split into separate tools: list_findings, list_timeline, list_todos, extract_iocs, verify_findings. Violates pattern:tool (single responsibility).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Export case data to external format
Join a Valhuntir gateway using join code for remote client setup
Set evidence directory to read-only using bind mount
Merge case data from multiple sources
Migrate case from examiners/ directory layout to flat layout
Alias for cmd_dashboard - launch Examiner Portal
Register evidence file with SHA-256 hash and optional description
Reject staged findings and timeline events with optional reason
Generate forensic investigation report from case findings and timeline
Restore a case from backup with optional OpenSearch and ledger skip
Review case status, audit trail, findings, timeline, TODOs, and IOCs with filtering and verification options
Manage Valhuntir gateway service (start, stop, status, logs)
Run interactive setup or connectivity test. Routes to subcommands: vhir setup client (Configure LLM client), vhir setup test (Test connectivity), vhir setup join-code (Generate join code)
Manage investigation TODOs: create, list, update, close, reopen
Unlock evidence directory for new files
Update Valhuntir CLI to latest version
Check if analyst has a password configured (checks new location /var/lib/vhir/passwords/, falls back to legacy config.yaml)
Require password confirmation for approval actions. Returns (mode, password) tuple. Blocks AI-via-Bash by reading from /dev/tty with no echo
Prompt y/N via /dev/tty for interactive confirmation. Returns True if confirmed
Set up a new approval password for analyst. Prompts twice to confirm. Returns raw password string for HMAC re-signing
Verify a password against stored PBKDF2 hash (new location, fallback old)
Security-sensitive tools (cmd_approve, cmd_exec, require_confirmation, setup_password) lack explicit permission/scope declarations. No audit trail guidance in descriptions. Violates pattern:scope-declaration and pattern:audit-trail.
cmd_exec accepts arbitrary command arrays with only a 'purpose' string. No validation, sanitization, or injection guards documented. Violates pattern:tool-gateway (untrusted input handling).