Loading ToolBench
Offline pentest documentation aggregator + unified search with an MCP server to attach an AI model. Pulls curated security knowledge bases (HackTricks, PayloadsAllTheThings, OWASP guides, LOTL databases), indexes markdown/yaml into a full-text search index, and serves a web UI and MCP interface.
Grimoire has 11 well-named tools with clear, domain-specific descriptions (avg 156 chars). All tools have explicit input schemas with typed parameters and descriptions. However, output schemas are not documented in the visible code, responses are described narratively but not formally specified. Error handling is minimal; tools lack recovery guidance and actionable error messages. Parameter descriptions are adequate but lack format constraints (enums, ranges, patterns). No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. The grimoire_run tool is particularly concerning: it accepts arbitrary shell commands with only narrative safety checks ('refuses destructive commands'), no formal validation, and no dry-run capability for an IRREVERSIBLE operation.
List the available source categories and their sources.
Gather grounded source material for a technical checklist on a topic: runs several targeted searches and returns the matching documentation snippets with citations. Feed this to build a concrete, source-backed checklist.
Return the engagement context for THIS assessment: operator-declared targets/IPs, interfaces, hardware, SIM/telecom and RF parameters, plus host-detected interfaces, USB devices and SDRs. Read this first and tailor every step and command to the actual targets and hardware in scope.
Detect the runtime environment: whether this is an RF-Swift container, the OS and package manager, whether RF-Swift install scripts are reachable, and root/sudo availability. Use this to decide how to install tools.
Return the full text of one indexed document (markdown, notebook converted to markdown, or extracted PDF text). Use the source+path from a grimoire_search result to read the whole page before answering.
grimoire_run lacks formal input validation and dry-run capability for an IRREVERSIBLE operation. Description says 'refuses destructive commands' but provides no enum of allowed commands, no regex pattern, no validation logic visible in schema. Agents cannot safely plan destructive operations.
No output schemas documented for any tool. Responses are described narratively ('returns ranked matches with source, category, path and a snippet') but not formally specified as JSON Schema. LLMs cannot reliably extract fields or plan downstream tool calls.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2026-07-28+ | v2 |
Install a missing tool (RF-Swift recipe if available, else host package manager). Requires --mode assist|auto.
Resolve HOW a missing tool would be installed (RF-Swift script recipe first, then the host package manager) and return the ordered commands - WITHOUT running anything.
Run a shell command for the engagement. Refuses destructive commands and (when a scope is set) commands aimed at out-of-scope hosts. Requires --mode assist|auto. Authorized engagements only.
Full-text search across every indexed offline security knowledge base (HackTricks, PayloadsAllTheThings, OWASP WSTG/MASVS/ASVS, the LOTL DBs, RE/OSINT/DFIR/Bluetooth/WiFi/SDR sources, ...). Returns ranked matches with source, category, path and a snippet. Use this to ground answers and checklists in real documentation.
Assemble a broad reading set for a topic: runs a wide sweep of related searches across the whole corpus and returns the deduped matching docs with citations. Use this to write a complete, source-backed tutorial. Pair with grimoire_fetch_doc to read the most relevant pages in full.
Check whether a tool/binary is installed and where.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels declared in metadata (READ_ONLY, WRITE, IRREVERSIBLE). Agents cannot infer safety properties from tool definitions alone.
Parameter descriptions lack format constraints. 'query' in grimoire_search has no length limits, pattern, or examples. 'timeout' in grimoire_run has no min/max bounds (schema shows default 300, max 1800 but description does not). Agents cannot validate inputs before calling.
No error handling guidance. Tools lack recovery hints (e.g., 'If tool not found, call grimoire_plan_install first'). Agents cannot self-correct on failures without explicit error messages.