Loading ToolBench
Keyless, self-contained lookup for the most-referenced NIST cybersecurity / privacy CONTROLS: SP 800-53 Rev 5 (security & privacy controls, incl. control enhancements) and SP 800-171 Rev 3 (CUI security requirements). The full catalog is embedded at build time from official NIST OSCAL content — no network access at runtime.
Three well-named, read-only lookup tools with complete input schemas and strong descriptions (194 - 392 chars, well above baseline). All parameters are typed and described. Tool names follow verb_noun pattern (nist_control, nist_control_search, nist_control_family). Schemas are properly structured with required fields. However, output schemas are not documented, LLMs cannot predict response structure. Error handling is minimal (nist_control returns a 'found' flag but lacks recovery guidance). No tool annotations (readOnlyHint, idempotentHint) despite all being read-only. Parameter descriptions could be more prescriptive about valid formats (e.g., 'AC-2', 'ac-2', 'AC-02' are all accepted but this flexibility is not formally constrained).
Look up the full text of one NIST 800-53 security control or NIST 800-171 CUI security requirement by id — the control statement (requirement prose), discussion/guidance, and related controls. Use for "what does NIST control AC-2 require", "NIST 800-53 AC-2 access control", "FedRAMP control SC-7 boundary protection", "NIST authenticator management IA-5", "NIST CUI control 800-171 3.1.1". Forgiving id: accepts "AC-2", "ac-2", "AC-02", control enhancements "AC-2(1)" or "AC-2.1", and CUI ids "3.1.1" / "3.01.01". Source: SP 800-53 Rev 5 and SP 800-171 Rev 3 (official NIST OSCAL, public domain). Examples: {"id":"AC-2"}, {"id":"SC-7"}, {"id":"IA-5"}, {"id":"AC-2(1)"}, {"id":"3.1.1"}.
List every NIST control in a family — the FedRAMP/800-53 control-family baseline. Use for "list all NIST access control controls", "NIST 800-53 AC family", "what controls are in identification and authentication", "NIST system and communications protection family". Input a family code ("AC", "IA", "SC", "AU", "SI", …) or a family name ("access control"). Returns the family title plus every control id + title (including enhancements). Source: SP 800-53 Rev 5 (+ SP 800-171 Rev 3). Examples: {"family":"AC"}, {"family":"identification and authentication"}, {"family":"SC"}.
Keyword search across NIST 800-53 security/privacy controls and NIST 800-171 CUI requirements by title and requirement text. Use when the id is unknown — "NIST security control for least privilege", "NIST control about session lock", "NIST 800-53 encryption at rest", "NIST cybersecurity control multi-factor authentication", "access control / identification and authentication / system and communications protection". All query tokens must match (case-insensitive AND). Optional family filter (code like "AC", "IA", "SC" or a family name like "access control"). Returns matching {id, family, title, snippet}. Source: SP 800-53 Rev 5 + SP 800-171 Rev 3. Examples: {"query":"least privilege"}, {"query":"multi-factor authentication","family":"IA"}, {"query":"boundary protection"}.
Output schemas not documented. LLMs cannot predict response structure (e.g., nist_control returns {found, id, family, family_title, title, statement, guidance, related, source} but this is not declared in the tool definition).
No tool annotations despite all tools being read-only and idempotent. Missing readOnlyHint and idempotentHint metadata that would help agents reason about safety and retry behavior.
Error handling lacks recovery guidance. nist_control returns {found: false, suggestions: []} but does not tell the LLM what to do next (e.g., 'Try nist_control_search() with keywords').
Parameter 'limit' in nist_control_search lacks min/max constraints in schema. Code enforces 1 - 50 but schema does not declare this, forcing LLMs to guess valid range.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
nist_control_search description mentions 'All query tokens must match (case-insensitive AND)' but does not explain what happens if no results match or how to refine the query.