Loading ToolBench
A multi-agent security investigation platform with three specialized agents (Fusion, Log Analysis, Threat Intel) that query external security systems with per-request authorization checks before tool dispatch, preventing unauthorized data access at the orchestration layer.
Three tools with complete input schemas and descriptions, but significant gaps in output documentation, error handling, and composition patterns. All tools are READ_ONLY and properly scoped, but descriptions lack actionable guidance for LLM selection. No parameter validation rules, enum constraints, or recovery guidance. Output schemas are inferred from code comments rather than formally documented. Tool composition is weak, no clear chaining IDs or pagination support documented.
Submit a specific piece of log text to Project 1's real classifier for analysis. Routed through the same authorization framework as every other tool in this system.
Query fused threat intelligence data for an indicator, with per-field authorization checked BEFORE any Accumulo access is attempted — a denied field is never queried, not just filtered from the response.
Retrieve threat intelligence documents relevant to the query, with per-document authorization checked before each document is included in the result.
Output schemas not formally documented. Code comments describe return structure (FusionQueryResult, LogClassificationResult, ThreatIntelResult) but no JSON Schema provided to LLM. LLMs cannot plan downstream calls without knowing response field names and types.
No error handling guidance. Tools return authorization denials (denied_fields, denial_reason) but no recovery instructions. LLM cannot determine if denial is retryable, user-fixable, or fatal. Missing pattern: 'If denied, try with lower classification requirement' or 'Contact security team.'
Parameter descriptions lack validation rules. 'requested_families' accepts array of strings but no enum constraint or valid values documented. 'query' parameter has no length limit or format guidance. LLMs will hallucinate invalid family names or oversized queries.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
session_id parameter appears in all three tools but no documentation of what constitutes a valid session_id, how to obtain one, or what happens if invalid. LLM cannot self-correct if session_id is malformed.
No pagination or result limits documented. query_fusion_data and query_threat_intel could return large result sets (multiple data families, many documents) but no limit, offset, or cursor parameters. Large responses will exhaust context window.