Loading ToolBench
Demonstrates structured logging of Agentic calls to MCP servers. Provides phishing email analysis through IOC extraction and threat intelligence reputation checking.
PhishingTriage has three security-focused tools with decent naming conventions but significant gaps in parameter descriptions and output schemas. Tool names follow action-verb patterns (extract_, get_, get_), and descriptions are present but minimal. However, parameter descriptions are sparse, output schemas are not formally documented, and error handling lacks recovery guidance. The server demonstrates basic competence but falls short of production-grade quality.
Extracts Indicators of Compromise (IOCs) such as URLs, domains, IP addresses, and file hashes from the body of an email.
Checks the reputation of a given MD5 file hash against a threat intelligence source.
Checks the reputation of a given URL against a threat intelligence source.
Output schemas are not documented. The tools return dict objects (e.g., {"success": True, "iocs": iocs}) but there is no formal JSON Schema or field documentation describing what the agent should expect. LLMs cannot reliably parse or chain unspecified output.
Parameter descriptions are minimal or missing. 'email_body' has a description, but 'url' and 'file_hash' descriptions are one-liner placeholders that don't explain format, length, or validation rules. LLMs cannot validate input constraints.
Error handling lacks recovery guidance. Exceptions are caught and re-raised (e.g., 'Failed to get URL reputation for {url}: {str(e)}') but do not instruct the LLM on next steps: is the error retryable? Should the agent ask the user? Should it try a different tool?
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 55 | <=2025-11-25 | v2 |
| 2026-03-09 | C | 63 | - | v1 |
extract_iocs_from_email uses simple regex that may miss or misidentify IOCs. The tool comment states 'Production use requires more robust parsing,' indicating the implementation is incomplete. Additionally, the tool does not document what IOC types it supports or their format constraints.
No input validation constraints are documented (e.g., URL format, MD5 hash length). LLMs may pass invalid values; the tools will silently fail or return incorrect results instead of guiding self-correction.
Tool descriptions are under 100 characters and generic. They state WHAT the tool does but not WHEN to use it or how it fits into the phishing triage workflow. Agents relying on tool descriptions to prioritize calls will struggle.