Loading ToolBench
This server exhibits severe gaps across all quality dimensions. Tool naming follows a 'do-*' prefix pattern which is non-standard and fails to convey intent clearly (do-ffuf, do-masscan, do-nmap, do-sqlmap are cryptic verbs that don't describe the action type). Parameter descriptions are present but minimal (10-50 chars typically), falling well below the 72-char baseline for A+ tools. No output schemas are documented, the source code shows tool registration but provides zero visibility into what these tools return, making it impossible for LLMs to chain calls or extract structured data. Error handling is absent from the visible code; no guidance on retry behavior, permission checks, or recovery paths. The server implements security-sensitive tools (port scanning, SQL injection testing, mobile app analysis) with no visible rate limiting, permission gates, audit trails, or input sanitization. Tool composition is poor: uploading a file and scanning it are separate tools that require multi-step orchestration; no batch variants exist despite clear need (scanning multiple files). While parameter schemas are present (using Zod), they lack constraints (enums for valid options, ranges for numeric inputs), forcing LLMs to guess valid values. No tool annotations (readOnlyHint/destructiveHint) are visible despite WRITE risk on uploadFile. The server appears functional at a basic level but would not pass production code review.
Run ffuf with specified URL
Run masscan with specified target MASSCAN is a fast port scanner. The primary input parameters are the IP addresses/ranges you want to scan, and the port numbers.
Run nmap with specified taget
Run sqlmap with specified URL
Generate and retrieve a comprehensive security analysis report in JSON format for a scanned mobile application. This report includes detailed findings about security vulnerabilities, permissions, API calls, and other security-relevant information.
Retrieve a list of recently performed security scans on the MobSF server, showing mobile applications that have been analyzed, their statuses, and basic scan information.
Tool naming violates verb_noun convention. Prefix 'do-' is generic and non-standard. LLMs cannot infer intent from 'do-ffuf' alone without reading extended descriptions. Standard would be 'run_ffuf', 'scan_with_ffuf', or 'fuzz_url'. Affects: do-ffuf, do-masscan, do-nmap, do-sqlmap (4 tools, 44% of server).
No output schemas documented. Tools return command-line output (stdout from ffuf, masscan, nmap, sqlmap execution) with no schema definition. LLMs cannot know what fields to extract, cannot chain tools effectively, and must parse unstructured text. MobSF tools return API responses but structure is undocumented. Violates pattern:response-shaper and pattern:tool.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 38 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Retrieve detailed scan logs for a previously analyzed mobile application using its hash value. These logs contain information about the scanning process and any issues encountered.
Scan a file that has already been uploaded to MobSF. This tool analyzes the uploaded mobile application for security vulnerabilities and provides a comprehensive security assessment report.
Upload a mobile application file (APK, IPA, or APPX) to MobSF for security analysis. This is the first step before scanning and must be done prior to using other analysis functions.
Minimal parameter descriptions (10-50 chars vs 72-char baseline). Example: do-nmap has description 'Run nmap with specified taget' (typo: 'taget'). Parameter 'target' described as 'Target ip to detect open ports' is unclear about format (single IP? CIDR range? Hostname?). ffuf_args, masscan_args, nmap_args, sqlmap_args lack any guidance on format, quoting, escaping, or valid values.
No enum constraints on parameters accepting discrete values. MobSF's reScan is boolean but no guidance on when to set true. ffuf_args, masscan_args accept arbitrary strings with no validation, LLMs may pass malformed arguments, causing silent failures or injection attacks. Parameters like 'port' (do-masscan) lack range constraints (is 0 valid? 65535? 999999?).
No visible error handling or recovery guidance. If ffuf crashes, nmap times out, or sqlmap returns no results, code does not surface actionable error messages. No guidance for LLM on retry logic, permission requirements, or next steps. Violates pattern:recovery-guide and pattern:error-classification.
Security-sensitive tools lack permission gates and audit trails. uploadFile accepts arbitrary file paths; no visible validation against path traversal (../../../etc/passwd). Scanning tools (ffuf, nmap, masscan, sqlmap) can be weaponized for reconnaissance but no rate limiting, permission checks, or logging visible. No scope declarations (e.g., 'requires: security:scan'). Violates pattern:permission-gate, pattern:scope-declaration, pattern:audit-trail, pattern:tool-gateway.
No tool annotations visible. Tools marked as WRITE risk (uploadFile) lack destructiveHint annotation. Tools with side effects (send network traffic, modify files) lack idempotentHint declarations. Scanning tools lack readOnlyHint annotation. These annotations help LLMs understand consequence and retry safety. Violates 2026-07-28 spec requiring tool annotations.
No pagination support on list tools. getRecentScans accepts page and pageSize but returns no 'total_count', 'next_cursor', or metadata indicating whether more results exist. Without pagination hints, LLMs cannot know if they've retrieved all data or if they need additional calls. Violates pattern:paginated-result.
Poor tool composition. uploadFile and scanFile require separate calls; no composite 'upload_and_scan' exists. Multi-file scanning requires N sequential uploadFile + scanFile calls instead of a batch variant. Error handling is per-call; partial failures (3 of 5 scans succeed) offer no recovery path. Violates pattern:tool and composition best practices.
No downstream-tool data preservation. If scanFile returns results, it likely contains a hash, file name, and scan status, but if getJsonReport needs that hash and it's not in scanFile's output, LLM must make an extra getRecentScans call to recover it. Response chaining broken. Violates mxe:include-chaining-ids.