Loading ToolBench
MCP server for interacting with OnSecurity API to retrieve security assessment data including rounds, findings, notifications, prerequisites, and blocks
Server provides 5 READ_ONLY tools with good naming conventions (verb_noun pattern: get_rounds, get_findings, get_notifications, get_prerequisites, get_blocks). All tools have non-empty descriptions (74-109 chars, within recommended 10-1024 range). Input schemas are present with type definitions and parameter descriptions. However, critical gaps exist: (1) Output schemas are completely undocumented, no indication of what fields the response contains or structure agents should expect; (2) Parameter descriptions lack format constraints (e.g., which values are valid for 'sort'?); (3) No error handling guidance documented; (4) No pagination limits specified despite accepting page/limit params; (5) Parameters like 'sort' and 'search' lack enums or format specifications, inviting hallucinated values from LLMs. The code shows TypeScript interfaces defining response types (RoundResponse, FindingResponse, etc.) but these are not exposed in the tool schema for agent discovery. The server uses environment variable injection for secrets (ONSECURITY_API_BASE, ONSECURITY_API_TOKEN), which is correct.
Retrieve security assessment blocks (reusable finding templates) with filtering and pagination
Retrieve security findings with filtering, pagination, and search capabilities
Retrieve notification messages with pagination and filtering
Retrieve prerequisites for security assessment rounds with filtering and pagination
Retrieve security assessment rounds with pagination and filtering support
Output schemas are not documented. Response types are defined in TypeScript interfaces (RoundResponse, FindingResponse, etc.) but not exposed in tool output schemas. LLMs cannot know which fields to expect in results, preventing effective downstream planning and field extraction.
Parameter 'sort' lacks enum or format constraint. Description states 'Sort field and direction' but does not specify valid sort fields, field names, or direction syntax (asc|desc). LLMs will hallucinate invalid sort values. Should document: 'Field name and direction (e.g., name:asc, created_at:desc). Valid fields: [list].'
Parameter 'search' lacks clarification on which fields it searches. Description: 'Search term to filter rounds by name or other fields' is vague. Should specify exact fields and any format requirements (min/max length, regex pattern).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Pagination parameters 'page' and 'limit' lack constraints. No specification of valid ranges (e.g., page ≥ 1, limit 1-100). Code does not validate or document maximum result limits. Per production baselines, result caps should be 20-50 items per page, documented in the tool description.
No error handling guidance documented. The code contains try-catch with console.error, but tool descriptions do not guide LLMs on recovery. Missing patterns: (1) Which errors are retryable vs. fatal? (2) What should the agent do if a resource is not found? (3) What does 'null' return mean?
Parameter 'status' in get_findings lacks enum constraint. Description: 'Filter findings by status' does not enumerate valid status values (e.g., open, closed, resolved, in_review). LLMs cannot validate their own input and will pass invalid statuses.
Tool descriptions do not clarify mutual dependencies. Example: get_findings accepts both round_id and client_id, which is required? What if both are omitted? What if both are provided? Dependencies should be documented per parameter.