Loading ToolBench
MCP server for querying Rapid7 InsightOps logs with support for logset queries, polling, and listing available logsets
Four tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. All tools have descriptions (10-50 chars, below the 50-200 char baseline for LLM optimization). Input schemas present but sparse: most parameters lack detailed constraints, format guidance, or validation rules. No output schemas documented. Error handling returns generic error text without recovery guidance. Tool names follow verb_noun pattern (query*, poll*, list*) but descriptions are minimal. No tool annotations (readOnlyHint, etc.). Composition is reasonable, tools are single-purpose, but parameter interdependencies undocumented.
List all available Rapid7 logs sets
Poll the status of a running Rapid7 log query using its query ID
Query Rapid7 logs with specified parameters for an entire log set
Query Rapid7 logs with specified parameters for a logset identified by name
Descriptions are too brief (10-50 chars vs. 50-200 char baseline). 'Query Rapid7 logs with specified parameters for an entire log set' lacks WHEN to use, prerequisites, or what the response contains. LLMs cannot reliably select between queryRapid7Logset and queryRapid7LogsetByName without richer context.
No output schemas documented. Callers cannot know what fields to expect from responses. Code returns JSON.stringify(data) as text, LLMs must parse unstructured output, wasting tokens and risking extraction errors.
Parameter descriptions lack format guidance and constraints. 'from' and 'to' state ISO8601 format but no examples or validation rules. 'perPage' has default=100 but no min/max bounds, LLM could pass 10000 and break the API. 'query' syntax hint ('where("search term", loose)') is buried in description, not formalized.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
Error handling returns generic 'Error: <message>' text without recovery guidance. 'API request failed: 401 Unauthorized' tells the LLM nothing, should say 'API key missing or invalid. Ensure RAPID7_API_KEY environment variable is set.' Errors are not categorized as retryable vs. fatal.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All four tools are read-only and idempotent, but this is not declared in the schema. Agents cannot infer safety properties without explicit hints.