Loading ToolBench
Exposes NMIS monitoring data via the Model Context Protocol so that AI assistants can query node status, metrics, events, and inventory.
Server exposes 5 read-only monitoring tools with HTTP transport. Tool names follow verb_noun convention (nmis_list_*, nmis_get_*) and are descriptive. All tools have descriptions (100-200 chars, within baseline). Input schemas are present with proper JSON Schema structure and parameter descriptions. However, output schemas are not documented, LLMs cannot predict response structure for planning. Parameter descriptions are adequate but lack format constraints (e.g., no enum for 'concept' values, no range limits). Error handling is absent from visible code, no guidance on retryability, user-fixable errors, or recovery paths. No tool annotations (readOnlyHint present in metadata but not in schema). Missing pagination support for list tools despite potential for large result sets.
Get the latest collected metrics for a node and concept. Metrics use OTel-inspired field names. For catchall/ping, returns per-subconcept results (health, tcp, laload, etc.). For interface, returns per-interface results.
Get detailed status and health metrics for a specific NMIS node, including overall status (reachable/degraded/unreachable), reachability, availability, response time, system description, and uptime.
List active NMIS events and alerts. Optionally filter by node name.
List inventory instances for a node and concept, showing index, description, and available data fields. Useful for discovering what instances exist before fetching metrics.
List all NMIS monitored nodes with basic status (name, group, type, host, health, reachability). Returns a summary for every node.
Output schemas not documented. LLMs cannot predict response structure, field names, or types for downstream planning. Breaks tool chaining and forces exploratory calls.
'concept' parameter in nmis_get_latest_metrics and nmis_list_inventory accepts free-form strings instead of enum. Description lists valid values (interface, catchall, device, etc.) but no schema constraint. LLMs will hallucinate invalid concept names.
No pagination support (limit, offset, page_size, next_cursor) on list tools. nmis_list_nodes and nmis_list_events could return hundreds of items, exhausting context window. No documented result limits.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
No error handling guidance in visible code. No recovery hints (e.g., 'Node not found. Try nmis_list_nodes() to see available nodes'). Agents cannot self-correct on failures.
Tool annotations (readOnlyHint) declared in metadata but not in JSON Schema. Schema lacks 'readOnly' or 'destructive' properties. LLMs cannot infer safety from schema alone.