Loading ToolBench

Reading the current corpus…

isis-mcp MCP server · F · ToolBench

← Browse

isis-mcp

github.com/yazeedh/isis-mcp

MCP server for managing Cisco IS-IS network routing configuration and monitoring

2026-07-28+STDIO5 tools

38/100

Definition Quality44

Protocol Readiness50

Supportability21

Spec posture

Inferred effective spec 2026-07-28+·current baseline 2026-07-28

Static source inference · medium confidence

Current-spec patterns detected

Summary

ISIS-Network-Agent has 5 tools with basic verb-noun naming (get_*, set_*, configure_*), but critical gaps undermine quality. All tools have descriptions (10-60 chars), but 3 read-only tools lack input schemas entirely (schema score = 0). Two write tools have partial schemas with parameter descriptions. No output schemas documented. Error handling is minimal, validation exists but lacks recovery guidance. No tool annotations (readOnlyHint/destructiveHint). The server exposes credentials via environment variables (correct), but lacks audit logging and permission gates for destructive operations. Composition is reasonable (5 focused tools), but missing pagination, structured output, and error classification.

Tools (5)

configure_isis_netwriteauthsource verified52/100

Configures the IS-IS NET. Format: 49.0001.1921.6800.0001.00

get_device_inforead onlyauthsource verified38/100

Retrieves the hostname, version, and uptime of the router.

get_interfacesread onlyauthsource verified38/100

Lists all interfaces and their current IP/Status.

get_isis_neighborsread onlyauthsource verified38/100

Shows all active IS-IS adjacencies.

set_interface_statewriteauthsource verified55/100

Enables (up) or Disables (down) a specific interface.

Issues (5)

CRITICALtool ↗

Three read-only tools (get_device_info, get_isis_neighbors, get_interfaces) have no input schemas.

get_device_infoget_isis_neighborsget_interfaces

CRITICALresponse-shaper ↗

No output schemas documented for any tool. LLMs cannot plan downstream calls or extract structured data. All tools return raw strings from CLI commands.

get_device_infoget_isis_neighborsget_interfacesconfigure_isis_netset_interface_state

HIGHconfirmation-request ↗

Destructive tools (configure_isis_net, set_interface_state) lack tool annotations (destructiveHint) and confirmation/dry-run support. No recovery guidance in error messages.

configure_isis_netset_interface_state

HIGHrecovery-guide ↗

Error messages are generic ('Error: Invalid IS-IS NET format'). No recovery guidance, no actionable next steps for LLM. Missing error classification (retryable vs user-fixable).

Recommendations

Add explicit input schemas to read-only tools. Even with no parameters, declare Input: {} with type: 'object', properties: {}, required: [].
Document output schemas for all tools. E.g., get_device_info returns {hostname: string, version: string, uptime: string}. Parse CLI output into structured JSON.
Add tool annotations: mark get_* tools with readOnlyHint: true; mark configure_* and set_* with destructiveHint: true.
Enhance error messages with recovery guidance. E.g., 'Invalid IS-IS NET format. Expected: 49.XXXX.XXXX.XXXX.XXXX.00 (e.g., 49.0001.1921.6800.0001.00). Call get_device_info to verify current config.'
Implement audit logging: log tool calls with timestamp, caller identity, parameters, and result to a file or syslog. Add permission checks before destructive operations.

Score history

Overall score trend

First recorded score · v2 rubric

38/100

Scored	Grade	Overall	Spec posture	Rubric
2026-09-23	F	38	2026-07-28+	v2