AI Chatbot using local LLM via Ollama and IP Fabric MCP server for network inventory and topology queries
Server has 12 tools with basic descriptions and parameter schemas. All tools follow verb_noun naming (ipf_get_*, ipf_set_*). However, descriptions are generic and lack LLM-optimized guidance on WHEN to use each tool vs. similar ones. Parameter descriptions are minimal (e.g., 'Optional dict of filters' lacks format/constraint details). No output schemas documented. Error handling returns generic error objects without recovery guidance. Most tools are READ_ONLY inventory queries with repetitive structure, suggesting composition issues. Baseline: avg tool description ~60 chars (well below 194-char production baseline); params lack type constraints (enums, ranges, patterns).
Get available columns for a specific table type
Get IP Fabric connection information and status
Get devices from IP Fabric inventory
Get interfaces from IP Fabric inventory
Get neighbor discovery data from IP Fabric
Get platforms from IP Fabric inventory
Parameter descriptions lack format constraints and validation rules. 'Optional dict of filters' does not specify filter key names, value types, or examples. LLMs cannot infer valid filter structures.
No output schemas documented. Tools return raw API responses without specifying field names, types, or structure. LLMs cannot plan downstream calls or extract required data reliably.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | <=2025-11-25 | v2 |
Get routing table data from IP Fabric
Get sites from IP Fabric inventory
Get all available snapshots from IP Fabric
Get vendors from IP Fabric inventory
Get VLAN data from IP Fabric
Set the active snapshot for queries
Error handling returns generic {'error': str(e)} without recovery guidance. LLMs cannot determine if errors are retryable, user-fixable, or fatal. No actionable next steps provided.
Nine tools (ipf_get_devices through ipf_get_neighbors) are nearly identical in structure and parameters. This violates single-responsibility principle and forces LLMs to reason about subtle distinctions. Consider a unified 'ipf_query_inventory' tool with a 'table_type' enum parameter.
Tool descriptions are under 50 characters and lack WHEN/WHY guidance. 'Get devices from IP Fabric inventory' does not explain when to call this vs. ipf_get_interfaces or ipf_get_sites. LLMs conflate similar tools.