AI-augmented IT ops workbench: spec-driven, multi-stack, multi-UI. Provides orchestration for ticket summarization, knowledge base management, wiki generation, and integration with multiple LLM providers.
OpsPilot exposes a single tool, kb_search, with a well-structured input schema and clear description. The tool name follows verb_noun convention (search_*), and the description (169 chars) explains what it does, when to use it, and what it returns (chunk_id, document_id, content, citation block). Input schema is complete with type definitions, required fields, and constraints (top_k: 1-20, default 5). However, the 'query' parameter description is minimal (22 chars) and lacks guidance on expected format or language support beyond 'zh/en'. Output schema is documented in prose but not formally specified in JSON Schema. No error handling guidance is provided for edge cases (empty results, malformed queries, KB unavailable). The tool is read-only and well-scoped, but lacks the depth expected for production-grade tools.
Hybrid (vector + FTS5) retrieval over the OpsPilot KB. Use it to find SOP / runbook chunks relevant to the ticket. Returns up to top_k hits, each with chunk_id, document_id, the chunk content, and a citation block (source_path/line_start/line_end/heading_path) to plug into your final structured JSON.
Output schema not formally documented in JSON Schema format. Description mentions 'chunk_id, document_id, chunk content, citation block' but no structured schema is provided for LLM to parse response fields.
Parameter 'query' description is too brief (22 chars: 'Natural-language query (zh/en).') and lacks guidance on expected length, format, or examples of effective queries.
No error handling guidance. Tool description does not explain what happens on empty results, malformed queries, or KB unavailability, leaving LLM without recovery strategies.
Single tool limits composition patterns. OpsPilot exposes only kb_search; no complementary tools for KB management, document upload, or result filtering. Agents cannot compose multi-step workflows.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
No pagination or result limiting guidance in tool description. While top_k defaults to 5 and caps at 20, the description does not explain why this limit exists or what happens if more results are available.