Loading ToolBench
MCP Server for CM3588 NAS Kit knowledge base and SSH operations
This server has 11 tools with reasonable naming and Russian-language descriptions, but several definition quality gaps limit its score. Most tools have input schemas present (JSON Schema with types and descriptions), but lack critical elements: no output schemas are documented in the source, error handling guidance is minimal, and many descriptions lack sufficient context about WHEN to use each tool vs. alternatives. The tool names follow verb_noun convention (save_knowledge, search_knowledge, etc.), which is a positive. However, descriptions are inconsistent in quality, some are very brief (list_categories: 'Список всех категорий в базе знаний' = 'List all categories in the knowledge base', ~35 chars) and fall below the 50-100 char sweet spot. The server's Qdrant backend and semantic search capability are architecturally sound, but the interface definition lacks the polish expected of production tools. Code review shows lazy initialization of KnowledgeStore (_knowledge_store global), no explicit error recovery paths in tool logic, and no validation of enum values (e.g., category must be one of: hardware, voice-pipeline, npu, docker, troubleshooting, but this is not enforced as an enum in the schema).
Создать пошаговый гайд.
Задокументировать конфигурацию сервиса. Читает конфиг с CM3588 и сохраняет в базу знаний.
Получить историю изменений.
Получить полную запись по ID.
Список всех категорий в базе знаний.
Список записей в категории.
Залогировать изменение на CM3588. ВАЖНО: Вызывай после любых изменений на устройстве!
Output schemas not documented. For tools returning lists or dicts, the response structure is invisible to LLMs. E.g., search_knowledge returns list[dict] with {id, title, category, tags, preview}, but this is inferred from code, not documented in the tool schema.
Category and service_name parameters should be constrained as enums. Currently free-form strings. JSON Schema should declare: category: {type: string, enum: [hardware, voice-pipeline, npu, docker, troubleshooting]}. This prevents LLM hallucination of invalid categories.
Descriptions vary wildly in length and clarity. list_categories (35 chars) is below the 50-char minimum; get_changelog (35 chars) lacks context. Rewrite each to 50-150 chars explaining WHAT, WHEN, and expected return.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 21 | - | v1 |
Залогировать решение проблемы.
Сохранить знание о CM3588 в базу. Используй для документирования: - Настроек оборудования (микрофон, камера, GPIO) - Конфигураций сервисов (Whisper, Piper, Wyoming) - Решений проблем - Оптимизаций NPU
Поиск в базе знаний (семантический).
Обновить существующую запись.
No error recovery guidance. When a tool fails (e.g., entry_id not found in update_knowledge), the current code returns a string message 'Запись {entry_id} не найдена'. LLMs cannot parse this to decide whether to retry, search, or ask the user. Errors should be structured and include guidance.
No parameter constraints on numeric inputs. limit parameter in search_knowledge, list_knowledge, and get_changelog has no min/max bounds. LLMs may pass absurd values (limit: 10000). Add constraints: {type: integer, minimum: 1, maximum: 100}.
Tool composition: log_change, log_solution, and save_knowledge overlap in responsibility. log_solution internally creates a KnowledgeEntry, but this side effect is not documented. Unclear when to call save_knowledge vs. log_solution. One should be renamed or refactored to avoid LLM confusion.
Incomplete source visibility. The document_config tool implementation is truncated in the provided source code (ends mid-variable name 'resu'). Cannot verify error handling, timeout behavior, or actual return type. Capstone rule applied: tool capped at 73.
No idempotency guarantees. Repeated calls to save_knowledge, log_change, or log_solution with identical inputs will create duplicate records. Agents retry on ambiguous failures, idempotent operations require deduplication logic or at-most-once semantics. Not evident in code.