Loading ToolBench
面向建筑行业知识工作从业者与 AI 研发团队的 Skills、Workflow 与多 Agent 工具包 / Building-industry AI agent skills for BIM, IFC, RAG, GraphRAG, project evidence work, code review, and runtime governance.
ArchSight AIOS exposes 17 tools via STDIO with basic parameter schemas but significant quality gaps. Tool names use colon-separated namespaces (e.g., 'architecture:health', 'knowledge:init') rather than verb_noun convention, making intent less discoverable. Descriptions are present but brief (10-50 chars typical), lacking WHEN/WHY context. Parameter descriptions exist but are minimal ('Path to JSON file'). No output schemas documented. No error handling guidance visible. No tool annotations (readOnlyHint/destructiveHint). Tools like 'capability:call' and 'install' perform destructive operations without confirmation patterns. The codebase shows CLI argument parsing but no structured MCP response shaping or recovery guidance.
Scan normalized architecture facts and apply baseline ratchets.
Authorize and call a registered local Capability adapter.
Check repository assets and user-level installation.
Detect drift in Hermes runtime prompt sync.
Perform a dry-run of Hermes runtime prompt sync.
Validate Hermes runtime prompt sync.
Add AI rules and .ai governance files to a project.
Tool names use colon-separated namespace convention (e.g., 'knowledge:init', 'architecture:health') instead of verb_noun pattern. LLMs cannot infer action intent from names alone; 'knowledge:init' is ambiguous vs. 'create_knowledge_pack'.
Descriptions are uniformly brief (10-50 chars) and lack WHEN/WHY context. E.g., 'Create a Knowledge Pack workbench with optional sample content' does not explain when to call this vs. other knowledge tools or what 'workbench' means to an LLM.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2026-07-28+ | v2 |
Install AIOS assets into user-level assistant locations.
Compile a Knowledge Pack into a JSON file.
Evaluate a Knowledge Pack.
Create a Knowledge Pack workbench with optional sample content.
Inspect a compiled Knowledge Pack.
Query a Knowledge Pack for relevant information.
Check a Knowledge Pack workbench for correctness.
Validate the project AI template output.
Create a Markdown document-writing workbench.
Check a Markdown document-writing workbench.
No output schemas documented. Tools return results but LLMs cannot plan downstream calls or extract required fields. E.g., 'knowledge:lookup' returns results but schema for result structure is not visible.
Destructive operations ('install', 'knowledge:compile', 'init') lack confirmation or dry-run patterns. No error guidance for recovery. LLMs cannot safely retry or understand consequences.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. LLMs cannot distinguish safe reads from destructive writes, increasing risk of unintended side effects.