Loading ToolBench
Contract-first visual control plane between developers and AI coding agents. Provides MCP tools for block graph management, acceptance verification, proposal handling, and agent orchestration.
38 tools with highly variable quality. Most have basic descriptions (50-150 chars) but lack parameter-level documentation. Input schemas present but sparse: many parameters undescribed (e.g., 'block_id' in read_block has no description). No output schemas documented. Error handling absent, tools return raw results with no recovery guidance. Naming is inconsistent: verb_noun pattern mostly followed, but some tools like 'subagent_schema_syncer' and 'subagent_verifier' are vague about what they actually do. No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Composition is poor: many tools do multiple things (e.g., 'sima_fill_from_chat' extracts, fills, and proposes in one call). Parameter descriptions missing entirely for most tools.
PR3.5: accept a pending LLM proposal — applies structural changes to graph.json + checks.log trace
Apply queued distillates into block memory files
Build deterministic context-pack json for block. profile (S-4): design (full ~5-15K), backend-fix (~2-4K), ui-fix (~1.5-3K), acceptance-only (~0.5-1.5K). Default: design.
R-7.84: verify a block + all its dependents (transitive closure). Returns {block_id, verdict, cascade: [{block_id, verdict, ...}]}.
PR-2 (b.acceptance-verifier-loop): run a single deterministic evidence collector. evidence_kind ∈ {exit_code, fs_glob, file_diff, log_grep, selftest_run}. Returns {verdict, evidence, reasoning, raw, duration_ms}.
Create/init block in atlas graph and docs
Parameter descriptions missing across all 38 tools. E.g., 'block_id' in read_block, list_dependencies, create_block has no description explaining what a block_id is or how to obtain it. LLMs cannot infer parameter semantics from names alone.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. E.g., does 'read_block' return markdown text, structured fields, or both? Does 'list_proposals' return proposal IDs, full objects, or summaries?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
Queue distilled chat insight for nightly ingestion
Generate wiki, auto_tz and roadmap
Generate ТЗ/auto_tz.md from atlas
Run sync checks then generate bundle if all pass
Generate atlas WIKI.md
Batch-ingest transcript JSONL into queue and apply automatically
Append distilled chat insight to decisions/patterns/checks of block
List dependencies from depends_on.md
PR3.5: list pending LLM proposals for existing blocks
Append check record to block checks.log
Append dead-file mark in block tasks
Run validators + generators and write atlas/nightly_report.md
PR-1 (b.acceptance-verifier-loop): parse atlas/blocks/<id>/acceptance.md into structured assertions (id, label, text, checked, evidence_kind, evidence_spec). Default evidence_kind = llm_judge when not declared in YAML block.
Read all markdown files for block
PR-1 (b.operator-profile-learner): read aggregated operator profile (profile.json). When no profile exists yet — returns warming_up.
PR-1 (b.operator-profile-learner): re-aggregate operator profile from current repo signals (transitions / checks.log / llm_traces / proposals). Idempotent.
PR3.5: reject a pending LLM proposal with a reason
Render atlas/WIKI.md to atlas/wiki.html
PR4.5: run the configured coding agent (claude / codex / cursor) on the given block; writes prompt to atlas/agent_invocations/ and logs an agent_invocation check
Update mission.md for block
Overwrite depends_on.md entries
Overwrite provides.md entries
Overwrite tasks.md body
Phase R-2/R-7.19: take a chat transcript and orchestrate the full pipeline — extract insights, fill weak fields on existing blocks, propose 1-3 new blocks. Saves a plan to atlas/proposals/ (root) or atlas/clients/<client_id>/proposals/ (per-tenant). The operator reviews the plan in the «✦ Предложения» panel. **Pass client_id to target a specific tenant; otherwise root atlas is used.**
Phase R-3: scan Claude Code session jsonl files (~/.claude/projects/*/), detect new conversational turns since the last run, and either propose (dry-run plan) or auto-apply via sima_fill_from_chat. Use this when the operator says «sima, check the latest chats» or to power a periodic background sweep. Returns {plan, new_turns, skipped_reason?}.
Phase N-3: walk all atlas blocks, run all consistency validators, return drift report.
Phase N-3: run acceptance verifier + LLM-validator (mission vs reality) on a block (or --all).
Phase N-3: regenerate WIKI.md / wiki.html / roadmap.md / auto_tz.md from canonical graph + block files.
Run atlas validators (contracts/dependencies/acceptance)
Change block status in graph
Atomic update for title/status/depends/provides/tasks/mission
PR-2 (b.acceptance-verifier-loop): parse acceptance.md AND collect evidence per assertion in one shot. Returns {block_id, assertions: [...], counts: {pass, fail, skipped}, verdict}.
No error handling or recovery guidance. Tools provide no actionable error messages. E.g., if 'create_block' fails because block_id already exists, does it return a conflict error? Can the LLM retry with a different ID? No guidance provided.
Vague tool names reduce discoverability. 'subagent_schema_syncer', 'subagent_verifier', 'subagent_wiki_builder' do not clearly convey what they do. Names like 'sync_schema', 'verify_contracts', 'generate_wiki' would be clearer. LLMs struggle to distinguish between similar-sounding tools.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot distinguish safe read-only tools from destructive ones. E.g., 'generate_wiki' and 'generate_validated_bundle' modify state but are not marked as destructive. Agents may retry them unnecessarily or fail to recognize side effects.