Loading ToolBench
DevSpan demonstrates solid tool design with consistent naming (verb_noun pattern), comprehensive parameter descriptions, and explicit tool annotations. All 32 tools have clear descriptions (avg ~150 chars) and input schemas with typed parameters. However, output schemas are undocumented, responses are inferred from code, not declared in tool definitions. Error handling lacks recovery guidance. Some parameter descriptions could be more prescriptive about constraints and formats. Tool composition is excellent (single responsibility, proper chaining via refs), but pagination and result limits are absent from list tools.
Add a short nickname for an entity. Globally unique. Example: add_alias("component:api-gateway", "gw") — then resolve_ref("gw") returns the component.
Attach an external reference (Jira, GitHub, etc.) to a Task.
Attach a Governance ref to any entity.
Append text to a subtopic (non-idempotent).
Add a ref to the always_include list (auto-merged into every task pack).
Bind a workspace directory to a Repo slug.
Read the current config (always_include refs, workspace bindings).
Output schemas undocumented. Tool definitions declare input schemas but no return types are visible in the tool registration. LLMs cannot plan downstream calls or extract fields without knowing response structure.
List tools lack pagination. list_components, list_repos, list_tasks, list_governance_entities have no limit, offset, or cursor parameters. Large result sets will blow context windows and degrade LLM reasoning.
Error handling lacks recovery guidance. Tools return success/failure dicts but do not provide actionable error messages or suggest next steps. E.g., if resolve_ref finds no matches, it should suggest available alternatives.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | <=2025-11-25 | v2 |
Remove a ref from the always_include list.
Remove a workspace binding.
Create a new Component entity. A Component is any coherent unit of functionality that you hold context about: an internal subsystem, an external service, an API, a database, a library, or a tool. The `kind` tag lets you filter later.
Create a new Governance entity — a reusable guideline. Governance entities hold cross-cutting rules that apply to multiple components/repos/tasks: security policies, API design standards, branding guidelines, user preferences. Any other entity can reference them via `governance: [...]` in its frontmatter or via add_governance. When assembling a task pack, governance refs cascade: the task's own governance + governance attached to each linked component/repo + parent components of linked repos. All deduped.
Create a new Repo entity. Repos hold long-lived facts about a codebase: style, testing, layout, local-dev setup, conventions. A repo optionally belongs to a parent Component (e.g. gateway-proxy-repo -> component:api-gateway).
Create a new Task entity. Tasks are transient units of work (a feature, a bug, an investigation) that compose context by linking specific Component/Repo/Governance refs via link_task. Scaffolded subtopics: goal, plan, notes.
Delete an entity and ALL its subtopics. Destructive. Also cleans up: aliases, task_links referencing this entity, governance links, uses-graph edges, and nullifies repo->component parent pointers. Requires an entity-level ref (no subtopic).
Read the current context for an entity or a single subtopic. Accepts an entity ref ("component:api-gateway") or a subtopic ref ("component:api-gateway/auth"). For entity refs, returns the meta + all subtopics.
Assemble a Task's full context pack: task notes + governance + linked refs.
Import external content (from Confluence, GitHub, etc.) into a subtopic with source tracking.
Link a Component/Repo/Governance ref into a Task's context pack.
List all Components, optionally filtered by kind.
List all Governance entities.
List all Repos.
List all Tasks.
Refresh imported content from its external source.
Rebuild the SQLite FTS5 index from disk markdown files.
Remove an alias from an entity.
Remove an external reference from a Task.
Remove a Governance ref from an entity.
Resolve a fuzzy query ("the gateway", "gw", "auth-gw") to candidate refs. Checks exact slug, exact alias, and case-insensitive name substring. Returns ranked candidates [{ref, source, score}] — call this before writing if the user refers to something by nickname or description.
Write or replace the content of a subtopic.
Suggest Component/Repo refs to link into a Task based on its description and the `uses` graph.
Tail the DevSpan activity log (last N lines).
Remove a link from a Task.
Minimal descriptions on list/config tools. list_repos, list_tasks, list_governance_entities, config_get have descriptions under 50 chars. Insufficient context for LLM tool selection.
No dry-run or confirmation for destructive operations. delete_entity is marked destructive but offers no confirmation step or preview. Agents can accidentally delete entire entity trees.