Loading ToolBench
Graphene demonstrates solid definition quality with 19 well-structured tools covering a persistent context graph domain. All tools have descriptions (avg 180 chars, within 10-1024 baseline) and explicit JSON Schema input definitions with typed properties. Naming follows verb_noun convention (read, search, upsert_node, learn, link, delete_node). However, output schemas are not documented, LLMs cannot predict response structure for downstream chaining. Parameter descriptions are present but inconsistent in depth; some lack format/constraint guidance (e.g., 'node_name' accepts 'repo:name' syntax but this is buried in tool descriptions rather than param descriptions). Error handling is absent from visible code, no recovery guidance, categorization, or actionable error messages. Batch tool lacks detailed schema for nested node/edge/observation objects. Multi-repo resolution logic is complex and underdocumented in parameter descriptions.
Bulk create/update nodes, edges, and observations in one transaction. Nodes and edges are upserted; observations are appended. All names accept repo:name in multi-repo sessions; from and to in edges must resolve to the same repo.
Remove a node and all its edges and observations.
Remove a user-level fact by category and subject.
Read user-level facts (preferences, expertise, conventions). No arguments returns all facts.
Write a user-level fact. One fact per category+subject pair; writing to an existing pair replaces the content.
Export all global facts to a portable markdown bundle at path, for moving user-level facts between machines.
Output schemas not documented. LLMs cannot predict response structure (fields, types, nested objects) for tools like read(), search(), status(). This breaks tool chaining, agents cannot extract IDs or references for downstream calls without trial-and-error.
Error handling absent. No recovery guidance, error categorization, or actionable messages visible in code. Tools like delete_node (destructive) and batch (multi-step) lack confirmation or dry-run patterns. Agents cannot distinguish retryable vs fatal failures.
Multi-repo resolution logic (repo:name syntax, repo inference from covers/entry_points) is complex but documented only in tool descriptions, not in parameter descriptions. LLMs must parse tool-level prose to understand parameter format, violates pattern:tool-description.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
Import global facts from a portable markdown bundle at path, merging by category+subject. Facts already present locally with different content are left untouched and reported as skipped unless overwrite is set.
Append a learned observation to a node. Observations are append-only and never overwrite existing ones.
Create an edge between two nodes. Bidirectional types (related_to, mirrors) automatically create edges in both directions. Idempotent: re-linking updates the reason. Names accept repo:name in multi-repo sessions; from and to must resolve to the same repo (cross-repo edges are not supported).
Remove a repo-scoped fact by category and subject.
Read repo-scoped facts (conventions, context, decisions specific to this project). No arguments returns all project facts. In multi-repo sessions, repo is required.
Write a repo-scoped fact. One fact per category+subject pair; writing to an existing pair replaces the content.
Read the context graph. No arguments returns the full index (all node names, types, summaries). With a name argument, returns the full node including outgoing edges, incoming dependents with neighbor summaries, and observations. In multi-repo sessions, index and node results include a repo field, and name accepts repo:name to disambiguate.
Delete a wrong or outdated observation by id.
Search across nodes, observations, project facts, global facts, and edge reasons. Multi-word queries match any word and rank by relevance. Returns at most the top 20 results, each with a truncated snippet.
Check which nodes have stale context by comparing their covered files against git changes since their last_commit.
Get a bounded snapshot: node index, stale nodes, current HEAD, and project/global fact counts and keys (never fact or observation bodies). Automatically injected at session start by the hook, but can be called manually to refresh. For observation or fact content, call read(name), project_read(), or global_read().
Remove an edge between two nodes. Without type, removes all edges between them. Names accept repo:name in multi-repo sessions.
Create a new node or merge-update an existing one. Only provided fields are changed on update. Metadata is shallow-merged. In multi-repo sessions, name accepts repo:name to target a specific repo; a bare name updates the existing node if exactly one repo has it, otherwise the repo is inferred from covers/entry_points paths.
Batch tool input schema lacks detail for nested objects. 'nodes', 'edges', 'observations' are arrays but their element schemas are not visible. LLMs cannot construct valid batch payloads without trial-and-error.
No enum constraints on edge types or fact categories. Tools like link() accept 'type' as free-form string (description says 'related_to, depends_on, mirrors, or extends' but no enum). Agents may hallucinate invalid values.