Loading ToolBench
A knowledge graph and semantic search engine with HTTP API, supporting record management, curation, and temporal queries
Gramaton has 13 tools with basic naming and descriptions, but significant gaps in schema completeness and parameter documentation. Tool names follow verb_noun convention (Save, Search, GetRecord), but descriptions are brief (avg ~80 chars, below the 194-char baseline). Input schemas are present but lack detailed parameter descriptions, most parameters have only 1-2 word descriptions ('The record ID', 'Search query text'). No output schemas documented. Error handling is absent from tool definitions. Read-only tools (Search, GetRecord, GetStatus, GetStats) are well-separated from write tools, but composition issues exist: SaveBatch and SessionSave both handle bulk operations without clear differentiation. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present.
Classify a record with temporality, keywords, and summary information
Add a record to a collection with field values
Create a new collection for organizing related records
Retrieve a record by ID with all properties
Get detailed statistics about the store including record counts and index metrics
Get server and store status including node count, branch, and curation state
Save a record with content, keywords, summary, and metadata to the knowledge graph
Save multiple records in a batch operation with support for chunking and async processing
Parameter descriptions are minimal (1-2 words). 'The record ID', 'Search query text', 'Session name' lack context on format, constraints, or when to use. LLMs cannot infer whether ID is a UUID, integer, or slug; whether query supports regex; or whether name has length limits.
No output schemas documented. Tools return data but LLMs have no visibility into response structure. Search, GetRecord, GetStatus, GetStats lack documented return types, forcing LLMs to guess field names and types for downstream composition.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
Search the knowledge graph using BM25 full-text and vector similarity
Prepare a session for saving segments
Save segments within a session with optional promotion to memory records
Start a new session for capturing and organizing related records
Update properties of an existing record including content, summary, and metadata
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Write tools (Save, Update, Classify, SessionSave, CollectionAdd) are not marked as destructive. Read tools (Search, GetRecord, GetStatus, GetStats) are not marked as read-only. Agents cannot distinguish safe from unsafe operations without reading descriptions.
SaveBatch and SessionSave both handle bulk operations. SaveBatch accepts 'Items' array with chunking parameters; SessionSave accepts 'segments' array. Naming and parameter structure differ, creating confusion about when to use each. No clear differentiation in descriptions.
No error handling guidance in tool definitions. Tools lack descriptions of failure modes, recovery steps, or actionable error messages. Agents cannot plan retries or fallbacks.