Loading ToolBench
Hive demonstrates solid foundational quality with clear, action-oriented tool names (remember, recall, forget, list_memories) and consistent parameter schemas. All 8 tools have descriptions and typed input schemas. However, descriptions are terse (avg ~40 chars, well below the 194-char baseline), parameter descriptions lack depth and constraint guidance, and output schemas are not documented. Error handling is minimal, no recovery guidance or actionable error messages visible. The server lacks tool annotations (readOnlyHint, destructiveHint) despite having clear risk classifications. Security is well-handled via OAuth token validation, but the tool interface could be more LLM-optimized.
delete a memory by key
list memories by tag
list distinct tags for the caller's memories
token-budget-aware context pack
health check (auth-only, no storage access)
retrieve a memory by key
store a memory
Tool descriptions are too brief (avg 40 chars vs 194-char baseline). Descriptions like 'store a memory' and 'retrieve a memory by key' lack context on WHEN to use each tool, prerequisites, or what the LLM should expect. LLMs cannot reliably select between similar tools without richer guidance.
Parameter descriptions lack constraint guidance. 'tags' in remember is described as 'optional tags for organizing memories' but does not specify format (comma-separated? array? max length?). 'topic' in summarize_context and pack_context lacks guidance on expected content or length. LLMs will guess at valid formats.
Output schemas are not documented in the source code. The code shows tool definitions but does not expose what fields (e.g., memory_id, created_at, tags) are returned by remember(), recall(), or list_memories(). LLMs cannot plan downstream tool calls without knowing the response structure.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | <=2025-11-25 | v2 |
synthesise memories into a summary
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are missing. The Risk field in the spec (READ_ONLY, WRITE, DESTRUCTIVE) is not exposed to the MCP protocol. Clients cannot determine which tools are safe to retry or which modify state without reading descriptions.
Error handling is not visible in the tool definitions. No recovery guidance (e.g., 'If memory not found, try list_memories() to see available keys'), no error categorization (retryable vs user-fixable), and no actionable error messages. Agents will not know how to recover from failures.