Loading ToolBench
MCP server exposing Seshat's Knowledge Layer to external agents via the Model Context Protocol. Provides semantic search, entity retrieval, fact storage, session context, and observation querying capabilities.
The server defines 6 tools with mixed quality. All tools have names starting with action verbs (seshat_*) and descriptions present, which is good. However, schemas are incomplete for several tools, descriptions lack LLM-optimization depth, and error handling is not visible in the provided code. Parameter descriptions are present but often generic. The tools form a coherent domain (knowledge graph operations + delegation), but individual tool quality is inconsistent. No evidence of output schema documentation, error recovery guidance, or security considerations in the visible code.
Delegate a sub-task back to Seshat for processing (reverse delegation). Requires the 'delegate' scope in the caller's access token. Delegated tasks run through Seshat's normal governance pipeline.
Retrieve a specific entity from the knowledge graph by its ID, including relationships, access history, and confidence scores.
Read conversation history from a Seshat session. Useful for delegation context -- understand what was discussed before.
Query Seshat's execution traces, cost data, and performance metrics. Filter by trace ID or return the most recent observations.
Search Seshat's knowledge graph for entities and relationships. Returns ranked results with freshness and confidence metadata.
Store a new entity or relationship in the knowledge graph. The source is tagged as 'external_agent' for provenance tracking.
Output schemas not documented. Tools return data (e.g., 'ranked results with freshness and confidence metadata' for seshat_search_knowledge) but the actual response structure is not visible. LLMs cannot plan downstream calls or extract fields without knowing the output shape.
Descriptions lack LLM-optimization depth. seshat_query_observations description (58 chars) is below the recommended 10 - 1024 range sweet spot (194 avg for production tools). seshat_store_fact (67 chars) and seshat_delegate (206 chars) are borderline. Descriptions do not explain WHEN to use each tool vs. similar ones, or what prerequisites exist.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 62 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Parameter descriptions are present but generic. E.g., 'Free-text search query' (seshat_search_knowledge.query) does not guide the LLM on query syntax, expected keywords, or when to use prefix vs. full-text search. seshat_delegate.details lacks any description of what structured fields are valid.
Error handling and recovery guidance not visible. No evidence of error response schemas, actionable error messages, or guidance for retryable vs. fatal failures. seshat_delegate mentions 'requires delegate scope' but no error is documented for permission denial.
seshat_delegate.type enum values lack context. 'linear_issue', 'knowledge_query', 'decomposition' are opaque to an LLM, the description should clarify which task type to use for common scenarios (e.g., 'use linear_issue for bug reports or feature requests').
seshat_store_fact and seshat_delegate accept 'metadata' and 'details' objects with no type constraints or field documentation. Free-form objects invite misuse and hallucinated fields. seshat_store_fact.metadata should document valid keys and value types.
No pagination or result limits documented. seshat_search_knowledge.limit defaults to 10, seshat_get_session_context.limit defaults to 20, but seshat_query_observations has no documented default and no maximum stated. seshat_get_entity returns 'relationships', is that paginated?
Security not addressed. seshat_get_session_context accepts a session_id parameter with no documented access control. Who can read whose session history? seshat_delegate requires 'delegate' scope but no error handling for permission denial is visible. seshat_store_fact allows writing facts, what prevents unauthorized writes?