Loading ToolBench
The server has 22 tools with reasonable naming and schemas, but suffers from inconsistent description quality and missing parameter documentation. Most tools have descriptions between 34-200 characters (within the 10-1024 baseline), but several lack depth about error conditions, prerequisites, or when to use them vs. alternatives. Parameter descriptions exist but are often generic ('poem identifier (ID or title)' repeated across many tools). Output schemas are implicit in result models (Poem, SearchResult, SyncResult) but not explicitly documented in tool definitions. No tool annotations (readOnlyHint/destructiveHint) despite clear semantic distinctions between READ_ONLY, WRITE, and DESTRUCTIVE operations. Error handling is minimal, no recovery guidance in descriptions. The catalog_tools.py excerpt shows reasonable implementation but lacks per-tool error documentation.
Add poems to an existing chain. If chain doesn't exist, creates it. If positions provided, assigns those positions.
Validate and write quality scores to a poem's frontmatter.
Create a new chain with initial poems. Args: chain_id (unique identifier, will be normalized), poem_ids (list of poem IDs to include), ordered (if True, assign positions 1, 2, 3...).
Create a new nexus file and refresh the registry.
Create a new submission record on disk. Validates submission data, checks venue and poem catalog, writes to disk with backup, and optionally syncs catalog.
Delete a nexus, optionally cleaning its tag off poems.
Find poems scoring >= min_score on all requested quality dimensions.
Missing tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite clear risk semantics marked in spec (READ_ONLY, WRITE, DESTRUCTIVE). LLM cannot infer which tools are safe to retry or irreversible without annotations.
Descriptions lack explicit error recovery guidance. E.g., 'Create a new chain' does not say what happens if poem_ids contain invalid poems, or how the LLM should respond to failure. Many tools marked WRITE lack dry-run or confirmation step guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Prepare poem and nexus data for theme analysis by the MCP agent. Returns poem content and available themes for the agent (Claude) to analyze. The agent will identify which themes match and provide confidence scores.
Find poems similar to a given poem through metadata connections. Scores candidates by: shared nexus membership (weight 3.0), shared chain co-membership (weight 2.0), shared plain tags (weight 1.0), same form (weight 0.5).
Get all nexuses (themes/motifs/forms) from the registry. Returns complete registry with all nexus entries, organized by category. Use this to discover available themes, motifs, and forms for tagging poems.
Get a poem by ID or title.
Get list of poems needing theme enrichment for agent analysis. Returns poems with minimal or no tags for the agent to analyze and suggest themes. The agent can analyze multiple poems and suggest which themes to apply.
Read quality scores + notes from a poem's frontmatter.
Server metadata + catalog statistics.
Link a poem to a nexus by adding the nexus's canonical tag to the poem's frontmatter. Safely updates the poem's tags field in frontmatter, preserving all other fields. Creates a backup before modification. Automatically resyncs catalog after update.
List submissions with optional filtering by venue, status, or poem.
Unified search/filter/sort over poems. Supports text search, filtering by state/form/tags/chain/quality, and sorting by relevance/title/dates/word count.
Count tagged poems per nexus and update the registry.
Remove poems from a chain. If compact_positions is True, renumber remaining poems to close gaps.
Scan the vault and rebuild the in-memory catalog indices.
Scan submissions and auto-regenerate venue files.
Validate that all poem tags match a nexus canonical_tag.
Parameter descriptions are generic and repetitive. 'Poem identifier (ID or title)' appears across 5+ tools. Descriptions should clarify: when to use ID vs title, whether lookup is automatic, error behavior if ambiguous. Current form is low signal.
Output schemas not explicitly documented in tool definitions. SearchResult, SyncResult, Poem models exist in code, but tool descriptions do not state 'Returns SearchResult with {poems: [], total_matches: int, query_time_ms: float}'. LLM cannot plan downstream calls without knowing response structure.
No pagination guidance in list/query tools. query_poems accepts 'limit' but description does not state default (appears to be config.search.default_limit), max cap, or whether results are truncated. list_submissions, get_poems_for_enrichment lack explicit pagination documentation.
Inconsistent parameter naming conventions. Some tools use 'poem_ids' (array), others use 'identifier' (string). Ambiguity around 'poem' vs 'poem_id' vs 'poem_title'. No explicit guidance on which form to prefer. Increases LLM confusion when juggling 22 tools.
delete_nexus marked DESTRUCTIVE but description lacks confirmation or dry-run guidance. No mention of 'cleanup_poems' boolean purpose or consequences. LLM could accidentally delete tags from all poems without realizing impact.
Relationship constraints not documented. E.g., 'add_poems_to_chain' says 'If chain doesn't exist, creates it' but doesn't document whether this is atomic, what happens if poem_ids are invalid, or whether positions are required vs optional.