Loading ToolBench
Source-grounded chunk validation and three-agent deliberation over ingested documents via MCP protocol
Anchor demonstrates solid tool design with 11 well-named, verb-prefixed tools covering document ingestion, search, retrieval, and validation workflows. All tools have descriptions (avg 180 chars, within 10-1024 baseline) and input schemas with typed parameters. However, output schemas are not documented in the source code, only input schemas are visible. Parameter descriptions are present but vary in specificity; some lack format constraints or range guidance. Error handling is mentioned (errorReporting=true) but recovery guidance is not evident in tool descriptions. The tool composition is strong: each tool has a single responsibility, and chaining is well-supported (e.g., anchor_list_documents → anchor_describe_document → anchor_retrieve). No security issues detected (no credentials in params). Naming is consistent (anchor_* prefix, verb_noun pattern). Descriptions are LLM-optimized and include context on when to use each tool (e.g., 'Use this to see what's available before calling search/retrieve/ask').
Submit a deliberation over a document. Blocks for up to wait_seconds (default 120) so one tool call can return the deliberation result inline. On timeout returns the job_id so the agent can poll via anchor_get_ask_result.
Full structure of one document: chapters, sections, summaries at every level. No raw chunk text (use anchor_retrieve for that).
Poll the result of a deliberation by job_id. Returns the full job envelope including status, final_response, and supporting evidence.
Poll the result of an ingest job by job_id. Returns the full job envelope including status, phase, percent_complete, message, and final document metadata.
Ingest a document from a server-side path. Returns immediately with a job_id; the pipeline (parse → summarise → embed → persist) takes minutes on a full book and runs on the server's ingest pool. Poll progress via anchor_get_ingest_result or block via the client SDK.
Output schemas not documented in source. Tool descriptions explain what is returned (e.g., 'Returns top-k chunks wrapped with ancestor stack'), but formal output schema definitions are not visible in McpToolRegistry.java. LLMs cannot reliably plan downstream calls without knowing response field names and types.
Parameter format constraints missing. 'wait_seconds' lacks min/max bounds (e.g., 1 - 300); 'k' (top-k results) lacks range guidance; 'limit' and 'offset' lack documented bounds. Without explicit constraints, LLMs may pass invalid values (e.g., k=10000, wait_seconds=999999).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | A | 82 | 2025-06-18+ | v2 |
Upload a local document to the server via multipart, then ingest it. PDF, EPUB, DOCX, RTF, HTML, plain text — the server dispatches to PDFBox or Tika based on file extension. Returns a job_id for progress polling; same idempotency (re-upload → same content hash → same stable document_id) as anchor_ingest.
List ingested documents (id, title, summary, ingestion timestamp, chapter/chunk counts). Use this to see what's available before calling search/retrieve/ask.
Vector-only stance approximation — no LLM call. Two embedding round trips, returns a topical relevance + heuristic stance score in [-1, 1]. Use as a pre-filter before anchor_ask, not as a substitute for the deliberation.
Semantic retrieval within a document (or corpus-wide). Returns top-k chunks each wrapped with their full ancestor stack (paragraph / section / chapter / document summaries) — no follow-up read needed to understand context. Pass document_id to scope, omit for corpus-wide.
Semantic search across all documents — top-k by cosine of the query embedding against each document's stored summary embedding. Topical relevance, NOT stance — for stance use anchor_quick_validate or anchor_validate_chunk.
Source-grounded validation of a specific chunk against a query. Three-agent deliberation: retriever finds supporting/opposing chunks, validator scores them, synthesizer produces a final stance with evidence.
Error recovery guidance not visible in tool descriptions. While errorReporting=true, descriptions do not include recovery hints (e.g., 'If chunk_id not found, call anchor_retrieve to discover valid chunks'). This forces LLMs to guess next steps on failure.
Pagination limits not enforced in descriptions. anchor_list_documents accepts 'limit' but no max is stated; anchor_search_documents accepts 'k' but no upper bound is documented.
Tool composition: anchor_ingest and anchor_ingest_upload both ingest documents but via different paths (filesystem vs upload). Descriptions could clarify when to use each (e.g., 'Use anchor_ingest for server-side files; anchor_ingest_upload for client-side files'). Currently, LLMs must infer the distinction.