Loading ToolBench
MCP server that compresses, summarizes, and filters context to minimize LLM token usage
Compendium has 17 tools with complete input schemas and descriptions, but quality is uneven. Most tools have clear, action-oriented names (filter, compress, summarize, chunk, resolve, cache_store, cache_get, cache_invalidate, count_tokens, stats, llm_status, help) and non-trivial descriptions (50 - 150 chars). However, parameter descriptions are sparse or generic. For example, 'filter' has a 'query' param described only as 'Optional query for BM25-relevant line filtering', no guidance on format or expected content. 'compress' lacks detail on what 'force' actually does beyond 'Force compression even for small inputs'. Output schemas are not documented in the tool definitions, LLMs cannot predict what fields to expect. Error handling is minimal; no recovery guidance or actionable error messages visible. The 'smart' parameter pattern (summarize_smart, filter_relevant) is underdocumented, unclear when to use smart vs non-smart variants. Composition is reasonable (each tool has one job), but chaining is not obvious from descriptions.
Retrieve cached text / chunk by key
Drop one cache key or clear the session cache
Park a bulky payload outside the prompt
Split corpus into cmp:// chunks
Dense representation of text/code/logs
Domain-aware stdout/stderr scrub (git/cargo/npm/docker/…)
Count tokens with the active backend
Output schemas not documented. LLMs cannot predict return structure (fields, types, pagination). Tools like 'summarize', 'rerank', 'chunk' return complex objects but no schema is visible in definitions.
Parameter descriptions lack actionable detail. 'query' param in filter/rerank/filter_relevant described only as 'Query to...' with no guidance on format, length, or expected content. LLMs cannot infer constraints.
Undocumented parameter relationships. 'summarize_smart' has both 'smart' and 'summarize' objects; unclear which takes precedence or when to use each. 'rerank' accepts 'items', 'text', or 'map', mutually exclusive but not stated.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | <=2025-11-25 | v2 |
Strip ANSI/boilerplate; densify JSON; keep/drop regexes
Query-aware keep of relevant lines
List actions, or usage + example for one action
Probe the local LLM URL / models
BM25 rank candidates for a query
Fetch chunk content by id
Redact secrets + neutralize prompt-injection phrases
Session token savings per action
Hierarchical summary (conversation / file tree / outline)
Local-SLM dense summary (heuristic fallback)
No error handling guidance. Tools return errors but no recovery hints (e.g., 'LLM unavailable, try fallback=true' or 'Invalid chunk id, call chunk() first'). Agents cannot self-correct.
Smart vs non-smart tool variants confuse composition. 'summarize' and 'summarize_smart' both exist; LLMs must reason about when to use each. No guidance on fallback behavior or when smart mode is unavailable.