Loading ToolBench
Python SDK and CLI for the ctxd platform. Search and fetch content from ctxd-connected apps, manage CLI authentication, and inspect indexed data.
Three read-only tools with basic descriptions and minimal input schemas. Tool names follow verb_noun convention (search, fetch_document, get_profile) which is good. However, descriptions are sparse (15-59 characters), lacking context for LLM selection. Schemas are minimal, only 1-2 simple string parameters with descriptions. No output schemas documented. Error handling exists in code (CtxdError, CtxdAuthError, CtxdProtocolError) but tool responses don't include recovery guidance. No parameter constraints (enums, ranges). All tools are READ_ONLY which is a security positive. Tool naming is clear but descriptions need expansion to reach A/B territory.
Fetch one indexed document by document UID
Show connected integrations and indexed file tree for the authenticated user
Search indexed app content using ctxd DSL
Minimal tool descriptions lack LLM-optimization guidance. 'Search indexed app content using ctxd DSL' (59 chars) doesn't explain WHEN to use search vs fetch_document, what 'DSL tokens' are, or what a typical response looks like. Production baseline: 194 chars avg, with WHAT/WHEN/HOW context.
No output schemas documented. async_client.py shows SearchResult, DocumentResult, ProfileResult are model_validated, but tool definitions returned to the MCP client don't include 'outputSchema'. LLMs cannot plan downstream tool calls or extract required fields without knowing the structure.
Parameter constraints missing. 'query' in search tool has no description of valid DSL syntax, length limits, or examples of operators. Documentation refers to 'ctxd DSL' but doesn't explain it inline. LLMs will hallucinate invalid queries.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Error responses in code (CtxdError, CtxdProtocolError) are generic and don't guide LLM recovery. 'MCP SSE response did not contain a data line' is a protocol error, should suggest retrying or checking API_KEY. Tool definitions lack error classification (retryable/user-fixable/fatal).
No tool composition documentation. search() returns document_uid which fetch_document() accepts, good chain. But tool descriptions don't hint at this workflow. LLMs won't infer 'search first, then fetch_document' without explicit guidance.