Loading ToolBench
Typed client and MCP server for the answerLoops Agent API, providing tools for semantic knowledge base search, FAQ retrieval, support ticket management, and grounded answer generation.
The server provides 8 well-named, verb-prefixed tools with structured input schemas and reasonable descriptions. However, there are significant gaps: output schemas are completely undocumented (neither in the code samples nor in tool definitions), error handling guidance is minimal, and several tools lack complete parameter documentation. Most tools fall into the 'fair-to-good' range with actionable descriptions but missing output specs and error recovery patterns.
Open a new support ticket on behalf of a user. Runs the same AI triage/answer pipeline as every other channel (Discord, Slack, email) — the ticket may get auto-answered if confidence is high, otherwise it queues for human review. Use this when search_kb and generate_answer don't resolve the question and a human needs to see it.
Generate a grounded answer to a question using the organization's knowledge base, without opening a ticket. Returns the answer plus a confidence score. Counts against the org's monthly deflection limit — if the limit is reached, returns an error instead of generating for free.
Get the organization's most recently generated FAQ digest — a markdown summary of the top questions resolved this week, grouped by category.
List support tickets for the organization, optionally filtered by status, priority, or category. Returns the most recent tickets first, plus a next_cursor to page through the rest — pass it back as `cursor` to keep going until next_cursor comes back null.
List files and directories in a GitHub repository path.
Output schemas completely undocumented for all 8 tools. LLMs cannot plan downstream tool calls or extract the right data without knowing what fields to expect.
Error handling completely absent. No error recovery guidance, no classification of errors as retryable/user-fixable/fatal, no actionable error messages documented.
Three code/GitHub tools (searchCode, readFile, listFiles) lack clear WHEN-to-use guidance and relationship documentation. LLM may conflate them or call wrong tool. Also, hardcoded per_page=5 in searchCode is unexplained and non-configurable.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | <=2025-11-25 | v2 |
Read the content of a file from a GitHub repository.
Search for code in a GitHub repository using the GitHub API search endpoint.
Semantically search the organization's knowledge base (published Q&A articles promoted from resolved support tickets). Use this before answering a question yourself — it's grounded in what this specific community/product has already answered.
Parameters 'repo' (owner/repo format) and 'orgId' (credential lookup) in GitHub tools lack explicit validation rules and examples. LLM must guess format.
get_tickets pagination is well-documented, but other tools (search_kb, get_faq, generate_answer) never mention pagination, result limits, or what happens with large datasets.
Tool composition relationships unclear. When should an LLM call search_kb vs generate_answer? When should generate_answer vs create_ticket be chained? Tool descriptions hint at workflows but don't document them explicitly.