Loading ToolBench
WPILib RAG system as MCP server with version and language-specific documentation retrieval
5 tools with generally clear names and reasonable descriptions, but significant gaps in parameter descriptions, output schemas, and error handling. Most tools lack documented return types. Parameters exist but many lack detailed descriptions. No tool annotations (readOnlyHint, destructiveHint). Error handling is minimal, no recovery guidance or actionable error messages. The RAG-focused tools (query_wpilib_docs, embed_query) have decent descriptions but weak parameter docs. Utility tools (list_available_versions, get_latest_version) are sparse. No pagination despite list tools. No documented output schemas. This is typical C-grade community work, functional but not production-ready.
Generate embedding vector for a query using Voyage AI. Returns the embedding vector that can be used for client-side processing. This is useful for caching embeddings or performing client-side similarity searches.
Return the latest WPILib version (defaults to '2025' if database empty)
List available languages for a specific version (or all languages)
List all available WPILib versions in the database
Retrieve relevant WPILib documentation chunks for the specified version and language. Returns formatted documentation chunks with citations that you can use to answer questions. This tool performs retrieval only - you generate the answer from the retrieved chunks. If unsure, use version '2025'.
No documented output schemas for any tool. LLMs cannot plan downstream calls or extract expected fields without knowing return structure. E.g., query_wpilib_docs returns 'formatted documentation chunks with citations' but the actual JSON structure is never defined.
Minimal parameter descriptions for utility tools. get_latest_version and list_available_versions have empty property dicts, no guidance on what they return or when to use them. list_available_languages has optional 'version' param but no description of what happens when omitted.
No error handling guidance in any tool. Descriptions do not explain failure modes, recovery steps, or what the LLM should do if a call fails. E.g., 'query_wpilib_docs' could fail if the version is invalid, the database is empty, or the Voyage AI API is unreachable, none of these are documented.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 42 | - | v1 |
No tool annotations. None of the tools declare themselves as read-only (readOnlyHint=true), destructive, or idempotent. This prevents agent frameworks from applying safety guardrails or optimizing request ordering.
list_available_versions and list_available_languages have no pagination or limit parameters despite being list tools. If there are hundreds of versions or languages, the entire response bloats the context window. No documented limit or pagination guidance.
The 'version' parameter in query_wpilib_docs is a free-form string with no documented format, range, or fallback. Code comment says 'Available versions: 2025' but schema enum is built dynamically from the database (or empty if DB is absent). LLMs have no static reference for valid values and may hallucinate version numbers.
embed_query description mentions 'Returns the embedding vector that can be used for client-side processing' but does not document the vector format (array of floats? dimensions? range?). LLMs cannot use a tool if they don't know the output shape.