Documentation search server that searches and extracts content from LangChain, LlamaIndex, and OpenAI documentation
Single tool server with a reasonable action verb name ('get_docs') and an informative docstring. However, critical gaps undermine production readiness: (1) Input parameter descriptions exist but are minimal (~30-40 chars each), falling short of the baseline 72 chars; (2) No output schema documentation, the tool returns either a string or an error, but the response structure is not formally described for LLMs to plan downstream actions; (3) The parameter 'library' accepts one of three known values ('langchain', 'openai', 'llama-index') but is not declared as an enum, relying instead on runtime validation that throws ValueError, this violates the constrained-input pattern and forces LLMs to guess valid options; (4) No error guidance in the tool description, when search_web times out or returns zero results, the tool returns generic strings ('No results found', 'Timeout error') that give agents no recovery path; (5) No explicit output schema in the FastMCP registration, meaning LLMs cannot anticipate the response structure; (6) Tool description is 335 characters, which is within baseline (p90=392) but could be more specific about when/why to use it and what to do with the results.
Search the documentation for a given query within a specific library. This tool uses web search constrained to the documentation domain of the selected library and extracts visible textual content from the top results.
Input parameter 'library' accepts only 3 values ('langchain', 'openai', 'llama-index') but is NOT declared as an enum in the schema. Instead, validation happens at runtime and throws ValueError. This forces LLMs to hallucinate valid values and wastes tokens on failed calls.
Output schema is completely undocumented. The tool returns 'str | None' in practice (extracted text, error messages, or None on timeout), but FastMCP registration has no explicit output schema annotation. LLMs cannot predict what fields to expect or how to chain this tool to others.
Error messages are generic and provide no recovery guidance. When no results are found, the tool returns 'No results found', the LLM does not know whether to retry, try a different query, or ask the user. When a timeout occurs, 'Timeout error' is returned with no suggestion to retry or fail gracefully.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 43 | - | v1 |
Parameter descriptions are minimal (26 - 40 characters) and lack context for LLM selection. The 'query' param says 'The documentation query (e.g., 'Chroma DB')', but example values in descriptions encourage LLMs to reuse them literally. Should replace with a constraint-based description and no examples.
No tool annotation hints (readOnlyHint, destructiveHint, idempotentHint). While this tool is read-only, explicit readOnlyHint annotation would inform clients that it is safe to retry and does not modify state, improving agent planning.