Loading ToolBench
codewiki-mcp demonstrates solid definition quality with three well-named, read-only tools backed by proper JSON Schema. All tools have clear, actionable descriptions (136-186 chars, within the 10-1024 baseline range). Input schemas are properly typed with constraints. However, output schemas are not documented, the code does not show what fields/structures are returned, preventing LLMs from planning downstream calls. Tool annotations (readOnlyHint) are absent, though risk classification is correct. Error handling lacks actionable recovery guidance, defaulting to generic API errors. Parameter descriptions are good but could be more explicit about format constraints (e.g., what constitutes a valid repo identifier). The tool composition is excellent, each tool has a single responsibility, and the search→fetch→ask chain is logical. No security concerns (no secrets in params, all READ_ONLY). Named parameters like 'repo' and 'question' match the chat data model well, though 'repo' could be more explicit about accepting 'owner/repo', URLs, or natural language (this is mentioned in the description but could be a formal constraint).
Ask a natural-language question about a repository indexed in codewiki.google
Fetch generated wiki content for a repository from codewiki.google
Search repositories indexed by codewiki.google
Output schemas not documented. None of the three tools document what they return. Code inspection does not reveal return type definitions. LLMs cannot plan downstream calls or know what data is available (e.g., after search_repos, do I get repo_url or github_url? Can I pass it directly to fetch_repo?). This forces agents to guess or make exploratory calls.
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are absent. All three tools are READ_ONLY and idempotent, but the schema does not declare this via tool annotations. Clients cannot auto-categorize these as safe tools without reading descriptions.
Error handling lacks actionable recovery guidance. If a repo is not found or the API is down, the tool likely returns a generic error. No documented error cases or recovery steps (e.g., 'Repository not found. Try search_repos() to find similar repositories'). Agents cannot self-correct from failures.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 67 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
'repo' parameter could be more explicit about valid formats. Description says it accepts 'owner/repo, URL, or natural language' but does not define a pattern or format formally. An enum or regex would reduce ambiguity and let LLMs construct valid identifiers.
No pagination support on search_repos. Even though limit defaults to 10 and caps at 50, there is no way to retrieve the next page (no cursor, offset, or total_count in the documented response). Large result sets may be truncated without the LLM knowing.