Loading ToolBench
Kagi MCP server using session-based access (no official API required)
Server defines 2 tools with good naming (verb_noun pattern) and reasonably detailed descriptions. Both tools have proper input schemas with types and parameter descriptions. However, there are significant gaps: no output schema documentation, missing error handling guidance, no parameter validation rules in descriptions, and limited actionable error messages. The descriptions are solid (180-250 chars) but lack the precision needed for LLM self-correction (e.g., no constraint rules for language codes, engine names, URL format). Per-tool analysis follows.
Fetch web results based on one or more queries using Kagi Search. Use for general search and when the user explicitly tells you to 'fetch' results/information. Results are from all queries given. They are numbered continuously, so that a user may be able to refer to a result by a specific number.
Summarize content from a URL using the Kagi Summarizer. The Summarizer can summarize any document type (text webpage, video, audio, etc.) Note: This tool uses Kagi's internal summarizer endpoint accessed via session token. This is experimental and may break if Kagi changes their internal API.
No documented output schema for either tool. Tools return formatted strings, but LLMs cannot predict structure, field names, or how to extract data for downstream composition.
Parameter descriptions lack constraint rules. 'limit' parameter has no min/max bounds; 'target_language' has no validation rule (e.g., ISO 639-1 codes); 'engine' enum values documented but no guidance on behavior differences.
Error handling is minimal. Code raises ValueError and RuntimeError, but descriptions do not explain recovery paths (e.g., 'If search fails due to rate limit, wait X seconds before retrying' or 'Session token expired, check config'). No guidance for LLM on next steps.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | <=2025-11-25 | v2 |
kagi_summarizer description warns 'endpoint may change without notice' but provides no fallback strategy or health check. If the endpoint breaks, the tool silently fails without actionable guidance.
kagi_search_fetch accepts 'limit' but description says 'Default: all available', no numeric bound stated. LLMs may pass absurdly large limits (e.g., 10000) that time out or exhaust memory.
kagi_summarizer 'target_language' param accepts any string with no format rule. Description says 'e.g., EN' but does not specify ISO 639-1 vs 639-2 vs full names. LLMs will guess (e.g., passing 'English' instead of 'EN').