MCP server that exposes Zulip REST API capabilities as tools for LLMs
Scoring was not performed
Missing output/response schemas. Tool definitions declare input schemas but do not document what fields or structure the API returns. This forces LLMs to reason blind about what data is available for downstream calls (e.g., does get_messages return message_ids for follow-up edits? Does list_streams return stream_ids for get_stream_topics?).
Descriptions lack LLM-specific guidance. Many descriptions simply state what the tool does (e.g., 'Retrieve messages from Zulip') but do NOT explain WHEN to use it vs similar tools (get_messages vs get_message?), what edge cases exist, or dependencies (e.g., 'Call list_streams first if you only have a channel name'). LLMs need explicit decision guidance.
Parameter descriptions lack actionable constraints. For example, 'anchor' in get_messages says it accepts 'newest', 'oldest', 'first_unread' or a message ID, but does not explain when each is appropriate. 'narrow' accepts complex filter syntax but describes it as an array without showing valid examples or format. LLMs cannot parse implicit rules.
Mutually exclusive parameters not documented. get_messages can use 'anchor' + num_before/num_after OR 'message_id' alone, but the schema does not make this explicit. LLMs may pass both, leading to ambiguous API behavior or errors. create_draft uses 'to' as an array of IDs for private messages but a single channel ID for streams, this overloading is not clearly flagged.
No error handling or recovery guidance visible in tool definitions. Tools like delete_message, delete_draft, delete_scheduled_message are marked DESTRUCTIVE but include no confirmation, dry-run option, or recovery guidance. LLM callers have no way to know what to do if a delete fails or how to prevent accidental data loss.
No pagination guidance or limits documented. Tools returning lists (list_users, list_streams, get_drafts, get_scheduled_messages) accept no limit or page parameters. The descriptions do not state how many items are returned or if there is a cap. This risks blowing the context window with very large result sets.
Inconsistent identifier handling. Some tools accept human-readable names (e.g., send_message takes channel name 'general'), others require IDs (e.g., get_stream_topics requires stream_id). This forces agents to do unnecessary lookups. Design should be consistent: either all accept names+IDs or all require IDs with a separate discovery tool.
Missing tool annotations. None of the tools declare their read-only or destructive nature within the MCP schema (readOnlyHint, destructiveHint). While the Risk field in the rubric notes marks them (READ_ONLY, WRITE, DESTRUCTIVE), this is not exposed via MCP tool annotations that would help clients surface warnings or disable undo.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | F | 14 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 46 | - | v1 |