Loading ToolBench
This repository contains multiple MCP server implementations and client integrations for Klavis AI, including servers for 12306 ticket search, Affinity CRM, Airtable, and various LLM framework integrations (OpenAI, Claude, LangChain, LlamaIndex, Mastra, CrewAI, etc.)
Single tool 'search_tickets' has a bare-minimum definition. Tool name is verb-based (positive), but description is minimal (58 chars, below rubric baseline of 194), lacks context about when to use it vs alternatives, and omits key operational details like return structure, rate limits, or error guidance. Input schema is present with basic types and descriptions, but lacks constraints (no enums, no format specs for the date parameter). No output schema is visible in the provided source. The server provides STDIO transport only, capping protocol readiness severely. Overall definition quality is poor, barely functional, with missing operational guidance and no documented output structure.
Search for 12306 railway tickets based on route and date parameters
Missing output schema documentation. No documented return structure, field types, or pagination guidance. LLM cannot reliably parse results or chain downstream calls.
Tool description severely under baseline (58 chars vs 194 char average). Lacks WHEN to use, prerequisites, rate limits, error conditions, and operational context. Insufficient for LLM selection and planning.
Date parameter accepts free-form string with no format validation, enum, or pattern constraint. Description says 'YYYY-MM-DD' but LLM may pass invalid dates (e.g., '2024-13-45'). Requires client-side validation.
Station parameters (from_station, to_station) accept arbitrary strings. No enum of valid stations provided, no lookup tool to resolve ambiguous names (e.g., 'Beijing' vs 'Beijing South'). LLM has no way to find correct station codes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No pagination, result limit, or output cardinality guidance. A search could return 0 - 1000+ tickets; no indication of how many are returned, whether results are capped, or how to iterate if results exceed limit.
No error handling documentation. No guidance on what happens if stations are invalid, date is in past, or API is rate-limited. Error messages will be opaque to LLM.