Loading ToolBench
A collection of MCP server implementations for educational purposes, including Weather, Travel Booking, and Knowledge Base servers
This is a teaching workshop with 6 tools across multiple exercise files. Overall definition quality is below average. Most tools have schemas and basic descriptions, but descriptions are often generic or incomplete. Parameter validation is minimal. Error handling is weak. Naming is generally acceptable (verb-noun pattern), but several descriptions lack specificity about when/why to use each tool. The codebase shows incomplete implementations (exercise files with TODO comments, syntax errors in travel_server.py exercise version), which indicates this is an in-progress tutorial rather than production-ready code. Tool definitions visible in solution files are better than exercise stubs.
Book transportation for a trip. Args: booking_id: The trip booking ID to link transportation to transport_type: Type of transport (flight, train, bus, car) departure: Departure location arrival: Arrival location departure_time: Departure date and time in YYYY-MM-DD HH:MM format Returns: Transportation booking confirmation details
Book a trip and save the booking details to a file. Args: traveler_name: Full name of the traveler destination: Destination city/country start_date: Trip start date in YYYY-MM-DD format end_date: Trip end date in YYYY-MM-DD format budget: Total budget in USD Returns: Booking confirmation details including booking ID
Retrieve the entire knowledge base as a formatted string. Returns: A formatted string containing all Q&A pairs from the knowledge base.
Gets current weather information for the given coordinates using Open-Meteo API. Args: latitude: The latitude coordinate longitude: The longitude coordinate Returns: Dictionary containing weather information
Returns a simple greeting.
Generic and incomplete descriptions. 'greet' description is only 'Returns a simple greeting.' (26 chars, lacks context for when/why to use); 'get_knowledge_base' lacks any detail about what the knowledge base contains or format of returned data.
Missing parameter descriptions in some tools. 'greet' tool has a 'name' parameter with only 'Name to greet', no indication of format, length, or constraints. 'get_knowledge_base' has no parameters but no description of what the knowledge base structure contains.
Incomplete/broken implementations in exercise files. travel_server.py exercise version has syntax errors (missing value for 'recommendations' variable, incomplete book_trip function) and TODO comments. These tools cannot actually be invoked.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 45 | - | v1 |
Recommend a trip based on destination, budget, and duration. Args: destination: The destination city/country for the trip budget: The total budget in USD duration_days: Number of days for the trip Returns: A formatted string with trip recommendations
No input validation or error handling. Tools catch generic exceptions and return error dicts, but provide no guidance on retry, user-fixable errors, or specific invalid values. Example: get_weather returns {"error": "..."} with no hint on what went wrong or what to do next.
No output schemas documented. Tools return dicts, strings, or complex objects, but no schema definitions appear in docstrings or tool registration. LLMs cannot know what fields to expect.
Ambiguous parameter names and missing enum constraints. 'transport_type' in book_transportation accepts free-form strings but description says '(flight, train, bus, car)', should be an enum, not a suggestion. No validation prevents 'airplane' or 'carriage'.
No confirmation or dry-run pattern for destructive operations. book_trip and book_transportation write to files but provide no way to preview or confirm before execution. Agents could accidentally book unwanted trips.
Tool descriptions in parameter docstrings use example values (e.g. 'YYYY-MM-DD HH:MM format' for departure_time) but do not enforce via regex pattern. LLMs may pass '2024/01/15 3pm' instead.