Loading ToolBench
MCP vs Traditional Function Calling Benchmarking Study - Agricultural advisory system for field monitoring, irrigation management, crop scheduling, and soil analysis
AgriAdvisor has 14 tools with basic structure and reasonable domain coherence, but exhibits significant quality gaps in parameter descriptions, output schema documentation, and error handling. Naming is generally clear and action-oriented (get_*, calculate_*, activate_*, log_*), but multiple tools lack depth in their descriptions and parameter annotations. The server demonstrates awareness of security best practices (PUMP_API_KEY injection in activate_irrigation), but this is undermined by incomplete schema documentation across most tools. Average tool description is ~110 characters, below the 194-char production baseline. Only 4 of 14 tools have fully described parameters; the remainder have missing or minimal parameter descriptions. No tools document output schemas explicitly. Error messages exist but lack recovery guidance in most cases.
Activate or deactivate the irrigation pump for a field. The LLM calls this with field_id and action only. The secret PUMP_API_KEY is resolved server-side and never exposed to the LLM. This demonstrates MCP's security model: API keys stay on the server.
Add two numbers.
Calculate the area of a rectangular field in acres and square feet.
Calculate specific fertilizer requirements based on soil test and yield goals
Divide a by b. Returns an error if b is zero.
Return the planting and harvest schedule for a given crop.
Three tools (get_detailed_soil_analysis, calculate_fertilizer_requirements, get_irrigation_system_status) have NO visible input schemas or parameter descriptions in the provided source code. Only names and brief descriptions are shown from server_professional.py, but implementation is not included.
Arithmetic tools (add, subtract, multiply) have extremely minimal descriptions ('Add two numbers', 'Subtract b from a', 'Multiply two numbers'). These are 10-24 characters, well below the 194-char production baseline. No context provided for WHEN an LLM should call these domain-irrelevant math tools instead of doing arithmetic in-context.
No tool documents its output schema. Code shows return statements (e.g., activate_irrigation returns dict with 'success', 'field_id', 'action', etc.), but no schema or output documentation is provided in docstrings. LLMs cannot reliably extract required fields or plan downstream tool calls without knowing expected output structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 30 | - | v1 |
Get comprehensive soil analysis including chemical, physical, and biological properties
Provides the current moisture and health status of a specific field.
Get current status of irrigation equipment and recent activity
Get a simulated weather forecast for a farm location (1–5 days).
Log a sensor reading (moisture, temperature, pH, etc.) for a field.
Multiply two numbers.
Recommend an irrigation action based on current soil moisture level.
Subtract b from a.
Error handling lacks recovery guidance. Error returns are present (e.g., 'Field not found', 'Division by zero'), but do not tell the LLM what to do next. Should include actionable recovery steps like 'Try search_fields() with a partial name' or 'Pass a non-zero divisor'.
Tool composition includes arithmetic tools (add, subtract, multiply, divide) that are out of scope for an agricultural domain server. These suggest the server is a benchmark/test harness rather than a production-grade AgriAdvisor. Mixing unrelated domains reduces clarity and wastes tool slots.
Parameter 'duration_minutes' in activate_irrigation has a default (30) that applies only when action='start', but the description does not explain this conditional behavior. LLMs may pass duration_minutes when calling with action='stop', expecting it to be honored.
Field identifiers are hardcoded in enums (Field_A, Field_B, Field_C, Field_D) with no discovery mechanism. If fields are dynamic, the server should offer a list_fields() tool. If static, the hardcoded set is undocumented for the LLM.
Resources feature is declared (resources=true) but implementation is incomplete in the provided code. Only a stub for get_safety_policy is shown. No other resources are documented, and their schema/output format is not specified.