Loading ToolBench
A diabetes risk prediction server using machine learning with SHAP-based explainability. Provides tools to predict diabetes risk and explain feature contributions using SHAP values, along with clinical guidelines and model information.
Two tools with complete input schemas and descriptions, but significant quality gaps. Both tools have verb-noun naming (diabetes_risk_explainer, diabetes_risk_predictor) which is good. Descriptions are present (194-250 chars) and exceed the 20-char minimum, but lack clarity on WHEN to use each tool and how they differ. Parameters have types (float) and descriptions, meeting baseline requirements. However, output schemas are not formally documented, the code shows dict returns but no structured schema definition. Error handling exists but is generic (catches Exception broadly). No tool annotations (readOnlyHint, idempotentHint) despite both being read-only operations. Resources are well-documented (2 resources with clear JSON structures), but tools themselves lack the precision expected for production use.
Generates a SHAP-based explanation for the diabetes risk prediction. This tool returns the contribution of each input feature (age, BMI, and diabetes pedigree function) to the model's prediction. It uses SHAP values to quantify how much each feature pushes the prediction toward or away from diabetes (class 1) versus no diabetes (class 0).
Predicts diabetes risk and explains feature contributions using SHAP. Args: age (float): Age of the individual in years. bmi (float): Body Mass Index (kg/m^2). diabetes_pedigree_function (float): Family history score. Returns: dict: { 'prediction': 0 or 1 (diabetes risk), 'probability': float (0–1), 'explanation': dict of feature-wise SHAP values showing contribution to the prediction. } Notes: - SHAP values indicate the contribution of each feature to the model's decision. They are not percentages. - LLMs or client apps must use these SHAP values for explanation — do not hallucinate.
Output schemas not formally documented. Both tools return dicts but no JSON Schema definition provided for LLM to understand response structure (fields, types, required keys).
Tool descriptions lack differentiation. Both tools mention SHAP explanations; unclear when to call explainer vs predictor. Descriptions should state: explainer returns only SHAP values; predictor returns prediction + probability + explanation if positive.
No tool annotations despite both being read-only, idempotent operations. Missing readOnlyHint and idempotentHint annotations that would signal to agents these are safe to retry.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Error handling is overly broad. Both tools catch generic Exception and return inconsistent error structures (one returns 'error' + 'details', other returns 'explanation' + 'note'). Should validate inputs early and return structured errors with actionable guidance.
Parameter descriptions lack constraints. 'age' and 'bmi' have no stated ranges (e.g., age 0-150, bmi 10-60). LLMs may pass invalid values; tool should validate and return 'Invalid age: must be 0-150' rather than silent failure.