Loading ToolBench
A FastMCP server demonstrating public-health surveillance data access with multiple operational modes: native tools, code-mode wrapping, hybrid execution, and skills-as-resources integration. Includes synthetic epidemiological data (cases, deaths, hospitalizations, vaccinations, outbreak alerts) across 50 regions and 4 countries.
Server has 12 tools with basic schemas and descriptions, but significant gaps limit production readiness. Nine surveillance tools (regions_list through neighbor_regions) have reasonable descriptions (50-200 chars) and typed parameters with enums for constrained inputs (disease, vaccine, severity). However, three critical issues: (1) execute tool accepts arbitrary Python code with minimal safety documentation, no mention of sandbox limits, timeout behavior, or error recovery; (2) create_acme_email and file_linear_ticket lack parameter validation guidance and error handling; (3) output schemas are inferred from code, not explicitly documented in tool definitions. No tool includes error recovery guidance, permission scopes, or audit trail documentation. The time_series tool's metric parameter uses a colon-delimited format ('cases:flu') that requires LLM parsing, should be split into separate parameters or use enums. Overall structure is present but lacks production-grade rigor.
Daily case counts, filterable by region, disease, and date range. Dates are ISO strings (YYYY-MM-DD). Returns one record per (day, region, disease). Diseases: flu, measles, covid19, rsv, pertussis, norovirus.
Provision a new @acme.example email address for an Acme employee. Use this when onboarding a new hire at Acme. Returns the provisioned email address and a temporary password.
Daily death counts, same filters as cases_query.
Run Python code with access to every other tool on this server via `await call_tool(name, params)`. Use this for multi-step work that would otherwise require many round-trips (per-item loops, aggregations, statistics). For simple single-call lookups, prefer calling the relevant tool directly. The code runs in a sandbox with `await call_tool(name, params)` injected in scope. Return the final value with `return ...` (the sandbox expects an explicit return).
File a ticket in Linear under the specified project.
execute tool lacks safety documentation and error recovery guidance. No mention of sandbox limits (15s timeout, 200MB memory), exception handling, or what happens on code errors. LLMs cannot determine if failures are retryable or fatal.
time_series metric parameter uses colon-delimited format ('cases:flu', 'vacc_doses:mmr') requiring LLM string parsing. Should split into separate parameters (metric_type enum + disease/vaccine param) or provide enum of valid metric strings.
Output schemas not explicitly documented in tool definitions. Descriptions state what tools return (e.g., 'Returns one record per (day, region, disease)') but lack formal schema with field types and required fields. LLMs cannot reliably extract nested data.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Daily hospital admissions + current inpatient counts, same filters.
IDs of regions that share a border with the given region.
Current outbreak alerts. Severities: low, moderate, high.
Full demographic profile for a single region.
List surveillance regions with basic demographics. Args: country: optional filter by country name.
Daily time series for a region and metric over the last N days. metric: 'cases:<disease>', 'deaths:<disease>', 'admissions:<disease>', 'vacc_doses:<vaccine>', 'coverage:<vaccine>'. Returns {"dates": [...], "values": [...]} with len == window_days.
Daily vaccination doses + cumulative coverage percent. Vaccines: flu_vax, mmr, covid_booster, pertussis_combo.
create_acme_email and file_linear_ticket lack error handling guidance. No mention of what happens if employee already exists, if Linear project is invalid, or if assignee cannot be found. LLMs cannot self-correct on failures.
No permission scopes declared for any tool. create_acme_email and file_linear_ticket are destructive (WRITE risk) but lack scope annotations (e.g., 'requires: admin:users, write:linear'). Audit trail and least-privilege configuration impossible.