Loading ToolBench
MCP server for NetLogo — create, run, and analyze agent-based models
Server has 21 tools spanning NetLogo modeling and market simulation. Tool definitions show deliberate structure with descriptions and parameters, but exhibit significant gaps: ~40% of tools lack complete input schemas visible in source, parameter descriptions are inconsistent (some detailed, others minimal), and output schemas are not documented. Naming is verb-first and clear (generate_audience, run_campaign, export_view). However, several tools combine multiple concerns (e.g., create_campaign includes stimulus creation and run specification), and error handling guidance is minimal. The codebase is well-organized and uses FastMCP properly, but the tool interface needs refinement for production LLM agents.
Fit heuristic cognition model parameters to real campaign data (CSV).
Execute a NetLogo command (e.g., 'setup', 'ask turtles [ forward 1 ]').
Create and save a campaign (one audience x one or more ad/email variants).
Create a new NetLogo model from a source string.
Export a plot as PNG or CSV.
Export the NetLogo world view as a PNG image.
Generate and save a synthetic audience from a YAML spec. The audience is a frozen, seeded population of personas plus a social network — the same spec always produces the same people, so campaign variants can be tested against an identical audience.
Output schemas not documented. Tools like get_campaign_report, get_audience, run_campaign, and interview_persona return complex objects (reports with metrics, persona data, event logs) but no output schema is visible in source code. LLMs cannot plan downstream operations without knowing what fields are available.
Generic parameter descriptions. Many tools accept free-form YAML/JSON strings (spec_yaml, campaign_yaml, spec_json) with minimal inline guidance on required vs optional fields, valid schemas, or examples. LLMs struggle to construct valid specs without detailed parameter descriptions.
Limited error handling guidance. Tools that modify state (generate_audience, create_campaign, run_campaign, calibrate) do not describe failure modes, recovery strategies, or what to do if a run hangs, a CSV is malformed, or a NetLogo model fails to load. Error responses like 'ToolError(str(exc))' provide no actionable recovery path.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 69 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 50 | - | v1 |
Show a saved audience: spec summary, archetypes, and sample persona cards.
Summarize the results of a completed campaign run.
Return metadata about the currently loaded model.
Conduct a multi-turn interview with a persona from a saved audience.
List saved synthetic audiences.
List saved campaigns.
List saved or example NetLogo models.
Download and load a model from CoMSES Net.
Load a NetLogo model file (absolute or relative to $NETLOGO_MODELS_DIR).
Execute a NetLogo reporter expression and return its value.
Run a saved campaign against its audience and log every reaction. Runs every variant x replicate with a PAIRED design (identical audience, reach and network randomness per replicate across variants), so A/B differences are attributable to the creative. Cognition: personas react via the configured backend — a live local LLM when SYNTH_LLM_MODE=live (Ollama etc.), otherwise the deterministic heuristic model.
Run a BehaviorSpace experiment against the loaded model.
Search CoMSES Net for published models.
Set a global variable or breed attribute.
No pagination or result limits documented. Tools like list_audiences, list_campaigns, list_models, and search_comses do not document maximum result counts, pagination parameters, or truncation behavior. Large result sets could exhaust LLM context.
Tool composition ambiguity. create_campaign accepts a campaign_yaml that includes both stimulus definitions AND run parameters (replicates, max_ticks, fidelity). This couples stimulus creation with experiment design, making it harder for agents to reuse a stimulus across multiple experiment configurations.
No explicit state management. Tools assume prior state (e.g., run_campaign expects a campaign_name to exist; interview_persona expects audience_name + persona_id to be valid). Errors on missing state are bare 'FileNotFoundError' with no suggestion of available alternatives.
No destructive operation confirmation. Tools like calibrate and run_campaign modify persisted state (save audiences, run experiments, fit parameters). No dry-run or confirmation mechanism to prevent accidental overwrites.