Loading ToolBench
An MCP server for managing agent-based simulations with Lua scripting, entity lifecycle management, message passing, and metrics tracking
Vivarium provides 15 tools with consistent naming conventions (all verb-noun style: load_project, initialize_project, etc.) and present descriptions. However, critical gaps in parameter validation, output schema documentation, and error handling reduce quality below production baseline. All tools follow the verb_noun pattern and have descriptions in the 50-200 character range (good). Input schemas are visible in the code and use proper JSON types. The main issues are: (1) no visible output schema documentation for any tool, responses are implied but not declared; (2) parameter descriptions exist but lack constraint information (ranges, enums, valid formats); (3) error messages in code are generic or missing actionable recovery steps; (4) no tool annotations (destructiveHint, readOnlyHint, idempotentHint) despite several tools being clearly destructive or read-only; (5) parameter relationships undocumented (e.g., snapshot parameter in load_project accepts 'latest' or a name, but no enum constraint visible). The code shows solid semantic design, tools are well-scoped and compose logically, but the MCP interface definition itself lacks the rigor expected of production agent tools.
Advance the simulation by running multiple time steps. Each step processes pending messages and executes entity update() functions. Use step_duration to control simulation time granularity.
Get the current state of a specific entity by its ID.
Get the current values of a specific metric by name.
Get the current values of multiple metrics by their names.
Get the overall state of the loaded project, including simulation time, entity count, and pending message count.
Initialize a new Vivarium project folder with starter scripts and an initial snapshot
No output schemas documented for any tool. Tool definitions lack return type descriptions, forcing LLMs to infer result structure from generic response types (e.g., LoadProjectResponse). This breaks tool chaining and downstream parameter mapping.
No tool annotations (destructiveHint, readOnlyHint, idempotentHint) present. Tools like load_project, advance_simulation, set_entity_state, save_project_snapshot, and load_project_snapshot are clearly destructive or write operations but lack explicit annotation. This forces LLMs to guess which tools are safe to retry.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
List all entities currently in the simulation. Returns their IDs which can be used as targets for sending messages.
List the names of all available metrics in the loaded project.
List available snapshot names for a loaded project
List all loaded projects
Load a project from its manifest and snapshot into the in-memory runtime
Load a snapshot into a loaded project
Save a snapshot for a loaded project
Set the state of a specific entity by its ID. The state must be a JSON object compatible with the entity's Lua script.
Unload a loaded project from memory by name
Parameter descriptions lack constraint information. Numeric params (steps, step_duration) have no min/max bounds. String params (snapshot, entity_id, metric_name) lack format constraints or enums. The snapshot param in load_project accepts 'latest' or a name, but this is documented in prose, not as an enum constraint.
Error handling in code is generic. Examples: 'Failed to unload project' (no actionable recovery), 'Invalid snapshot selection' (does not suggest valid options or how to list them). Errors should suggest next steps, e.g., 'Snapshot not found. Call list_project_snapshots(project_name) to see available snapshots.'
Missing pagination support. Tools like list_projects, list_entities, list_metrics, and list_project_snapshots have no documented limit, offset, or cursor parameters. If a project has many entities or snapshots, responses could exceed context limits.
Parameter relationships undocumented. For example, set_entity_state accepts an entity_id and state object, but the description does not explain what fields are valid in state or how they map to the entity's Lua script. This forces trial-and-error or requires the LLM to inspect script files.
No idempotency guarantees stated. Tools like load_project accept a replace_if_loaded flag, but idempotency of repeated calls is not documented. Agents may retry on transient failures and risk unintended state overwrites.