Loading ToolBench
Model Context Protocol server for Railyard — read and write access to Railyard datacenter projects and organisations over stdio, authenticated with a personal access token.
Railyard MCP demonstrates solid tool design with consistent naming (verb_noun pattern), comprehensive parameter descriptions, and proper schema definitions. All 21 tools are explicitly registered with descriptions and input schemas. Tool names are action-oriented (list_, get_, create_, update_, delete_, rename_, move_). Parameter descriptions are detailed and include constraints (e.g., minLength, enum values, regex patterns). However, output schemas are not documented in the tool definitions, responses are returned as JSON text without formal schema declarations. Error handling is present but generic (RailyardApiError wrapper). Some parameter descriptions could be more concise (currently 50-150 chars, baseline 72 chars). Tool composition is excellent: each tool has a single responsibility, and chaining is well-supported (e.g., list_orgs returns org ids that feed into other tools). Security is strong: credentials injected via environment, no secrets in parameters.
Check whether a project name is free in an organisation (and see the URL slug it would get). Optionally `exclude` a project id so a rename that keeps its own name reads as available.
Create a new project in an organisation. Pass a name; the id and slug are generated. Optionally pass an inline `project` document to set initial structure (containers, racks, etc.); if omitted, a minimal empty project is created.
Delete a project. Pass the project's id or slug. This is irreversible.
Export a project (either a saved project named by `ref`, or an inline `project` document) to a target format (CSV, YAML, JSON, etc.). Returns one or more files and a summary. Export does not change anything.
Fetch an organisation's billing state: plan, status (trialing/active/past_due/canceled), seat count, trial end date, current period end, and whether the org is entitled to edit and the caller can manage billing. Requires owner role.
Output schemas not documented. Tool definitions register input schemas but do not declare expected response structures. LLMs cannot plan downstream operations or validate returned fields without documented output schemas.
No pagination parameters visible in list tools. list_orgs, list_projects, list_members, list_invitations lack limit/offset or cursor parameters. Large result sets could exceed context windows.
Destructive operations (delete_project) lack confirmation/dry-run support. No pattern to prevent accidental deletion via agent retry loops.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
Fetch a project's full JSON document and revision ETag, addressed by id or URL slug. The project contains the complete estate model, including the flexible container hierarchy, racks and placements, device roles, review dismissals, cabling and power. Pass `revision` to update_project when applying work derived from this response so concurrent changes are not overwritten.
Invite a new member to an organisation by email. Pass the email and the role (viewer/editor/owner). Requires owner role.
List the export targets this Railyard build supports — each format's id (what export_project takes), a one-line description, and the file extension it produces.
List pending invitations to an organisation (email, role, created date). Requires owner role.
List the members of an organisation (id, email, name, role, joined date).
List the organisations the token's user belongs to, with each org's id, slug, role (viewer/editor/owner), plan and billing status. The id or slug is what you pass as the `org` argument to other tools.
List projects in an organisation (id, name, slug, last-updated time). Omit `org` to use the default organisation.
Move a project to a different organisation. Pass the project's id or slug and the target org (id, slug, or name).
Remove a member from an organisation. Pass the member's email. Requires owner role. You cannot remove the last owner.
Rename a project. Pass the project's id or slug and the new name. The URL slug is regenerated from the new name.
Revoke a pending invitation. Pass the invitation id (from list_invitations). Requires owner role.
Report the Railyard server's health and capabilities: version, the project schemaVersion it understands, whether persistence (a database) and authentication are enabled, and whether Stripe billing is configured. Useful to confirm the server is reachable and which features exist.
Change a member's role in an organisation. Pass the member's email and the new role (viewer/editor/owner). Requires owner role.
Update a saved project's document. Pass the project's id or slug, the full updated document, and the revision ETag from get_project to prevent concurrent-change conflicts. Only the fields you pass are updated; omitted fields are left unchanged.
Run design-level validation on a project (either a saved project named by `ref`, or an inline `project` document). Returns problems (errors and warnings) and a summary. Validation does not change anything.
Return the Railyard user the configured personal access token authenticates as (id, email, name). Use this to confirm the token works and which account it belongs to.
Error messages are generic. RailyardApiError wrapping provides HTTP status and message, but no recovery guidance (e.g., 'Try list_orgs() to find valid org ids' when org lookup fails).
Tool annotations present but incomplete. readOnlyHint and destructiveHint are set, but idempotentHint is missing. Agents cannot determine which write operations are safe to retry.