Loading ToolBench
Multi-component API gateway and control plane platform with federation, error tracking, and GitOps capabilities
This MCP server exposes 12 Makefile targets as tools, but suffers from fundamental definition quality issues. Tool names lack action verbs (e.g., 'seed-dev', 'demo-federation-setup'), parameter descriptions are minimal or absent in most cases, input schemas are either missing or severely incomplete, and error handling guidance is non-existent. The tools appear to be inferred from Makefile comments rather than explicitly defined with proper MCP tool registration. Output schemas are not documented. Most tools are infrastructure/demo commands that would be better exposed as resources or background jobs rather than callable tools.
Run 2-min live presenter demo (colorful output)
Start federation demo stack (Keycloak + LDAP + gateway + OPA)
Run federation isolation test (9/9 must pass)
Seed error snapshots into OpenSearch only
Verify OpenSearch error pipeline end-to-end
Migrate Kong config to STOA (KONG_SOURCE=kong.yaml STOA_OUTPUT=stoa/)
Seed dev data (full demo dataset) via Docker
Seed dev data locally (no Docker, requires DATABASE_URL)
Tool names lack action verbs. Names like 'seed-dev', 'demo-federation-setup', 'seed-staging' do not follow verb_noun pattern (get_, create_, update_, delete_, search_, list_, send_). LLMs cannot infer intent from the name alone and must rely entirely on descriptions.
Input schemas are missing or not visible for 7 of 12 tools (test-migrate-kong, test-migrate-kong-integration, demo-federation-setup, demo-federation-test, demo-federation-live, demo-opensearch-seed, demo-opensearch-test). Tools with empty input objects {} provide no parameter guidance and fail to constrain LLM behavior.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 34 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 67 | - | v1 |
Seed prod bootstrap (admin tenant + gateway) via Docker
Seed staging data (reduced realistic dataset) via Docker
Run Kong migration unit tests
Run Kong migration integration tests (requires Docker)
Parameter descriptions are minimal or generic. 'Seed profile (dev)' does not explain why the agent should use this tool over seed-staging or seed-prod, what data gets seeded, or the consequences. Descriptions lack WHAT, WHEN, and dependencies guidance.
Output schemas are not documented for any tool. LLMs do not know what fields to expect in responses, what IDs are returned for chaining, or whether tools are idempotent/safe to retry.
No error handling guidance. Tools lack descriptions of failure modes, recovery paths, or actionable error messages. An LLM calling seed-dev has no way to know whether a failed Docker build is retryable, whether it needs a service check, or whether it indicates a misconfigured DATABASE_URL.
Tool definitions appear to be inferred from Makefile comments rather than explicitly registered. No evidence of explicit MCP tool registration code visible in the provided source. Tool existence is inferred from Makefile targets and descriptions.
Parameter naming is inconsistent and ambiguous. 'SEED_PROFILE' (all caps, environment variable style) does not match MCP parameter conventions. 'from' and 'to' in migrate-kong are vague, 'from' could mean source file OR source system. No use of suffixes like _id, _name, _path to disambiguate parameter types.
No idempotency or confirmation patterns. Destructive operations like seed-prod (admin tenant + gateway setup) lack dry-run, confirmation, or rollback guidance. An agent could accidentally re-seed production without warning.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Agents cannot determine which tools are safe to retry and which have irreversible side effects. seed-dev, demo-federation-setup, and demo-opensearch-seed all modify state but lack destructiveHint.