Loading ToolBench
Kafka MCP Enterprise reference server for KIP-1318 - fail-closed security controls, stdlib-only (conformance / teaching; not the official ASF Java module).
Strong foundation with 13 well-named tools, comprehensive descriptions (avg 180 chars), and complete JSON schemas. All tools have explicit input schemas with type definitions and required fields. Descriptions clearly state side effects (read-only vs mutating vs destructive). However, output schemas are not documented, LLMs cannot predict response structure for planning. Parameter descriptions are present but some lack format/constraint details (e.g., 'config' object in alter_topic_config lacks guidance on valid keys). Error handling guidance is minimal. Security-sensitive tools (delete_topic, create_acls) reference _approval_token but lack recovery guidance if approval fails.
Reset committed offsets for a consumer group (`groupId` + `offsets` list of {topic, partition, offset}). Mutating. KIP Phase 1 tool; group should be idle. Use describe_consumer_group / kafka://groups/{id}/lag to inspect before reset.
Update dynamic config for an existing topic (`name` + `config` map). Mutating. Does not create or delete the topic. Use describe_topic to inspect current config; use create_topic only when the topic does not exist yet.
Read messages from a topic (bounded by maxMessages / hard_max_records). Omit groupId for Direct Partition Assignment (no consumer-group join/rebalance); set groupId only for classic group consume. Read path; may register values into session taint. Sensitive topics may require `_approval_token`. Prefer produce_message to write; prefer describe_topic for metadata only.
Create ACL bindings (resource, resourceType, operation, principal). Destructive authorization change; typically requires `_approval_token`. Binding resources must stay within topic/group scope prefixes. This reference does not list or delete ACLs; use broker-native admin for revoke/list outside this tool surface.
Create a topic with optional partitions, replicationFactor, and config. Mutating. Fails if the topic already exists or name is out of allowed prefixes / policy. Use alter_topic_config to change config later; use delete_topic to remove.
Output schemas not documented. LLMs cannot predict response structure (fields, types, pagination) for planning downstream calls or extracting data. Affects all 13 tools.
Parameter 'config' in alter_topic_config and create_topic lacks format guidance. LLMs cannot infer valid Kafka config keys (e.g., retention.ms, compression.type). Should document allowed keys or reference Kafka documentation.
Destructive tools (delete_topic, delete_consumer_group, create_acls) reference _approval_token but lack error recovery guidance. If approval fails, LLM has no guidance on next steps. Should document: 'If approval denied, contact admin at <email>' or similar.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2026-07-28+ | v2 |
Delete a consumer group and its committed offsets. Destructive; requires `_approval_token` when approval is enabled. Use list_consumer_groups to discover ids.
Permanently delete a topic and its data. Destructive; requires a valid HMAC `_approval_token` when approval is enabled. Irreversible for in-memory and broker data. Prefer alter_topic_config for non-destructive config changes.
Return cluster identity and broker/controller metadata. Read-only; no side effects. Use for health/context before mutating topics. Prefer list_topics for the topic catalog and kafka://health for circuit-breaker status.
Describe one consumer group (members, state, offsets) by `groupId`. Read-only. Use list_consumer_groups to discover ids. Does not change group membership.
Describe one topic by name: partitions, config, and metadata. Read-only. Requires `name`. Use list_topics first if the name is unknown. Does not create, alter, or delete the topic.
List consumer group ids. Read-only. Use describe_consumer_group for members, state, and offsets of one group. Does not join or leave groups.
List topic names known to the cluster. Read-only; no side effects. Use before describe_topic or produce/consume when you need the catalog. Prefer describe_topic for partitions/config of one topic.
Produce one message to a topic (`topic`, optional key/value/partition). Mutating write. Egress DLP may block secrets/PII in the value. Prefer consume_messages to read; does not create the topic if missing.
consume_messages parameter 'maxMessages' is optional but lacks default value guidance. Should state: 'Defaults to 10 if omitted; clamped to hard_max_records (e.g., 100).'
create_acls 'bindings' array parameter lacks item schema documentation. LLMs cannot infer required fields (resource, resourceType, operation, principal) or valid enum values for operation (READ, WRITE, ALTER, etc.).