Kafka Cluster with Strimzi¶

Strimzi provides a way to run Apache Kafka on OpenShift via the Strimzi Cluster Operator. It manages Kafka clusters, topics, users, and connectors using custom resources.

Note

This page uses the community Strimzi operator from community-operators. It is not Red Hat Streams for Apache Kafka. Use Red Hat Streams for Apache Kafka when the customer needs a supported Kafka operator.

Prerequisites¶

Cluster-admin access (for installing the operator)
Sufficient cluster resources (minimum 3 broker nodes recommended for production)

Install the Strimzi Operator¶

Create a namespace for the Kafka cluster:
```
oc new-project kafka
```

Install Strimzi from the Software Catalog:

cat <<EOF | oc apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: strimzi-kafka-operator
  namespace: openshift-operators
spec:
  channel: stable
  name: strimzi-kafka-operator
  source: community-operators
  sourceNamespace: openshift-marketplace
EOF

Wait for the operator to be ready:
```
oc get csv -n openshift-operators | grep strimzi
```
The phase should show Succeeded.

Deploy a Kafka Cluster¶

Strimzi now requires KRaft (no ZooKeeper). Create a dual-role node pool and a Kafka cluster:

Apply the Kafka cluster manifest:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: dual-role
  namespace: kafka
  labels:
    strimzi.io/cluster: my-cluster
spec:
  replicas: 3
  roles:
    - controller
    - broker
  storage:
    type: jbod
    volumes:
      - id: 0
        type: persistent-claim
        size: 10Gi
        kraftMetadata: shared
        deleteClaim: false
---
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
  namespace: kafka
  annotations:
    strimzi.io/node-pools: enabled
    strimzi.io/kraft: enabled
spec:
  kafka:
    version: 4.2.0
    metadataVersion: 4.2-IV0
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      default.replication.factor: 3
      min.insync.replicas: 2
    resources:
      requests:
        memory: "2Gi"
        cpu: "500m"
      limits:
        memory: "4Gi"
        cpu: "1"
  entityOperator:
    topicOperator: {}
    userOperator: {}

oc apply -f kafka-cluster.yaml

Wait for the cluster to become ready:

oc wait kafka/my-cluster --for=condition=Ready --timeout=300s -n kafka

Verify all pods are running:
```
oc get pods -n kafka
```

You should see pods for the dual-role brokers and the Entity Operator.

Create a Topic¶

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
  name: my-topic
  namespace: kafka
  labels:
    strimzi.io/cluster: my-cluster
spec:
  partitions: 3
  replicas: 3
  config:
    retention.ms: 604800000

oc apply -f kafka-topic.yaml

Test with a Producer and Consumer¶

Start a console producer:

oc run kafka-producer -ti \
  --image=quay.io/strimzi/kafka:latest-kafka-4.2.0 \
  --rm=true --restart=Never \
  -- bin/kafka-console-producer.sh \
  --bootstrap-server my-cluster-kafka-bootstrap:9092 \
  --topic my-topic

In a separate terminal, start a console consumer:

oc run kafka-consumer -ti \
  --image=quay.io/strimzi/kafka:latest-kafka-4.2.0 \
  --rm=true --restart=Never \
  -- bin/kafka-console-consumer.sh \
  --bootstrap-server my-cluster-kafka-bootstrap:9092 \
  --topic my-topic \
  --from-beginning

Expose Kafka Outside the Cluster (Optional)¶

To access Kafka from outside OpenShift, add a Route listener to the existing Kafka CR. Do not apply the snippet below as a standalone resource — it would replace your working cluster.

oc patch kafka my-cluster -n kafka --type merge -p '
spec:
  kafka:
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        type: route
        tls: true
'

Retrieve the bootstrap address:

oc get kafka my-cluster -n kafka -o jsonpath='{.status.listeners[?(@.name=="external")].bootstrapServers}'

Cleanup¶

oc delete kafka my-cluster -n kafka
oc delete subscription strimzi-kafka-operator -n openshift-operators
oc delete project kafka