Networking¶
OpenShift requires specific network configurations and firewall rules between nodes and external services. All node networking uses static IP assignments — there is no DHCP in this environment.
Network Requirements¶
| Network | Purpose | CIDR Example |
|---|---|---|
| Machine Network | Node-to-node communication | 10.0.0.0/28 |
| Pod Network | Pod-to-pod communication | 10.128.0.0/14 |
| Service Network | Kubernetes service IPs | 172.30.0.0/16 |
Machine Network Isolation
In a production environment, it is best to run the machine network in an isolated network in a separate vlan than your workloads. If you are planning to run OpenShift Virtualization, those VMs should run on a different vlan than the machine network.
Info
Do you have to use these values for the Pod network and Service network? No. But for the POC, just keep the defaults.
Static IP Assignments¶
Since there is no DHCP, every node requires a pre-assigned static IP. Gather the following for each node before installation:
| Field | Example |
|---|---|
| IP Address | 10.0.0.7 |
| Subnet Mask | 255.255.255.240 |
| Gateway | 10.0.0.1 |
| Primary DNS | 10.0.0.2 |
| Secondary DNS | 1.1.1.1 |
| NIC Interface | eno1 |
| MAC Address | aa:bb:cc:dd:ee:00 |
Do not reuse node addresses for VIPs or DNS. In the 10.0.0.0/28 examples used throughout this site:
| Role | Example |
|---|---|
| Gateway | 10.0.0.1 |
| DNS | 10.0.0.2 |
| API VIP | 10.0.0.3 |
| Ingress VIP | 10.0.0.4 |
| Installation host | 10.0.0.5 |
| Hub (SNO, if used) | 10.0.0.6 |
| Control plane 1 (rendezvous) | 10.0.0.7 |
| Control plane 2–3 | 10.0.0.8–10.0.0.9 |
| Workers 1–3 | 10.0.0.10–10.0.0.12 |
| Additional worker | 10.0.0.13 |
Example /28 is capacity-tight
A /28 has only 14 usable host addresses. The table above uses most of them for gateway, DNS, VIPs, install host, and a 6-node cluster. Treat 10.0.0.0/28 as an illustrative layout — use a larger subnet (for example /24) for real POCs that may add workers or hub/spoke nodes.
Tip
Document all static IP assignments in a spreadsheet or table before starting installation. The Assisted Installer will require this information for each host.
Network Architecture¶
Keep it Simple for POC
For POC environments, keep the networking as simple as possible. A single bond (or even a single NIC) for all traffic is perfectly acceptable. The multi-bond architectures described below are highly beneficial in production for performance isolation and fault tolerance, but we won't be engaging in large-scale performance testing in POC environments. Implement only what the customer already has available on their switches.
Simple (POC Recommended)¶
A single bond or NIC carrying all cluster traffic — management, storage, and VM data.
| Bond | Purpose | Mode | MTU |
|---|---|---|---|
| bond0 | All traffic | 802.3ad | 1500 |
MTU consistency with storage
If the storage array/VLAN is configured for jumbo frames (MTU 9000), the host interface/VLAN carrying storage traffic must also be 9000 end-to-end. Do not recommend MTU 1500 for all traffic and MTU 9000 for storage as simultaneous defaults. Choose one:
(a) Run storage on the same bond at MTU 1500 and do not enable jumbo on the array, or
(b) If the array/VLAN is 9000, configure a dedicated storage interface/VLAN at 9000.
Production Multi-Bond Architecture¶
In production environments with OpenShift Virtualization, the recommended architecture separates traffic across two bonds:
| Bond | Purpose | Mode | MTU | Switch Config |
|---|---|---|---|---|
| bond0 | Management / Cluster | 802.3ad | 1500 | Access port or VLAN |
| bond1 | Data plane (trunk) | 802.3ad | 9000 | Trunk port |
bond1 carries a trunked connection with multiple VLANs for different traffic types:
| VLAN on bond1 | Purpose | MTU | Notes |
|---|---|---|---|
| VLAN 100 | VM workload traffic | 9000 | Underlay network for VM-to-external access |
| VLAN 200 | Storage network | 9000 | Dedicated path to storage array |
| VLAN 300 | Live migration | 9000 | Dedicated path for VM live migrations |
After install, create the VLAN interfaces with NMState — see Configure Networking — Live Migration Network VLAN.
This separation provides:
- Performance isolation — Storage and live migration traffic don't compete with VM workload traffic
- Jumbo frames — The data plane bond runs MTU 9000 for storage and migration performance
- Security — Management traffic is isolated from tenant/VM traffic
- QoS — Network team can apply different QoS policies per VLAN
The post-installation Networking page has NNCP examples for configuring these bonds and VLANs after the cluster is installed.
Required Firewall Ports¶
| Port | Protocol | Source | Destination | Purpose |
|---|---|---|---|---|
| 6443 | TCP | All | Control Plane | Kubernetes API |
| 22623-22624 | TCP | Nodes | Control Plane | Machine Config Server (TLS and insecure) |
| 2379-2380 | TCP | Control Plane | Control Plane | etcd |
| 10250-10259 | TCP | All nodes | All nodes | Kubelet |
| 6081 | UDP | All nodes | All nodes | Geneve (OVN-Kubernetes) |
| 9000-9999 | TCP/UDP | All nodes | All nodes | Node services |
| 500 | UDP | All nodes | All nodes | IPsec IKE |
| 4500 | UDP | All nodes | All nodes | IPsec NAT-T |
| 30000-32767 | TCP/UDP | All nodes | All nodes | NodePort services |
| ICMP | — | All nodes | All nodes | Network reachability tests |
| 1936 | TCP | All nodes | All nodes | Ingress health check metrics |
| ESP | — | All nodes | All nodes | IPsec Encapsulating Security Payload |
The table above is node-to-node. Also open these paths from clients, the install host, and infrastructure services:
| Port | Protocol | Source | Destination | Purpose |
|---|---|---|---|---|
| 6443 | TCP | Operators, oc, CI |
API VIP | Kubernetes API |
| 80, 443 | TCP | Users and apps | Ingress VIP | OpenShift console and application Routes |
| 53 | TCP/UDP | All nodes, install host | DNS servers | Name resolution |
| 123 | UDP | All nodes | NTP servers | Time sync |
| 443, 623 | TCP/UDP | Install host | BMC / iLO / iDRAC | Virtual media and power control |
| 8080 | TCP | BMC / nodes | Install host | Agent-based ISO HTTP serve (if used) |
Storage protocol ports (when using external arrays)¶
Open these between cluster nodes and the storage array as required by your vendor and protocol. Not all apply to every POC:
| Port | Protocol | Typical use |
|---|---|---|
| 3260 | TCP | iSCSI |
| 4420 | TCP | NVMe/TCP |
| 2049 | TCP/UDP | NFS |
| 111 | TCP/UDP | NFS rpcbind (if required by your array) |
Fibre Channel does not use Ethernet ports — configure zoning on the FC fabric instead.
IPsec ports
Ports 500/UDP and 4500/UDP in the node-to-node table apply only when you enable OVN-Kubernetes IPsec. Skip them if IPsec is not in scope.
Outbound Access¶
The following external endpoints must be reachable from all cluster nodes (unless you are doing a disconnected install):
Container Registries
| Destination | Port | Purpose |
|---|---|---|
| registry.redhat.io | 443 | Core container images |
| registry.access.redhat.com | 443 | Core container images |
| access.redhat.com | 443 | Signature store for image verification |
| quay.io | 443 | Core container images |
| cdn.quay.io | 443 | Core container images (CDN) |
| registry.connect.redhat.com | 443 | Third-party certified operator images |
Tip
You can use *.quay.io instead of individually listing cdn.quay.io and cdn0[1-6].quay.io. You can also use *.access.redhat.com to cover the registry.
Cluster Access, Authentication, and Updates
| Destination | Port | Purpose |
|---|---|---|
| api.openshift.com | 443 | Cluster tokens and update checks |
| console.redhat.com | 443 | Assisted Installer, telemetry |
| sso.redhat.com | 443 | Authentication for console.redhat.com |
*.apps.{{ cluster_name }}.{{ base_domain }} |
443 | Cluster access, authentication, and Operator health checks (oauth, console, ingress-canary) |
If wildcard firewall rules are not allowed
If a wildcard is not allowed, explicitly allow oauth-openshift.apps.{{ cluster_name }}.{{ base_domain }}, canary-openshift-ingress-canary.apps.{{ cluster_name }}.{{ base_domain }}, and console-openshift-console.apps.{{ cluster_name }}.{{ base_domain }}.
Installation and Release Artifacts
| Destination | Port | Purpose |
|---|---|---|
| mirror.openshift.com | 443 | Mirrored install content and images |
| quayio-production-s3.s3.amazonaws.com | 443 | Quay image content in AWS |
| rhcos.mirror.openshift.com | 443 | RHCOS images |
| storage.googleapis.com | 443 | Release image signatures |
Telemetry (if not disabled)
| Destination | Port | Purpose |
|---|---|---|
| cert-api.access.redhat.com | 443 | Telemetry |
| api.access.redhat.com | 443 | Telemetry |
| infogw.api.openshift.com | 443 | Telemetry |
# Verify connectivity from a node to required endpoints
curl -s -o /dev/null -w "%{http_code}" https://console.redhat.com
curl -s -o /dev/null -w "%{http_code}" https://quay.io
curl -s -o /dev/null -w "%{http_code}" https://registry.redhat.io
curl -s -o /dev/null -w "%{http_code}" https://mirror.openshift.com
Details¶
Machine Network¶
For cluster-managed networking (Assisted/Agent-based default, keepalived VIPs), all control plane nodes and the API/Ingress VIPs must be on the same contiguous subnet (same L2 broadcast domain). Spanning multiple subnets requires user-managed networking plus an external load balancer. Node-to-node L3 routing alone cannot fail over the API or Ingress VIP. With static IPs, each node's network configuration is provided during the Assisted Installer setup.
NTP¶
All cluster nodes must have synchronized time. Provide an NTP server that is reachable from the cluster hosts.
Static-IP environments require explicit NTP configuration
With static IPs and no DHCP, DHCP NTP options are not available. Just having a reachable NTP server is not enough — nodes must be explicitly configured to use it:
- Assisted Installer: set
additional_ntp_sourceorntp_sourcesin the API - Agent-based: set
additionalNTPSourcesinagent-config.yaml
Proxy Configuration¶
If your environment requires an HTTP proxy for outbound traffic, configure proxy settings during cluster creation. Provide:
- HTTP Proxy URL
- HTTPS Proxy URL
- No Proxy list (include machine network, cluster network, service network, and internal domains)
Example noProxy value:
| Entry | Reason |
|---|---|
.basedomain.com |
Internal domain — do not proxy |
10.0.0.0/28 |
Machine network (node-to-node traffic) |
10.128.0.0/14 |
Cluster network (pod-to-pod traffic) |
172.30.0.0/16 |
Service network (ClusterIP services) |
localhost |
Loopback |
127.0.0.1 |
Loopback |
.cluster.local |
In-cluster DNS |
.svc |
In-cluster service DNS |
Warning
Do not include the API or Ingress VIPs in the noProxy list with a wildcard. The installer copies networking.machineNetwork, networking.clusterNetwork, and networking.serviceNetwork CIDRs into status.noProxy. If API/Ingress VIPs and node IPs are within machineNetwork, they are covered. If a user-managed load balancer or node IPs sit outside that CIDR, add those addresses/CIDRs (and the cluster base domain) to noProxy yourself.
TLS-Intercepting (MITM) Proxy¶
Many enterprise environments use a TLS-intercepting proxy (also called a man-in-the-middle or MITM proxy) that terminates and re-encrypts outbound HTTPS traffic for inspection. If your organization does this, OpenShift nodes will receive certificates signed by your proxy's internal CA rather than the original public CA. Without the proxy's root or intermediate certificate in the trust bundle, TLS connections to external services will fail with certificate validation errors.
How to Determine if You Have a MITM Proxy¶
From a machine on the same network as the cluster nodes, inspect the certificate chain for an external endpoint:
openssl s_client -connect quay.io:443 -showcerts </dev/null 2>/dev/null | openssl x509 -noout -issuer -subject
If the issuer shows your organization's internal CA (not a public CA like DigiCert, Let's Encrypt, etc.), your traffic is being intercepted by a MITM proxy.
You can also check with curl:
How to Obtain the Proxy CA Certificate¶
Contact your network security or infrastructure team and request the root CA certificate (and any intermediate certificates) used by the TLS-intercepting proxy. Common sources:
| Source | How to Obtain |
|---|---|
| Security / Network team | Request the CA cert directly |
| Corporate PKI portal | Download from your organization's certificate portal |
| Browser export | Visit an HTTPS site, inspect the cert chain, export the root |
| From the proxy itself | Some proxies publish their CA at a known internal URL |
You can also extract it directly from the connection:
openssl s_client -connect quay.io:443 -showcerts </dev/null 2>/dev/null | \
awk '/BEGIN CERTIFICATE/,/END CERTIFICATE/ {print}' > proxy-ca-chain.pem
The last certificate in the chain output is typically the root CA.
Required Certificate Format¶
The certificate must be in PEM format (Base64-encoded, with -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- markers). If you receive the certificate in a different format, convert it:
# Convert DER to PEM
openssl x509 -in proxy-ca.der -inform DER -out proxy-ca.pem -outform PEM
# Convert PKCS7 to PEM
openssl pkcs7 -in proxy-ca.p7b -inform DER -print_certs -out proxy-ca.pem
If there are multiple certificates in the chain (root + intermediate), concatenate them into a single PEM file:
Verify the Certificate¶
# Confirm it is a valid CA certificate
openssl x509 -in proxy-ca.pem -noout -text | grep -A1 "Basic Constraints"
# Test that it resolves the trust issue
curl --cacert proxy-ca-bundle.pem https://quay.io
Providing the Certificate to the Assisted Installer¶
In the Assisted Installer UI, when configuring proxy settings, paste the contents of your PEM-formatted CA bundle into the Additional Trust Bundle field. This ensures all cluster nodes trust the proxy's CA during and after installation.
Warning
If the additional trust bundle is not provided and your environment has a MITM proxy, the installation will fail when nodes attempt to pull container images from Red Hat registries.