Infrastructure¶
All nodes are bare metal servers in an on-premise environment. Provision compute resources that meet or exceed the minimum requirements for each node type.
OpenShift Terminology
A quick note on terminology. In OpenShift, the same server can be described at three layers:
- Host (
BareMetalHost, managed by the Bare Metal Operator) is the physical box and its BMC, answering how to power and provision the hardware - Machine (
Machine, managed by the Machine API and grouped into aMachineSet) is the infrastructure instance that backs a node and is the unit you scale which can be a physical box or a virtual machine - Node (
Node) is the Kubernetes-level member that runs a kubelet and that the scheduler places pods onto.
Required Hardware¶
Consider these minimum values — the more the better.
Do I need an installation host?
No, as long as the computer we are working from has full access to all the necessary tools or is able to get them without opening tickets (oc, openshift-install, kubectl, helm, kustomize, nmstatectl, git, podman, istioctl, etc). However, if the work is proceeding and the laptop or environment that is currently being used is not available (PTO, sick, etc), then everything comes to a halt. We highly recommend working from a centralized installation host to ensure not only access but also compatibility with the examples in this guide.
Installation¶
| Host Type | Count | CPU | Memory | Install Disk | Secondary Disk |
|---|---|---|---|---|---|
| Installation | 1 | 4 | 16 GB | 60 GB | --- |
| Control Plane | 3 | 16 | 32 GB | 120 GB | --- |
| Worker | 3 | 16 | 64 GB | 120 GB | 1 TB (opt) |
Fleet Management (Hub + Spoke)¶
| Host Type | Count | CPU | Memory | Install Disk | Secondary Disk |
|---|---|---|---|---|---|
| Installation | 1 | 4 | 16 GB | 60 GB | --- |
| Hub (SNO) | 1 | 16 | 64 GB | 120 GB | 1 TB |
| Spoke Control Plane | 3 | 16 | 32 GB | 120 GB | --- |
| Spoke Worker | 3 | 16 | 64 GB | 120 GB | 1 TB (opt) |
Machine Types
The installation and hub machine can be either a virtual machine or bare metal host. They just need network access to the other hosts.
CPU Architecture
For the POC, prefer the same vendor and generation CPU architectures across all machines.
etcd Disk Requirements
For the install disk, run etcd on a block device that can write at least 50 IOPS of 8 KB sequentially, including fdatasync, in under 10 ms. Heavily loaded clusters should target 500 sequential IOPS of 8 KB in 2 ms. See etcd Storage for the full disk performance requirements.
Network Interface Requirements¶
Only a single NIC is required for OpenShift. To perform more advanced networking, 4 x 10 GbE is recommended for bond0/bond1.
Machine Information¶
Collect the interface names, MAC addresses for ALL NICs, and the install disk location on the machines. Below is an example table of values needed.
| Hostname | hl01ocpinstall | Disk Hint | /dev/sda |
| BMC IP | BMC Credentials | ||
| Interface | MAC Address | Bond | IP Address |
| eno1 | A0:B1:C2:D3:E4:E0 | - | 10.0.0.5 |
| Hostname | hl01ocphub | Disk Hint | /dev/sda |
| BMC IP | BMC Credentials | ||
| Interface | MAC Address | Bond | IP Address |
| eno1 | A0:B1:C2:D3:E4:E1 | - | 10.0.0.6 |
| Hostname | hl01ocpcp01 | Disk Hint | /dev/sda |
| BMC IP | BMC Credentials | ||
| Interface | MAC Address | Bond | IP Address |
| eno1 | A0:B1:C2:D3:E4:ED | bond0 | 10.0.0.7 |
| eno2 | A0:B1:C2:D3:E4:E2 | bond0 | |
| eno3 | A0:B1:C2:D3:E4:E3 | bond1 | |
| eno4 | A0:B1:C2:D3:E4:E4 | bond1 |
| Hostname | hl01ocpcp02 | Disk Hint | /dev/sda |
| BMC IP | BMC Credentials | ||
| Interface | MAC Address | Bond | IP Address |
| eno1 | A0:B1:C2:D3:E4:E5 | bond0 | 10.0.0.8 |
| eno2 | A0:B1:C2:D3:E4:E6 | bond0 | |
| eno3 | A0:B1:C2:D3:E4:E7 | bond1 | |
| eno4 | A0:B1:C2:D3:E4:E8 | bond1 |
| Hostname | hl01ocpcp03 | Disk Hint | /dev/sda |
| BMC IP | BMC Credentials | ||
| Interface | MAC Address | Bond | IP Address |
| eno1 | A0:B1:C2:D3:E4:E9 | bond0 | 10.0.0.9 |
| eno2 | A0:B1:C2:D3:E4:EA | bond0 | |
| eno3 | A0:B1:C2:D3:E4:EB | bond1 | |
| eno4 | A0:B1:C2:D3:E4:EC | bond1 |
| Hostname | hl01ocpw01 | Disk Hint | /dev/sda |
| BMC IP | BMC Credentials | ||
| Interface | MAC Address | Bond | IP Address |
| eno1 | A0:B1:C2:D3:E4:F1 | bond0 | 10.0.0.10 |
| eno2 | A0:B1:C2:D3:E4:F2 | bond0 | |
| eno3 | A0:B1:C2:D3:E4:F3 | bond1 | |
| eno4 | A0:B1:C2:D3:E4:F4 | bond1 |
| Hostname | hl01ocpw02 | Disk Hint | /dev/sda |
| BMC IP | BMC Credentials | ||
| Interface | MAC Address | Bond | IP Address |
| eno1 | A0:B1:C2:D3:E4:F5 | bond0 | 10.0.0.11 |
| eno2 | A0:B1:C2:D3:E4:F6 | bond0 | |
| eno3 | A0:B1:C2:D3:E4:F7 | bond1 | |
| eno4 | A0:B1:C2:D3:E4:F8 | bond1 |
| Hostname | hl01ocpw03 | Disk Hint | /dev/sda |
| BMC IP | BMC Credentials | ||
| Interface | MAC Address | Bond | IP Address |
| eno1 | A0:B1:C2:D3:E4:F9 | bond0 | 10.0.0.12 |
| eno2 | A0:B1:C2:D3:E4:FA | bond0 | |
| eno3 | A0:B1:C2:D3:E4:FB | bond1 | |
| eno4 | A0:B1:C2:D3:E4:FC | bond1 |
Machine Details¶
NIC Naming¶
On modern RHEL (RHEL CoreOS included), NICs use predictable network interface names generated at boot based on hardware topology and firmware information:
eno1,eno2— onboard NICs (from BIOS/firmware)ens1f0,ens1f1— PCI Express slots ("s" = slot, "f" = function)enp3s0— PCI bus location (p3 = bus 3, s0 = slot 0)enx— fallback to MAC address if nothing else matches
Warning
If you don't know what the values will be (disk name, NIC names), boot the boxes with a RHEL ISO Boot iso and find out. Don't guess. You don't even need to install, just start the installer and you can see the information.
BMC / Out-of-Band Management¶
Each bare metal server must have out-of-band management access for remote operations:
| Management Type | Protocol | Purpose |
|---|---|---|
| IPMI | IPMI | Power control, virtual media |
| Redfish | HTTPS | Modern BMC API, virtual media |
| iLO / iDRAC | HTTPS | Vendor-specific BMC |
Virtual media (mounting the discovery ISO remotely) is the recommended method for booting nodes during installation.
Testing BMC Access¶
Verify Redfish API connectivity from the installation host to each BMC before starting the install. This confirms credentials, network access, and helps you identify the correct system ID for each server.
List available systems:
curl -sk https://{{ bmc_ip }}/redfish/v1/Systems/ -u {{ bmc_username }}:{{ bmc_password }} | jq '.Members'
The BMC address format varies by vendor:
| Vendor | Address Format |
|---|---|
| HPE iLO | redfish-virtualmedia://{{ bmc_ip }}/redfish/v1/Systems/1 |
| Dell iDRAC | redfish-virtualmedia://{{ bmc_ip }}/redfish/v1/Systems/System.Embedded.1 |
| Cisco CIMC | redfish-virtualmedia://{{ bmc_ip }}/redfish/v1/Systems/{{ system_id }} |
Verify power state:
curl -sk https://{{ bmc_ip }}/redfish/v1/Systems/1 -u {{ bmc_username }}:{{ bmc_password }} | jq '.PowerState'
Verify virtual media is available:
curl -sk https://{{ bmc_ip }}/redfish/v1/Managers/1/VirtualMedia -u {{ bmc_username }}:{{ bmc_password }} | jq '.Members'
Tip
If any of these commands fail, check that the BMC IP is reachable from the installation host, the credentials are correct, and HTTPS (port 443) is open between the installation host and the BMC network.
BIOS/Firmware Settings¶
Ensure the following on all nodes:
- Boot mode set to UEFI
- Secure Boot supported (optional but recommended)
- Boot order set to local disk (the ISO is mounted via virtual media)
- Hardware clock set to UTC
Disable POST Memory Test¶
Servers with large amounts of RAM (512 GB+) can spend 5–10 minutes running full memory diagnostics during every boot. For a POC where clusters are built and torn down frequently, disabling the POST memory test cuts each reboot cycle significantly with no reliability trade-off.
Dell iDRAC¶
racadm set BIOS.MemSettings.MemTest Disabled
racadm jobqueue create BIOS.Setup.1-1
racadm serveraction powercycle
Redfish equivalent:
curl -sk -X PATCH \
https://{{ bmc_ip }}/redfish/v1/Systems/System.Embedded.1/Bios/Settings \
-u {{ bmc_username }}:{{ bmc_password }} \
-H "Content-Type: application/json" \
-d '{"Attributes":{"MemTest":"Disabled"}}'
HPE iLO¶
Enable Memory Fast Training (skips full initialization after first good boot):
Redfish equivalent:
curl -sk -X PATCH \
https://{{ bmc_ip }}/redfish/v1/Systems/1/Bios/Settings \
-u {{ bmc_username }}:{{ bmc_password }} \
-H "Content-Type: application/json" \
-d '{"Attributes":{"MemFastTraining":"Enabled"}}'
Cisco CIMC¶
Redfish:
curl -sk -X PATCH \
https://{{ bmc_ip }}/redfish/v1/Systems/{{ system_id }}/Bios/Settings \
-u {{ bmc_username }}:{{ bmc_password }} \
-H "Content-Type: application/json" \
-d '{"Attributes":{"<memory-test-attribute>":"Disabled"}}'
Note
The exact attribute name varies between C-series generations and firmware versions (often POSTErrorPause or a toggle under the memory RAS group). Query /redfish/v1/Systems/{{ system_id }}/Bios to find the correct attribute name for your hardware.
Declarative via HostFirmwareSettings (Metal³ / Ironic)¶
If the bare metal nodes are managed by Metal³/Ironic (BareMetalHost operator), skip per-BMC scripting and drive it declaratively through HostFirmwareSettings:
apiVersion: metal3.io/v1alpha1
kind: HostFirmwareSettings
metadata:
name: {{ baremetalhost_name }}
namespace: openshift-machine-api
spec:
settings:
MemTest: "Disabled" # Dell
# MemFastTraining: "Enabled" # HPE
Look up the correct attribute names from the node's firmware schema:
The status.settings field shows current values and status.schema shows allowed values per attribute. This keeps the change in your GitOps repo and reapplies on reprovision.
Schedulable Control Plane¶
There is an ability to run the control plane nodes as schedulable, making them effectively worker nodes as well. This is discouraged unless required by the environment constraints, such as testing to run edge systems.