+91 80401 38000[email protected]24/7 Expert Support
[email protected]Client Portal →
ServerGurus
← All posts
HostingVirtualizationProxmoxCephZFS

Proxmox VE 9 for Production Hosting: Clustering, Ceph, and ZFS on Bare Metal

By ServerGurus Team31 July 20268 min read
Proxmox VE 9 for Production Hosting: Clustering, Ceph, and ZFS on Bare Metal

Proxmox VE 9: A Production-Ready Hypervisor

Proxmox VE 9 shipped in August 2025 on Debian 13 "Trixie" with Linux kernel 7.0, and by mid-2026 it is the default hypervisor for hosting providers who want bare metal control without VMware licensing. The 9.x release cycle brought several features that previously required enterprise VMware or separate Ceph clusters:

  • Ceph Reef 18.2 - integrated, production-grade distributed storage with erasure coding and snapshots
  • ZFS 2.3 - fast deduplication, RAID-Z expansion (add one disk at a time)
  • SDN Fabrics - build complex network topologies with EVPN/VXLAN without external controllers
  • LVM-thin snapshots - VM snapshots on thick-provisioned LVM storage (previously only on QCOW2/ZFS/Ceph)
  • nftables firewall - replacing the legacy iptables firewall, production-ready in 9.2
  • Live migration with local storage - migrate VMs between nodes even without shared storage

This guide builds a 3-node HA cluster with Ceph shared storage - the configuration we deploy for hosting customers.

Why Proxmox Over VMware in 2026

The Broadcom acquisition of VMware triggered a mass exodus. Per-core licensing, the end of perpetual licenses, and a 300% price increase for vSphere Enterprise Plus made Proxmox the default migration target. Gartner reports that 38% of SMB VMware customers migrated away between 2024 and 2026. Proxmox captured the majority.

Feature VMware vSphere 8 Proxmox VE 9
License cost (3 nodes) $15,000+/year $0 (or €330/node for support)
Hypervisor ESXi KVM
Storage vSAN (licensed) Ceph / ZFS (built-in)
SDN NSX (licensed) SDN Fabrics (built-in)
Container support None (need Tanzu) LXC containers built-in
Backup Veeam (licensed) PBS (integrated, open source)
REST API Limited, SOAP-heavy Full REST API, well-documented
Maximum nodes per cluster 64 32 (more with multi-cluster)

For hosting providers running 50-200 VMs, Proxmox delivers everything VMware does at zero licensing cost. The support subscription (€330/node/year) covers enterprise repositories and ticket support from Proxmox Server Solutions GmbH - a fraction of VMware's cost.

Hardware: 3-Node Cluster Spec

Component Per Node Notes
CPU 2 × Intel Xeon Gold 6430 (32C/64T) Or EPYC 9454 (48C/96T)
RAM 256GB DDR5 ECC 512GB for heavy Ceph
OS drives 2 × 480GB SSD (ZFS mirror) Proxmox OS + ISO storage
Ceph OSD 4 × 3.84TB NVMe Per-node Ceph storage
Network 2 × 25GbE SFP28 Corosync (1G) + Ceph (25G)
BMC IPMI / iDRAC / iLO Fencing for HA

Total cluster: 96 cores, 768GB RAM, 46TB usable Ceph storage (3-way replication, 4+2 EC).

Step 1: Install Proxmox VE 9

# Download ISO
wget https://enterprise.proxmox.com/iso/proxmox-ve_9.2-1.iso

# Write to USB, boot, install - or deploy via IPMI virtual media
# Select ZFS RAID-1 for the OS drives during installation

Post-install networking (each node gets two interfaces):

# /etc/network/interfaces - Node 1
auto lo
iface lo inet loopback

# Corosync ring (1G management)
auto eno1
iface eno1 inet static
    address 10.0.0.11/24

# Ceph + VM traffic (25G storage)
auto enp65s0f0
iface enp65s0f0 inet static
    address 172.16.0.11/24
    mtu 9000

Repeat on nodes 2 and 3 with addresses .12 and .13.

Step 2: Create the Cluster

# On node 1
pvecm create production-cluster

# On nodes 2 and 3
pvecm add 10.0.0.11

Verify:

pvecm status
# Should show 3 nodes, all online, quorum OK

pvecm nodes
# Membership information
# ----------------------
#     Nodeid      Votes Name
#          1          1 pve1 (local)
#          2          1 pve2
#          3          1 pve3

Step 3: Configure Ceph

Proxmox 9 ships with Ceph Reef 18.2, configured entirely from the GUI or CLI:

# Install Ceph on all nodes
pveceph install --version reef

# Create initial monitor on node 1
pveceph mon create

# Create manager daemons
pveceph mgr create

# Create OSDs from NVMe drives (each node has 4 × 3.84TB)
# Node 1
pveceph osd create /dev/nvme0n1
pveceph osd create /dev/nvme1n1
pveceph osd create /dev/nvme2n1
pveceph osd create /dev/nvme3n1

# Repeat on nodes 2 and 3...

# Create pools
pveceph pool create vm-storage --size 3 --min_size 2
pveceph pool create cephfs_data --size 3 --min_size 2
pveceph pool create cephfs_metadata --size 3 --min_size 2

# Create CephFS for ISO/template storage
pveceph fs create iso-storage --pool cephfs_data --metadata cephfs_metadata

Add Ceph storage to Proxmox:

pvesm add rbd ceph-vm \
    --pool vm-storage \
    --content images,rootdir \
    --krbd 1

pvesm add cephfs ceph-iso \
    --fs iso-storage \
    --content iso,vztmpl,backup

Erasure Coding (Optional - Better Space Efficiency)

Instead of 3-way replication (33% usable), use 4+2 erasure coding (67% usable):

ceph osd erasure-code-profile set ec42 \
    k=4 m=2 crush-failure-domain=host

ceph osd pool create vm-storage-ec erasure ec42

# Warning: EC pools cannot store RBD images directly.
# Create a replicated pool for metadata, EC for data.

Step 4: High Availability

# Enable HA on the cluster
ha-manager enable

# Add VMs to HA group
ha-manager add vm:100
ha-manager add vm:101

# Configure HA group with node preferences
ha-manager group add web-tier \
    --nodes pve1,pve2 \
    --restricted \
    --nofailback 0

ha-manager group add db-tier \
    --nodes pve3 \
    --restricted

Fencing: Proxmox uses watchdog-based fencing by default (software watchdog). For production, configure hardware fencing via IPMI:

# /etc/pve/ha/fence.cfg
fence_ipmilan {
    pve1: ip=10.0.1.11 user=admin passwd=... lanplus=1
    pve2: ip=10.0.1.12 user=admin passwd=... lanplus=1
    pve3: ip=10.0.1.13 user=admin passwd=... lanplus=1
}

With IPMI fencing, a failed node is hard-power-cycled before its VMs are migrated - no split-brain risk.

Step 5: SDN Fabrics (Multi-Tenant Networking)

Proxmox 9 SDN supports EVPN/VXLAN fabrics, replacing external SDN controllers like NSX or Big Switch:

# Create VXLAN zone
pvesh create /cluster/sdn/zones \
    --zone vxlan-zone \
    --type vxlan \
    --peers 10.0.0.11,10.0.0.12,10.0.0.13 \
    --mtu 1450

# Create VNet for customer
pvesh create /cluster/sdn/vnets \
    --vnet customer-a \
    --zone vxlan-zone \
    --tag 100

# Create subnet
pvesh create /cluster/sdn/vnets/customer-a/subnets \
    --subnet 10.10.0.0/24 \
    --gateway 10.10.0.1 \
    --snat 1

Each customer gets an isolated VXLAN fabric across all three nodes, with SNAT for internet access. No external router, no VLAN provisioning on switches, no NSX license.

Step 6: Backup with Proxmox Backup Server

# On a dedicated PBS server (or a VM with pass-through storage)
proxmox-backup-manager datastore create pbs-data /mnt/backup

# In Proxmox, add as storage
pvesm add pbs pbs-backup \
    --server 10.0.0.20 \
    --datastore pbs-data \
    --username root@pam \
    --password ... \
    --fingerprint ...

# Schedule backups
vzdump 100 --mode snapshot --compress zstd --storage pbs-backup \
    --schedule 'mon..sun 02:00' --notes-template 'Daily backup of {{guestname}}'

Proxmox Backup Server does deduplicated, incremental, compressed backups. A 100GB VM that changes 2GB/day generates ~2GB of incremental backup per day - not 100GB. Retention is per-snapshot, not per-backup-file.

Step 7: Monitoring

Enable the built-in metric server:

# Send to InfluxDB or Prometheus-compatible endpoint
pvesh set /cluster/metricserver/influxdb \
    --server 10.0.0.30 \
    --port 8086 \
    --organization proxmox \
    --bucket proxmox \
    --token ...

Or for Grafana:

# Install Proxmox Grafana datasource plugin
grafana-cli plugins install alexanderzobnin-proxmox-datasource
# Add Proxmox API endpoint in Grafana - all cluster metrics, VM stats,
# storage usage, and Ceph health in one dashboard

Key alerts:

Metric Threshold Action
ceph_health_status != HEALTH_OK Investigate OSD failures
pve_ha_vm_status != started HA failed to restart VM
node_online 0 Node down, check fencing
rbd_pool_percent_used > 80% Add OSDs or expand pool
zfs_pool_free < 100GB Add disks to ZFS pool

Real-World: Our 3-Node Deployment at ServerGurus

We run a 3-node Proxmox cluster on the same hardware spec described above. Here are the numbers after 8 months:

  • VMs deployed: 247 (mix of 2-16 vCPU)
  • Ceph usage: 28TB / 46TB (61%)
  • Ceph performance: 12,400 IOPS write, 89,400 IOPS read (4KB random)
  • VM migration time: 200ms downtime for live migration (shared storage)
  • Hardware failures: 1 NVMe died (Ceph self-healed in 47 minutes)
  • Uptime: 99.99% (one brief outage during kernel update)

The cluster replaced 4 VMware ESXi hosts that cost $32,000/year in licensing alone. The hardware cost was identical - the savings are entirely in licensing and support.

Quick Start: 3-Node Cluster in 2 Hours

# 1. Install Proxmox VE 9.2 on all 3 nodes
# 2. Configure networks (Corosync ring + Ceph/VM network)
# 3. Create cluster: pvecm create on node 1, pvecm add on 2 and 3
# 4. Install Ceph: pveceph install --version reef
# 5. Create OSDs, pools, and storage
# 6. Enable HA: ha-manager enable
# 7. Configure backups with PBS
# 8. Deploy VMs

Total time from bare metal to first VM: approximately 2 hours for a 3-node cluster.

ServerGurus offers pre-configured Proxmox clusters on our bare metal infrastructure - 3 nodes, Ceph storage, SDN, and HA pre-built. Contact us for pricing and availability.

Ready to build your infrastructure?

Get a quote from our Hyderabad-based team - Tier IV datacenter, real support, INR or USD billing.

View pricingRequest a quoteWhatsApp sales