Proxmox

Hardware overview

Host: Minisforum A2

  • RAM: 128GB
  • Storage:
    • 500GB NVMe (vmstore1)
    • 1TB NVMe (vmstore2)
  • Network: Connected to pfSense, Lab VLAN 192.168.200.0/22

Nested Virtualization Approach

Due to resource constraints, this lab uses nested virtualization with Proxmox VE:

  • Proxmox VE 8.x on bare metal Minisforum (Physical host with KVM/QEMU)
  • Nested ESXi VMs for VCF management and workload domains
  • VCF components (vCenter, SDDC Manager, NSX) as VMs on nested ESXi

Tip

Things will be running slow as we are doing nested ESXi, however, the performance is acceptable for a homelab scenerio.

Required CPU Flags for Nested Virtualization

Important

For nested ESXi to work properly on Proxmox with AMD processors, specific CPU flags must be enabled:

# In Proxmox VM config (/etc/pve/qemu-server/<vmid>.conf)
cpu: host
args: -cpu host,+svm,+npt

CPU Flag Explanation

Flag Full Name Purpose
host Host CPU Passthrough Passes the physical CPU features directly to the VM instead of emulating a generic CPU. Required for nested virtualization because ESXi needs to see the real CPU capabilities. Without this, ESXi cannot detect hardware virtualization support.
+svm Secure Virtual Machine AMD’s hardware virtualization extension (equivalent to Intel VT-x). This flag exposes AMD-V virtualization capabilities to the nested ESXi, allowing it to run VMs inside. ESXi requires hardware virtualization support to function as a hypervisor.
+npt Nested Page Tables AMD’s hardware-assisted memory virtualization (equivalent to Intel EPT). Enables two-level address translation where both the hypervisor and nested hypervisor can manage memory efficiently. Dramatically improves nested VM performance by reducing memory management overhead.

Why These Flags Matter

Without proper CPU flags:
┌─────────────────────────────────────────────────────────┐
│  Physical CPU (AMD Ryzen 9 9955HX with AMD-V + NPT)     │
│                         │                               │
│                         ▼                               │
│  ┌─────────────────────────────────────────────────┐    │
│  │  Proxmox (QEMU with generic CPU emulation)      │    │
│  │                         │                       │    │
│  │                         ▼                       │    │
│  │  ┌─────────────────────────────────────────┐    │    │
│  │  │  Nested ESXi                            │    │    │
│  │  │  ❌ Cannot see AMD-V/SVM                │     │    │
│  │  │  ❌ Cannot run VMs                      │     │   │
│  │  │  ❌ VCF deployment fails                │     │   │
│  │  └─────────────────────────────────────────┘    │   │
│  └─────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────┘

With cpu: host and +svm +npt flags:
┌─────────────────────────────────────────────────────────┐
│  Physical CPU (AMD Ryzen 9 9955HX with AMD-V + NPT)     │
│                         │                               │
│                         ▼ (passthrough)                 │
│  ┌─────────────────────────────────────────────────┐    │
│  │  Proxmox (QEMU with host CPU passthrough)       │    │
│  │                         │                       │    │
│  │                         ▼ (AMD-V exposed)       │    │
│  │  ┌─────────────────────────────────────────┐    │    │
│  │  │  Nested ESXi                            │    │    │
│  │  │  ✅ Sees AMD-V/SVM support              │    │    │
│  │  │  ✅ Can run vCenter, NSX, workload VMs  │    │    │
│  │  │  ✅ NPT improves memory performance     │    │    │
│  │  └─────────────────────────────────────────┘    │    │
│  └─────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────┘

Storage Layout per nested ESXi

Disk Size Purpose Proxmox Storage Notes
scsi0 40GB ESXi boot local-lvm (500GB NVMe) ESXi OS + scratch
scsi1 50GB vSAN cache vmstore2 (1TB NVMe) Mark as SSD in ESXi
scsi2 200GB vSAN capacity vmstore2 (1TB NVMe) Mark as SSD in ESXi

vSAN capacity calculation

Raw capacity:    3 hosts × 200GB = 600GB
Usable (FTT=1):  600GB ÷ 2 = 300GB (mirrored)

VCF Management VMs (actual usage with thin provisioning):
├── vCenter:      ~100-150GB
├── NSX Manager:  ~80-100GB
└── SDDC Manager: ~60-80GB
────────────────────────────
Total:            ~240-330GB  ✅ Fits within 300GB usable