홈랩 재구성 후기입니다

as-is

esxi 6.7

CPU : i5-10400t

MB : asrock H570 itx

pcie 없음

to-be

Hypervisor proxmox 7.2 ext4 grub

CPU : i5-10400t

MB : AORUS z490-e

HBA : Dell H310(it mode - lsi9211-8i)

GPU: rtx 3070

우선 목적이였던

1) rtx3070 패스스루 통한 게이밍용 데스크탑/RDP 겸용 VM 구성

2) 헤놀로지(DS918-42218)에 hba(vt-d)및 igpu(vt-g) 패스스루 통한 smart 데이터 및 트랜스코딩 활성화

는 전부 달성하였습니다.

iommu 그룹은 아래와 같습니다

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation Comet Lake-S 6c Host Bridge/DRAM Controller [8086:9b53] (rev 03)
IOMMU Group 10 00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:06b8] (rev f0)
IOMMU Group 11 00:1c.4 PCI bridge [0604]: Intel Corporation Device [8086:06bc] (rev f0)
IOMMU Group 12 00:1d.0 PCI bridge [0604]: Intel Corporation Comet Lake PCI Express Root Port #9 [8086:06b0] (rev f0)
IOMMU Group 13 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:0685]
IOMMU Group 13 00:1f.3 Audio device [0403]: Intel Corporation Comet Lake PCH cAVS [8086:06c8]
IOMMU Group 13 00:1f.4 SMBus [0c05]: Intel Corporation Comet Lake PCH SMBus Controller [8086:06a3]
IOMMU Group 13 00:1f.5 Serial bus controller [0c80]: Intel Corporation Comet Lake PCH SPI Controller [8086:06a4]
IOMMU Group 14 05:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03)
IOMMU Group 15 07:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I225-V [8086:15f3] (rev 03)
IOMMU Group 16 08:00.0 Non-Volatile memory controller [0108]: SK hynix Device [1c5c:174a]
IOMMU Group 17 lspci: -s: Invalid slot number
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 03)
IOMMU Group 1 00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) [8086:1905] (rev 03)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [GeForce RTX 3070] [10de:2484] (rev a1)
IOMMU Group 1 01:00.1 Audio device [0403]: NVIDIA Corporation GA104 High Definition Audio Controller [10de:228b] (rev a1)
IOMMU Group 1 02:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
IOMMU Group 1 02:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
IOMMU Group 1 02:00.2 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
IOMMU Group 1 02:00.3 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
IOMMU Group 2 00:02.0 VGA compatible controller [0300]: Intel Corporation CometLake-S GT2 [UHD Graphics 630] [8086:9bc8] (rev 03)
IOMMU Group 3 00:14.0 USB controller [0c03]: Intel Corporation Comet Lake USB 3.1 xHCI Host Controller [8086:06ed]
IOMMU Group 3 00:14.2 RAM memory [0500]: Intel Corporation Comet Lake PCH Shared SRAM [8086:06ef]
IOMMU Group 4 00:14.3 Network controller [0280]: Intel Corporation Comet Lake PCH CNVi WiFi [8086:06f0]
IOMMU Group 5 00:15.0 Serial bus controller [0c80]: Intel Corporation Comet Lake PCH Serial IO I2C Controller #0 [8086:06e8]
IOMMU Group 5 00:15.1 Serial bus controller [0c80]: Intel Corporation Comet Lake PCH Serial IO I2C Controller #1 [8086:06e9]
IOMMU Group 6 00:16.0 Communication controller [0780]: Intel Corporation Comet Lake HECI Controller [8086:06e0]
IOMMU Group 7 00:17.0 SATA controller [0106]: Intel Corporation Device [8086:06d2]
IOMMU Group 8 00:1b.0 PCI bridge [0604]: Intel Corporation Comet Lake PCI Express Root Port #17 [8086:06c0] (rev f0)
IOMMU Group 9 00:1b.4 PCI bridge [0604]: Intel Corporation Comet Lake PCI Express Root Port #21 [8086:06ac] (rev f0)
 

proxmox 설정은 아래와 같이 수정하였습니다

#/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt i915.enable_gvt=1 initcall_blacklist=sysfb_init"

#/etc/modprobe.d/vfio.conf (GPU&HBA)

options vfio-pci ids=10de:2484,10de:228b,1000:0072

#/etc/modprobe.d/kvm.conf

options kvm ignore_msrs=1

#/etc/modprobe.d/blacklist.conf (정상 작동 안하는거 같습니다 i915가 들어있는데 드라이버는 호스트에서 로드됩니다)

blacklist radeon
blacklist nouveau
blacklist nvidia
blacklist i915
blacklist mpt3sas
blacklist snd_hda_intel
 

상기 설정 이후 lspci -v로 확인시 다이렉트 패스스루 할 장비들은 Kernel driver in use : vfio-pci가 되어야 합니다