Boost throughput with scaling VMs while keeping the GPUs to a minimum
Boost throughput with scaling VMs while keeping…
This article describes some performance tests we conducted to explore the advantages of virtualizing NVIDIA GPUs with VMware vSphere for generative AI workloads. We tested LLAMA2-7b and LLAMA2-13b parameter models using various virtual GPU (vGPU) and multi-instance GPU (MIG) configurations on […]