I recently ran into a tricky issue while upgrading NVIDIA vGPU on a VMware ESXi host using an NVIDIA A2 GPU. On my Management Core Server, I replaced an NVIDIA Tesla P4 with an NVIDIA A2 because the A2 has more VRAM and supports newer vGPU releases. The P4 is still a capable GPU and…
Category: AI/ML & vGPU
Building a Custom Silent Cooler for NVIDIA A2 and L4 Data Center GPUs with n3rdware
My goal with my home lab has always been clear. Keep it as silent as possible and as cool as possible, without sacrificing performance. Recently I started working more seriously with the NVIDIA A2 and the NVIDIA L4. Both are data center GPUs designed to operate inside rack servers with strong, directed airflow. They do…
PNY NVIDIA RTX PRO 6000 96GB Blackwell Workstation Edition
My homelab just got a big upgrade to run large Private AI models, experiment, train, and have some fun. The PNY NVIDIA RTX PRO 6000 Blackwell Workstation Edition is an absolute beast—the most powerful workstation GPU ever made. (96GB GDDR7, 600W, PCIe 5, AI performance 4000 TOPS, RT Core Performance 380 TFLOPS, Single-Precision Performance 125…
Heading to NVIDIA GTC AI Conference – Let’s Connect! March 17-21, 2025 | San Jose
If you’re attending the NVIDIA GTC Conference this year, feel free to reach out and connect with me. Here are some seminars that I highly recommend. Heritage Meets Technology: Leveraging Virtualized Platforms for Architectural Conservation and Business Innovation [S72407] Thursday, Mar 20 – 2:00 PM – 2:40 PM PDT Mark Cichy, Principal, Director of Design…
How Fast Is My Home Lab?
I have been tuning my Home Lab network during my vacation. Private AI/ML, vGPU and VDI requires a fast Home Lab setup. I tweaked my Switch (Dell EMC S4112T-ON), UDM Pro, ESXi hosts, and also revisited the subnetting logic. I went over my servers’ BIOS CPU and RAM features and found out that default settings…
Silent Cooling Solution for the Nvidia L4 24 GB GPU
I am keeping this post very short, with mostly photos. I tested the cooling performance with different games. The GPU’s max power is 72W, though during my tests, it exceeded 75W. It’s also possible to limit it to 30W. I tested the GPU by running games like Black Myth: Wukong, Cyberpunk 2077, Uncharted 4: A…
Nvidia L4: Powerful Low-Power GPU for Nvidia AI Enterprise and Virtual GPU
I’ve been searching the internet for a long time to find a versatile GPU for AI and video graphics workloads that also supports vGPU and Nvidia AI Enterprise. Some of the GPUs I considered were the RTX 6000 Ada, A2, A10, L4, T4, A40, and A16. I was most drawn to the RTX 6000 Ada…
Deploying and Configuring Nvidia DLS for AI Enterprise and vGPU: Step-by-Step Guide
NB! At the end of the blog post, there is a YouTube video and an eBook – a photo-based step-by-step guide Download Nvidia vGPU Drivers for ESXi Download Nvidia vGPU License Server: Installing Nvidia vGPU Drivers on ESXi Deploying the NvidiaDLS OVA to vSphere Configuring Nvidia DLS (License) Server Installing Nvidia Drivers on a Windows…
Quick and Easy Guide to Installing Meta Llama 3.1 405B, 70B, 8B Language Models with Ollama, Docker, and OpenWebUI
I will show how easy and quick it is to install Llama 3.1 405B, 70B, 8B, or another language model on your computer or VM using Ollama, Docker, and OpenWebUI. It is so simple to install that even a grandmother or grandfather could do it. This is private AI, not cloud-based. All data is on…
Meta Llama 3.1 405B: GPU vs. CPU Performance Evaluation and RAM Considerations
It’s time to start testing various Private AI models, and fortunately, the timing is just right. Meta has just released six new AI language models. These models run on-premises and do not interact with the cloud or OpenAI’s ChatGPT. Llama 3.1 405B competes with leading models like GPT-4, GPT-4o, and Claude 3.5 Sonnet, while smaller…