I will show how easy and quick it is to install Llama 3.1 405B, 70B, 8B, or another language model on your computer or VM using Ollama, Docker, and OpenWebUI. It is so simple to install that even a grandmother or grandfather could do it. This is private AI, not cloud-based. All data is on…
Tag: Home Lab
Meta Llama 3.1 405B: GPU vs. CPU Performance Evaluation and RAM Considerations
It’s time to start testing various Private AI models, and fortunately, the timing is just right. Meta has just released six new AI language models. These models run on-premises and do not interact with the cloud or OpenAI’s ChatGPT. Llama 3.1 405B competes with leading models like GPT-4, GPT-4o, and Claude 3.5 Sonnet, while smaller…
Solving Real Business Problems with Private AI: Unlocking Efficiency and Productivity
I’m on a mission to find a company facing a real problem that can be solved using Private AI—AI that operates entirely within your own data center, without relying on cloud services. I need your help to identify the most challenging, time-consuming, or tedious issues within your company that could greatly benefit from an AI…
Home Lab 07.2024: A Snapshot of its Current State
Servers: TOTAL:CPU 144 Core/ 288 + 16 CoreRAM 1.8 TB + 128 GBNVMe 95.44 TBSSD 4.4 TBHDD 4.5 TBNetwork 100 GbE + 10 GbE AI/ML/vGPU Server CPU: Intel® Xeon® Platinum 8490H 60 Core 120THCPU Cooler: CPU NOCTUA NH-U14S DX-4677Motherboard: GIGABYTE MS33-AR0Network NIC 2x RJ45 10GBase-T portsNetwork NIC 1x RJ45 1GBase-T portsNetwork PCI-E 5.0 x16 100GbE…
Struggling with Crucial T705 NVMe and VMware ESXi 8. U3 Compatibility
I was so confident that the Crucial T705 4TB PCIe Gen5 NVMe M.2 SSD would work with VMware ESXi without any issues that I didn’t even bother to Google or check. I wanted to get and test what it feels like and how much of a performance boost it gives when using the fastest (R:…
Resolving vCenter 8.0 U3 Upgrade Issues: Easy Solutions and Step-by-Step Troubleshooting
The ESXi server upgrade was quick and problem-free, but the vCenter upgrade was more like a roller coaster for my two vCenters. Both had some issues, but now I will explain in detail what the problems were and how I resolved them. First Error: Pre-upgrade check resultError:Source vSphere ESX Agent Manager (EAM) upgrade failed to…
AI, ML, and vGPU Server Build – Part 3: The Server Case as a Double-Decker Bus
Those two mining frames in the photo illustrate my idea well, but I plan to use a PC case and build a frame on top of it where I can easily attach different GPUs, essentially connecting 2 PC cases together. The server motherboard is secured inside the PC case, and the GPUs are openly mounted…
Home Lab 2024: A Snapshot of its Current State
Servers: TOTAL:CPU 144 Core/ 288 + 16 CoreRAM 1.6 TB + 128 GBNVMe 89.44 TBSSD 4.4 TBHDD 4.5 TBNetwork 100 GbE + 10 GbE AI/ML/vGPU Server CPU: Intel® Xeon® Platinum 8490H 60 Core 120THCPU Cooler: CPU NOCTUA NH-U14S DX-4677Motherboard: GIGABYTE MS33-AR0Network NIC 2x RJ45 10GBase-T portsNetwork NIC 1x RJ45 1GBase-T portsNetwork PCI-E 5.0 x16 (x16…
Dell S4112T-ON: Cooling and silencing the fan noise of a 100 GbE switch
I’ve had this switch for a long time and I’m very satisfied with it. They say a picture is worth a thousand words, so I’d rather share pictures of how I rebuilt a very noisy data center top rack switch. The switch originally came with a powerful 3-fan setup designed for data centers, which I…
AI, ML, and vGPU Server Build – Part 2: Server Noise, Temperature, and Power
Noise: Server not operating: I used the Tadeto SL720 Sound Level Meter for measurement. When only the ESX-Core Server, router, and switch are turned on in my room, I measured the sound and obtained the following results. My computer desk, where I sit: 34.7 dBCabinet on the other side of the wall: 34.7 dB Right…