Hardware

GPU load monitoring applications: nvidia-smi vs nvtop vs nvitop vs KDE plasma systemmonitor.

Nvidia is about to release NVIDIA DGX Spark - little AI supercomputer on blackwell architecture with 128+GB unified RAM and 1 PFLOPS AI performance. Nice device to run LLMs.

Let’s compare prices for top-level consumer GPUs, that are suitable for LLMs in particular and AI overall. Specifically have a look at RTX 5080 and RTC 5090 prices. They have slighty dropped.

Just 3 month ago we didn’t see RTX 5090 in shops, and now here they are, but the prices are slightly higher then MRSP. Let’s compare the cheapest prices for RTX 5080 and RTX 5090 in Australia and see how it goes.

Top Automated system for some awesome job.

How PCIe Lanes Affect LLM Performance? Depending on the task. For training and multi-gpu inferrence - perdormance drop is significant.

I was hit hard by this issue. But you should investigate and test your PC if your BSOD is like mine. The cause was Intel’s CPU Degradation Issue in 13th and 14th generation.

I’ve got a theory to test - if utilising ALL cores on Intel CPU would raise the speed of LLMs? This is bugging me that new gemma3 27 bit model (gemma3:27b, 17GB on ollama) is not fitting into 16GB VRAM of my GPU, and partially running on CPU.

In the midst of the modern world’s turmoil here I’m comparing tech specs of different cards suitable for AI tasks (Deep Learning, Object Detection and LLMs). They are all incredibly expensive though.

When the Ollama server receives two requests at the same time, its behavior depends on its configuration and available system resources.

Installing ET-8500 on Windows is well documented in instruction. The ET-8500 Linux Driver installation is simple but not trivial.

Comparing prediction speed of several versions of LLMs: llama3 (Meta/Facebook), phi3 (Microsoft), gemma (Google), mistral(open source) on CPU and GPU.