Self-Hosting

Running LLMs locally is now practical for developers, startups, and even enterprise teams.
But choosing the right tool — Ollama, vLLM, LM Studio, LocalAI or others — depends on your goals:

Let’s compare prices for top-level consumer GPUs, that are suitable for LLMs in particular and AI in general. Specifically I’m looking at RTX-5080 and RTX-5090 prices.

The democratization of AI is here. With open-source LLMs like Llama, Mistral, and Qwen now rivaling proprietary models, teams can build powerful AI infrastructure using consumer hardware - slashing costs while maintaining complete control over data privacy and deployment.

Prometheus has become the de facto standard for monitoring cloud-native applications and infrastructure, offering metrics collection, querying, and integration with visualization tools.

Grafana is the leading open-source platform for monitoring and observability, transforming metrics, logs, and traces into actionable insights through stunning visualizations.

Kubernetes StatefulSets are the go-to solution for managing stateful applications that require stable identities, persistent storage, and ordered deployment patterns—essential for databases, distributed systems, and caching layers.

FLUX.1-dev is a powerful text-to-image model that produces stunning results, but its 24GB+ memory requirement makes it challenging to run on many systems. GGUF quantization of FLUX.1-dev offers a solution, reducing memory usage by approximately 50% while maintaining excellent image quality.

Configuring context sizes in Docker Model Runner is more complex than it should be.

Black Forest Labs has released FLUX.1-Kontext-dev, an advanced image-to-image AI model that augments existing images using text instructions.

Docker Model Runner is Docker’s official tool for running AI models locally, but enabling NVidia GPU acceleration in Docker Model Runner requires specific configuration.

Choosing the right headless CMS can make or break your content management strategy. Let’s compare three open-source solutions that influence how devs build content - driven applications.

I dug up some interesting performance tests of GPT-OSS 120b running on Ollama across three different platforms: NVIDIA DGX Spark, Mac Studio, and RTX 4080. The GPT-OSS 120b model from the Ollama library weighs in at 65GB, which means it doesn’t fit into the 16GB VRAM of an RTX 4080 (or the newer RTX 5080).

Docker Model Runner (DMR) is Docker’s official solution for running AI models locally, introduced in April 2025. This cheatsheet provides a quick reference for all essential commands, configurations, and best practices.

Running large language models (LLMs) locally has become increasingly popular for privacy, cost control, and offline capabilities. The landscape shifted significantly in April 2025 when Docker introduced Docker Model Runner (DMR), its official solution for AI model deployment.

In today’s computing landscape, virtualization has become essential for development, testing, and running multiple operating systems. For Linux users seeking a simple, intuitive way to manage virtual machines, GNOME Boxes stands out as a lightweight and user-friendly option that prioritizes ease of use without sacrificing functionality.

Multipass is a lightweight virtual machine manager that makes it easy to create and manage Ubuntu cloud instances on Linux, Windows, and macOS.