Self-Hosting

Every API call feels like a simple transaction - until enough of them accumulate that your fine-tuning data, evaluation harnesses, and tool schemas are all shaped around one vendor, and switching stops being a routing change.

Ollama is one of the easiest ways to run a local language model, but convenience can conceal the moment when a local experiment becomes a shared inference service that needs better scheduling and observability.

Docker Compose and Podman Quadlet solve overlapping problems but come from different design centers, and choosing between them depends on whether you think in application stacks or Linux services.

Docker Compose on a Linux server should start on boot, stop cleanly on shutdown, and survive reboots without manual intervention.

Installing Docker on Ubuntu should be simple, but in practice several Docker-shaped options compete for the same command name, each with different packaging, upgrade behavior, and security implications.

APT failures are common on long-lived Ubuntu machines, and they usually appear after a release upgrade, a third-party repository change, a removed PPA, a manually installed .deb, or an interrupted package installation.

llama.cpp router mode is one of the most useful changes to llama-server in years. It finally gives local LLM operators something close to the model management experience people expect from Ollama, while keeping the raw performance and low-level control that make llama.cpp worth using in the first place.

This page is a practical reference for agentic LLM inference tuning (temperature, top_p, top_k, penalties, and how they interact in multi-step and tool-heavy workflows).

You already chat to Hermes Agent from your phone with text. Now you want to talk to it directly and get spoken replies back. That is usually the right move, especially if you already use Hermes as a persistent self-hosted assistant. Typing long prompts on a small screen is slow and error-prone

Most AI agent stacks still treat security as a post-demo fix. NemoClaw starts from the opposite assumption and makes isolation, policy, and routing day-zero defaults.

Personal knowledge management spans Obsidian, Logseq, DokuWiki, Zettelkasten, and PARA — the right choice depends on whether you want a local note graph, a self-hosted wiki, or an outliner-driven workflow.

The quiet loophole that powered a wave of agent experimentation is now closed.

Vane is one of the more pragmatic entries in the “AI search with citations” space: a self-hosted answering engine that mixes live web retrieval with local or cloud LLMs, while keeping the whole stack under your control.

Text Generation Inference (TGI) has a very specific energy. It is not the newest kid in the inference street, but it is the one that already learned how production breaks -

Here I am comparing speed of several LLMs running on GPU with 16GB of VRAM, and choosing the best one for self-hosting.

Australia has RTX 5090 stock. Barely. And if you find one, you will pay a premium that feels detached from reality.