LLM Frontends

When I started experimenting with LLMs the UIs for them were in active development and now some of them are really good.

!- Jan - multiplatform ui for LLMs(jan-site_w678.jpg Jan - Frontend for LLMs - install)

Jan

• Jan(https://jan.ai/) is available for Windows, Linux and Mac.

Has dark, light and transparent themes.

!- Jan LLM frontend - main window(jan-self_w678.jpg Jan - Frontend for LLMs - example of reply to why selfhost)

Can connect to several existing backends like Anthropic, Cohere, OpenAI, NvidiaNIM, MistralAI etc, and host models on its own - see the Cortex section on the screenshot below - showing Jan downloaded and hosting locally Llama3 8b q4 and Phi3 medium (q4).

!- Jan LLM frontend - config options(jan-config_w678.jpg Jan LLM frontend - config options)

Pros (What I liked):

• Intuitive interface
• Ability to experiment with model temperature, topp, frequency and presense penalties and system prompts.
• Provides API server

Cons:

• Somehow slow on my ubuntu-based os. On windows it did run ok.
• Can connect to many backends, but all of them are managed. Would be nice to use Ollama option.
• Not many variants of the models available for self-hosting in Cortex. Not too many quantizations options either.
• Yes, Huggingface gguf is awesome. But I wanted
  • to reuse what ollama already downloaded loaded into VRAM
  • not host host same model everywhere

KoboldAI

KoboldAI

Very prominent one

Silly Tavern

Silly Tavern

Another very versatile one

LLM Studio

LLM Studio is not my favorite UI for LLMs, but it has better access to huggingface models.

Commandline Ollama

Yes, that’s also a user interfece, just a commandline one.

Need to run for llama3.1 LLM:

ollama run llama3.1

when done, send a command to exit ollama commandline:

/bye

cURL Ollama

Install cUrl you you haven’t done it yet

sudo apt-get install curl

To call local mistral nemo q8 llm hosted on ollama - create local file with the prompt p.json:

{
  model: mistral-nemo:12b-instruct-2407-q8_0,
  prompt: What is post-modernism?,
  stream: false
}

and now execute in bash terminal

curl -X POST http://localhost:11434/api/generate -d @p.json > p-result.json

the result will be in the file p-result.json

if you wan just to print the result:

curl -X POST http://localhost:11434/api/generate -d @p.json

Also:

Didn’t test those, but quite comprehensive list of LLM UIs:

• Streamlit
• Gradio
• Lobe Chat
• Text Generation WebUI
• Ollama WebUI
• Hugging Face Chat UI
• GPT-4All
• LocalGPT
• ChainFury
• Verba by Weaviate
• Chat UI Kit for React by ChatScope
• Reflex Chat
• Silly Tavern
• Amica
• Serge Chat
• LLM Multitool
• Sanctum AI
• KoboldAI
• AI Messenger
• Exui
• Spellbook Docker
• Lollms WebUI
• H2O GPT

Useful links

• Qwen3 Embedding & Reranker Models on Ollama: State-of-the-Art Performance
• Test: How Ollama is using Intel CPU Performance and Efficient Cores
• How Ollama Handles Parallel Requests
• Testing Deepseek-r1 on Ollama
• Install and configure Ollama
• Comparing LLM Summarising Abilities
• Compating different LLMs speed
• Self-hosting Perplexica - with Ollama
• LLMs comparison: Mistral Small, Gemma 2, Qwen 2.5, Mistral Nemo, LLama3 and Phi
• Ollama cheatsheet
• Markdown Cheatsheet
• Cloud LLM Providers