The Efficient Frontier of Open Models: Finding the Sweet Spot in 2026

In September 2026 the open-model efficient frontier sits between 25B and 34B parameters: near-frontier agentic work on one 24 GB GPU, at a fraction of 70B-class hardware.

That band shows up in deployments, not in a parameter-count slogan. Public leaderboards, single-GPU runs, and monthly API bills converge on models that stay close to frontier tool-use while the weights and the KV cache still fit hardware you can rent or own.

This page is the decision hub for that choice, inside the LLM performance cluster. It walks from small instruct models to 70B-class dense weights, then stops on the two architectures that define the frontier right now: Qwen3.8-27B, which is comfortable on one 24 GB card, and Llama 4 Scout, whose compute looks modest only because most of its experts stay idle. Token-per-second tables stay on the benchmark pages linked from each section.

By the end you should know which size to test first, how much VRAM the weights and the cache actually take, how a rented RTX 4090 compares with current Claude Sonnet token prices, and when a larger model is still the right call.

Why public benchmarks overstate production transfer

A model can sit within a couple of points of a frontier API model on MMLU or HumanEval and still fail the job you are shipping. Multi-tool flows drop the second tool call, refusal calibration drifts toward over-refusal on medical-adjacent queries, and P99 latency under burst load doubles. The leaderboard gap looked small because those benches measure capability on a fixed prompt set. They do not measure transfer onto your tools, your documents, and your latency budget.

The efficient frontier is the set of models that hold up on your workload at a cost you can keep paying. That takes a paired comparison on production-like tasks, with three signals recorded the whole way: tool-call success, refusal behaviour, and P99 latency. Future AGI’s write-up on cheap-frontier substitution makes the same point from the evaluation side: a small public-benchmark gap is not a licence to swap models.

Treat the scores below as a map of where to start the bake-off. They are dated to September 2026, and several headline coding numbers are vendor-run rather than independent.

Where 8B, 30B, and 70B actually sit

At the low end, an 8B-class instruct model will follow a short, explicit instruction and write a small function. Ask it to debug a failing CI pipeline across several services, recover when the first tool call returns an error, and keep the plan intact, and the run falls apart. That is a workload failure, not a leaderboard failure.

The middle of the range is where general-purpose agentic work currently clears a single prosumer GPU. Qwen3.8-27B is the reference dense model. Architectures in the same size band, including the Qwen 30B-class MoE compared in the Qwen3 30B vs GPT-OSS 20B head-to-head, are the ones to test before you spend on anything larger.

At the high end, a dense 70B model at 4-bit is on the order of 35–40 GB of weights before any KV cache. It does not fit a 24 GB card. You are into a 48 GB GPU, an 80 GB data-center card, or a two-GPU split, and the extra quality over a well-tuned 27B model is often marginal on coding agents, document pipelines, and support bots. Closed frontier APIs such as Claude Opus sit in a different budget entirely: you pay per token, and you do not carry the weights at all.

Before you pick a checkpoint, confirm the card and the free memory. Model weights are only part of the footprint. Context (the KV cache) sits on top.

# GPU name and free VRAM before picking a quant
nvidia-smi --query-gpu=name,memory.total,memory.used --format=csv

# after loading: confirm the process stayed on the GPU
nvidia-smi

If layers spill to CPU, generation throughput collapses. You can see the split in memory.used on the GPU and in the runtime’s offload log. For what dense and MoE checkpoints actually deliver on a 16 GB card, the measurements live in the 16 GB VRAM llama.cpp benchmarks and the Ollama comparison on a 16 GB RTX 4080. Those pages own the speed tables. This one only needs the fit decision.

Qwen3.8-27B: the 24 GB reference point

Alibaba’s Qwen3.8-27B is a dense model, not a mixture of experts. On Artificial Analysis it scores about 51 on the Agentic Index (the rounded figure; the underlying value is 50.9) and about 52 on the Intelligence Index at maximum reasoning effort. Those are different measurements. The Agentic Index is tool-use and multi-step tasks. The Intelligence Index is a broader capability blend. MindStudio’s hands-on bench frames the agentic score as just behind Kimi K2, a vastly larger mixture-of-experts model. Qubrid’s breakdown is the one to read next to it: the margin over the next model is under a point, and Qwen’s own SWE-bench Pro figure of 61.7 has not been reproduced as an independent run. The interesting result is the shape of the scores. A 27B model converts a compact budget into planning and tool use unusually well, and it does not lead a broad knowledge suite.

The architecture is why the long context is affordable. Of 64 layers, only 16 use full attention. The other 48 are Gated DeltaNet layers, a linear-attention design from the Gated Delta Networks line of work, and they keep a fixed recurrent state instead of a per-token key/value history. At FP16 the full-attention layers cost about 64 KB of KV cache per token. A conventional stack with the same layer count would cache roughly four times that. Locally Uncensored’s quant table puts Q4_K_M weights at 17.1 GB and IQ4_XS at 15.7 GB. Hardware Corner’s llama.cpp measurements on a 24 GB GPU landed near 18 GB at short context and about 22 GB at 64K, which is the practical target on an RTX 4090. A 16 GB card can hold an IQ4-class quant of the weights and almost no context. If that is your machine, stay with the Qwen 3.5 and 3.6 runs already measured on 16 GB, including Qwen 3.6 27B MTP versus standard decoding. How to budget the cache itself is covered in KV cache on 16 GB GPUs.

Hands-on reports, MindStudio’s among them, also find the model usable for coding, image analysis, and agent loops, which is the part a single index cannot show. Vision adds a small projector file on top of the text weights. Plan for it if the workload includes screenshots.

Llama 4 Scout: active parameters are not VRAM

Meta’s Llama 4 Scout is a different kind of efficiency. The Llama 4 model card lists 17 billion active parameters and 109 billion total, with 16 experts, native image input, and a 10 million token context window. Meta’s launch post says the INT4 checkpoint fits on a single H100. Decode compute tracks the 17B that fire for each token. Memory does not. Every expert still has to be resident, so the VRAM bill looks like a 100B-class model even though the FLOPs look like a 17B model.

That is the correction worth making before you budget a machine. Scout does not “run like a 3B model.” Active parameters set speed. Resident parameters set whether the card can hold the weights. On a 24 GB GPU, Qwen3.8-27B is the model that fits. Scout is the model that makes sense once you already have an 80 GB card and want 17B-class compute with a much larger expert pool and a very long context.

A June 2025 paper, Can Mixture-of-Experts Surpass Dense LLMs Under Strictly Equal Resources?, argues that MoE can beat a dense model when the comparison holds total compute and data fixed and the activation rate sits in a workable band. The practical reading is narrower than the headline. MoE wins on cost only after you have paid for the memory that stores the idle experts. If you are already serving a model of this size and want more tokens per second without changing the weights, speculative decoding is the neighbouring technique: a draft path proposes tokens and the main model verifies them.

Phi-4: the small-model exception for math

Microsoft’s Phi-4 is a 14B dense model. In the December 2024 technical report, simple-evals scores it at 80.4 on MATH, ahead of the GPT-4o figure in the same table, with a context window that the report extends to 16K. A Q4_K_M quant is commonly run in about 8 GB. That is real efficiency, and it is narrow. Phi-4 was trained for STEM question-answering. It is the model to try when the job is math or short technical reasoning and the machine is small. It is not the 2026 default for multi-tool agents, long documents, or vision. The frontier for that work is still the 25–34B band above, or Scout when the GPU is an H100-class card.

Self-hosted GPU hours versus per-token API prices

Prices move. These are mid-September 2026 figures, and they will not survive unchanged into 2027.

GPU Finder, checked on 18 September 2026, showed an in-stock RTX 4090 at about $0.26 per GPU-hour on Vast, with RunPod community cloud listed at $0.34. The six-month on-demand floor for a single 4090 sat between $0.11 and $0.60. At $0.34 and 730 hours in a month, a card that stays up all month is about $248 before tax, storage, and egress. At the $0.26 in-stock quote the same month is about $190. An earlier planning figure of $0.53 per hour is inside that six-month range, but it overstates what marketplace stock was actually asking in the second half of September.

On the API side, Anthropic’s Sonnet 5 pricing notes list $2 per million input tokens and $10 per million output tokens. Sonnet 4.6 remains at $3 and $15. BenchLM’s rate card, updated 3 September 2026, shows the same split. A workload of 100 million input tokens and 10 million output tokens is $300 on Sonnet 5 and $450 on Sonnet 4.6.

Put next to a 4090 rented all month at $248, that example does not say “self-hosting is always cheaper.” It says the two bills meet in the same neighbourhood for this volume. Sonnet 5 at a 10:1 input-to-output ratio costs $30 per 10 million input plus 1 million output. The $248 rental covers about eight of those blocks: roughly 80 million input tokens and 8 million output tokens. Above that, the rented card pulls ahead, provided you can keep it busy. Below it, the API is the cheaper line, and you are not paying for an idle GPU. Owned hardware replaces the rental with electricity and depreciation. The strategies around that bill — budgeting, caching, and fallbacks — are in cost optimization for LLM systems. The reason a lower token price can still be the wrong architecture is data gravity and API lock-in.

When a larger model is still the right call

The 25–34B band is the default for production workloads that repeat: coding agents, support bots, document processing, extraction, and automation. It is the wrong default in a few situations.

• Frontier reasoning on problems the smaller model has already failed in a paired test.
• Prose where the difference in voice is the product, and volume is low enough that token price is noise.
• Work that needs broad coverage across several specialist domains at once, where the 27B model keeps missing the same class of fact.
• Low-volume, high-stakes decisions, where the cost of a wrong answer exceeds a year of model rental.

In those cases, move up to a 70B-class open model or a frontier API, and keep the same three production signals. If the larger model does not move tool-call success, refusals, or P99 in the direction you care about, the extra size is not buying anything you can use.

How to pick a point on the frontier

Use this order. It is cheaper to discover that a 27B model is enough than to discover that an 80 GB card was unnecessary.

1. Start from the workload, not from the largest open weights you can download. Single-shot instructions and math can start at 8–14B. Multi-step tool use starts at Qwen3.8-27B if you have 24 GB, or at the best 16 GB quant you have already measured if you do not.
2. Separate active parameters from resident parameters. Scout’s 17B active figure is a compute claim. The 109B total is the VRAM claim. Budget the second number.
3. Quantize, then re-check memory. Q4_K_M of Qwen3.8-27B is a 24 GB conversation, with about 64K of context before the card is full. Run nvidia-smi after the load, at the context length you will actually serve.
4. Price the month, not the token. Compare 730 hours of the GPU you would rent with the input and output mix you already log. If you are under the break-even above, stay on the API until the volume arrives.
5. Decide on your own tasks. A paired run that records tool-call success, refusal behaviour, and P99 under burst is the test that public benches cannot replace.

The useful question in 2026 is which weights fit the card, the context, and the monthly bill while still clearing your own tasks. For a lot of agentic work, that point is a 27B hybrid model on one 24 GB GPU. Scout is the same idea at data-center memory: less compute per token than the parameter total suggests, and no discount on the VRAM.

References

1. MindStudio. “Qwen 3.8 27B Benchmarked: Agentic Index, Vision, and Reasoning Tests.” https://www.mindstudio.ai/blog/qwen-3-27b-local-benchmark
2. Qubrid AI. “Qwen3.8-27B Benchmarks: Official and Independent Results.” https://www.qubrid.com/blog/qwen38-27b-benchmarks-official-and-independent-results
3. Hardware Corner. “We Tested Qwen3.8 27B: How Much GPU and VRAM Do You Really Need?” https://www.hardware-corner.net/qwen3-8-27b-hardware-tests/
4. Locally Uncensored. “How to Run Qwen 3.8 27B Locally: VRAM, Quants and the Template Trap.” https://locallyuncensored.com/blog/how-to-run-qwen-3-8-27b-locally.html
5. Malik, Umesh. “Qwen3.8 27B VRAM: how to fit 262K context in 16 GiB, not 64.” https://umesh-malik.com/blog/qwen3-8-27b-vram-kv-cache-math
6. Meta AI. “The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation.” https://ai.meta.com/blog/llama-4-multimodal-intelligence
7. Meta. “Llama 4 model card.” https://github.com/meta-llama/llama-models/blob/main/models/llama4/MODEL_CARD.md
8. arXiv. “Can Mixture-of-Experts Surpass Dense LLMs Under Strictly Equal Resources?” June 2025. https://arxiv.org/html/2506.12119v1
9. Yang, S., Kautz, J., Hatamizadeh, A. “Gated Delta Networks: Improving Mamba2 with Delta Rule.” ICLR 2025. https://jankautz.com/publications/GatedDeltaNet_ICLR25.pdf
10. Abdin, M., et al. “Phi-4 Technical Report.” arXiv:2412.08905, December 2024. https://arxiv.org/html/2412.08905v1
11. GPU Finder. “RTX 4090 GPU Rental Prices & Live Availability.” Checked 18 September 2026. https://gpufinder.dev/gpu/rtx-4090
12. Anthropic. “What’s new in Claude Sonnet 5” (pricing: $2 / $10 per million tokens). https://platform.claude.com/docs/en/models/sonnet-5/whats-new-sonnet-5
13. BenchLM. “Claude API pricing.” Updated 3 September 2026. https://benchlm.ai/anthropic/api-pricing
14. Future AGI. “Evaluating Cheap Frontier Models in 2026: Substitution Without a Quality Cliff.” https://futureagi.com/blog/evaluating-cheap-frontier-models-2026
15. Northflank. “Qwen3.8-27B: Performance, benchmarks, GPU requirements & how to run it.” 17 August 2026. https://northflank.com/blog/qwen3-8-27b-performance-benchmarks-gpu-requirements-and-how-to-run-it