Strata is a free, open-source inference engine that runs Alibaba's 125-billion-parameter Qwen3.8-Flash-Next on an ordinary gaming PC — no subscription, no cloud, no data leaving your machine. Hosted on GitHub (github.com/Niko1221/Strata, MIT licence), it has grown to roughly 26,600 stars and a Hacker News front page, and it is the most immediately usable free alternative in this series.
First, a correction that matters: Strata is not a model. It is the engine that runs the model. Confusing the two is the most common misunderstanding in coverage of this project. The engine is Strata; the model is Qwen3.8-Flash-Next.
Strata: a 125B AI model on your gaming PC
Its own headline is "Run a 125-billion-parameter AI model on your own gaming PC". It is also not "Strata AI Solutions", an unrelated AI training and consulting firm, nor the same-named academic project on long-context caching.
The model underneath: Qwen3.8-Flash-Next, and the 125B vs 180B question
You will see two parameter counts for this model, and both are correct in different accountings. The Hugging Face model card states it plainly: "125B with 6B activated, plus 51B n-gram embedding and 4B MTP." Strata's headline counts the language model (125B). The full accounting adds the 51B n-gram embedding table and the 4B multi-token prediction layer, for about 180B parameters in total. Only about 6B parameters fire per token — that is the number that decides speed.
| Spec | Qwen3.8-Flash-Next |
|---|---|
| Total parameters | About 180B (125B language model + 51B n-gram embedding + 4B multi-token prediction) |
| Activated per token | About 6B |
| Architecture | Mixture of experts; preview of the Qwen4 architecture (Gated DeltaNet + Qwen Sparse Attention) |
| Layers | 48, arranged as 12 blocks of 3 Gated DeltaNet layers + 1 Qwen Sparse Attention layer |
| Experts | 512 per layer (24,576 in total); 10 routed + 1 shared active per token |
| Hidden dimension | 2,560; vocabulary 248,320 tokens |
| Context | 262,144 tokens native, extensible to 1,000,000 via YaRN scaling |
| Modality | Text, image and video in; text out; vision encoder included |
| Thinking | Hybrid thinking with adjustable effort (xhigh, medium, low); reasoning traces preserved across turns |
| Licence | Qwen Community Licence 1.0 for the weights; the Strata engine itself is MIT |
| Serving | Transformers, vLLM, SGLang, TokenSpeed; production serving via SGLang, KTransformers or vLLM |
The architecture is the point. Qwen Sparse Attention selects micro-blocks instead of individual tokens, which cuts long-context latency. Gated Residual runs four branches with data-dependent gates instead of one residual stream. And the n-gram embedding indexes 20 million bigrams and trigrams into a 51B lookup table at layer 2 — parameters that are cheap to look up and easy to offload, which is exactly what Strata exploits.
How a 125B model fits on a 12 GB card
A full-precision Qwen3.8-Flash-Next checkpoint is 172.78 GiB in FP8 and 335.28 GiB in BF16 — genuinely multi-GPU territory (the official recipe calls for two GB300-class GPUs at minimum). Strata does not try to fit that. It changes where each part of the model lives:
Where each part of the model lives on your PC
The split that makes a 125B language model run on a gaming PC: the GPU holds what is hot, RAM holds every expert, and the SSD holds the n-gram lookup table.
The GPU keeps the hot experts cached, plus attention, routers, shared experts, the output head and the multi-token prediction layer. System RAM holds all 24,576 experts pinned, and CPU kernels (AVX2/AVX-512) evaluate the experts that missed the GPU cache. The 28.8 GB n-gram table stays on the SSD, with only a few rows read per token through the OS page cache. Weights are compressed with GSQ-RCO quantization from ISTA-DASLab, and the multi-token prediction layer drafts the next few tokens while the main model verifies them at once, worth roughly 1.6 to 1.8 times faster responses. Because a mixture-of-experts model only fires about 6B parameters per token, compute stays cheap even though the full weight set rides along.
The benchmarks, read honestly
The benchmark table below is Qwen's own vendor-reported figures from the Hugging Face model card, not independent reproductions. That distinction matters twice over: the scores describe the full-precision model, not the quantized build you will actually run locally, and two methodology notes deserve daylight. DeepSWE 1.1 reports the highest score across two harnesses (Claude Code and mini-SWE-agent), and SWE-bench Pro was run on a refined version of the benchmark after the Qwen team corrected what it calls problematic tasks and re-evaluated every baseline. HLE was judged by GPT-4o rather than the benchmark's default grader.
| Benchmark | Qwen3.8-Flash-Next | Qwen3.8-27B | Qwen3.7-Plus | DeepSeek-V4-Flash-0731 | Claude Opus 4.6 (Max) |
|---|---|---|---|---|---|
| SWE-bench Pro | 62.5 | 61.7 | 55.8 | 56.0 | 53.4 |
| DeepSWE 1.1 | 58.7 | 42.2 | 16.5 | 54.4 | — |
| SWE-bench Multilingual | 81.0 | 73.8 | 75.8 | — | 77.5 |
| CoWorkBench | 73.9 | 70.7 | 65.1 | 45.1 | 68.2 |
| JobBench | 55.7 | 33.4 | 27.6 | 41.3 | 36.6 |
| Toolathlon | 73.5 | 67.1 | 50.6 | 70.3 | — |
| IFBench | 81.3 | 79.5 | 79.1 | 79.2 | 62.5 |
| GPQA Diamond | 91.7 | 89.2 | 90.3 | 90.8 | 91.3 |
| HLE | 35.9 | 30.8 | 34.7 | 33.8 | 40.0 |
| LiveCodeBench v6 | 91.9 | 90.3 | 89.6 | 90.6 | 88.8 |
| NL2Repo-Bench | 48.1 | 42.3 | 41.1 | 54.2 | 47.6 |
Agentic and coding benchmarks: Flash-Next against its comparison set
Scores are Qwen vendor-reported figures from the Qwen3.8-Flash-Next model card. "n/a" means the card publishes no score for that model on that benchmark. DeepSWE 1.1 is the highest score across two harnesses; see the caveats in the text.
Read the pattern, not just the wins. Flash-Next leads or ties its comparison set on most agentic and coding benchmarks despite activating fewer parameters than every model it is compared against except DeepSeek-V4-Flash. The gap is largest on tool-heavy, long-horizon work (JobBench 55.7 against 33.4 for the dense 27B model) — exactly what the architecture is built for. It trails where raw frontier reasoning dominates: HLE goes to Claude Opus 4.6 Max (40.0 against 35.9), and repo-level code generation on NL2Repo-Bench goes to DeepSeek-V4-Flash (54.2 against 48.1). On vision tasks the same shape holds: AndroidWorld 84.5, RealWorldQA 88.5, MathVision 95.7 with code-interpreter help, and OSWorld 2.0 partial score 52.3 — top scores on 8 of 10 published multimodal benchmarks.
The single biggest caveat, and the one community critics raise most: aggressive quantization trades real quality for a model that fits your RAM. The 62.5 on SWE-bench Pro describes the full model. A Q2 build or an expert-pruned Coder build on your PC will not score that. Treat the table as the ceiling, not the promise.
Parameters, compared: total size is not the same as cost
The comparison below is the one that explains why Strata matters. Total parameters decide how much memory a model needs. Activated parameters decide how much compute each token costs. Flash-Next is the extreme case: a huge total footprint with a tiny activated slice, plus a lookup table that can live on disk.
| Model | Total parameters | Activated per token | What it means for your PC |
|---|---|---|---|
| Qwen3.8-Flash-Next | About 180B (125B language model) | About 6B | Needs RAM + disk for the weights, but only 6B of compute per token — hence Strata |
| Qwen3.8-27B (dense) | 27B | 27B | Every parameter fires; FP8 build about 28 GB, fits one 80 GB card, or a single high-end consumer card at lower precision |
| Qwen3.7-Plus | 397B | 17B | Bigger total and bigger activated slice than Flash-Next, yet loses most agentic benchmarks to it |
| DeepSeek-V4-Flash-0731 | 284B | 13B | More than double Flash-Next's activated parameters, still behind on SWE-bench Pro and office work |
| Meta Muse Glimmer 30B | About 30B (dense) | 30B | A dense 30B model needs roughly 24–32 GB of VRAM to run locally — a different local-AI bargain (see our Glimmer breakdown) |
| Qwen3.6-35B-A3B | 35B | About 3B | The smaller sibling family that Strata 0.1.45 just gave vision support; the low-RAM alternative |
Strata speed on real hardware
These are author-measured figures from the Strata README: 4K answers, 32K prompts, on two ordinary gaming PCs. A token is about three quarters of a word, so 60 tokens per second is already faster than you can read.
| Hardware and size | Writes answers (tokens/sec) | Reads your prompt (tokens/sec) |
|---|---|---|
| RTX 5070 (12 GB) — Q2_0 | 94 | 2,650 |
| RTX 5070 (12 GB) — IQ2_XS | 79 | 2,090 |
| RTX 5070 (12 GB) — IQ3_XXS | 62 | 1,750 |
| RTX 5070 (12 GB) — IQ3_S | 53 | 1,620 |
| RTX 5070 (12 GB) — Coder | 55 | 2,180 |
| RX 9070 XT (16 GB) — Q2_0 | 60 | 1,160 |
| RX 9070 XT (16 GB) — IQ2_XS | 52 | 1,110 |
| RX 9070 XT (16 GB) — Coder | 44 | 1,420 |
Strata write speed by hardware and model size (tokens per second)
Author-measured figures from the Strata README (4K answers, 32K prompts; Q2_0 on engine 0.1.36, other rows on 0.1.26). An RTX 3090 (24 GB) is estimated by the project at roughly 100-140 tokens per second and is not charted because it is an estimate, not a measurement.
Three things to know about these numbers. First, smaller sizes are faster and slightly less smart; IQ3_S is the best quality and the slowest of the main sizes. Second, long context costs speed: at 128K context the same RTX 5070 setup runs roughly 46–74 tokens per second depending on size, and the first message of a chat reads at about 1 minute per 30,000 tokens (follow-ups start in seconds). Third, more VRAM helps a lot: the project estimates an RTX 3090 (24 GB) at roughly 100–140 tokens per second, and community reports put it near 70 tokens per second at 128K — both labelled estimates, not measurements. Version 0.1.45 pushes further: decode up to about 40% faster, an AMD R9700 card jumping from 83 to 118 tokens per second, and a 500K-token prompt reading 79% faster on AMD hardware.
Strata system requirements
| Requirement | What you need |
|---|---|
| Graphics card | NVIDIA GeForce RTX 20, 30, 40 or 50 series, or AMD Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700, or RX 6800 / 6900 series — 12 GB of VRAM or more. Two or three cards can share the model. |
| RAM | 32 GB or more. RAM decides which model size fits; 64 GB runs every size. See the picker below. |
| Disk | About 80 GB free; an SSD is strongly recommended because the first start is much faster. |
| System | Windows 10 / 11 or Linux, plus a current graphics driver from NVIDIA or AMD. |
| First start | Your PC can be slow or unresponsive for 1–3 minutes while Strata loads 35–55 GB into RAM. This is normal — wait, do not close the window. |
| Experimental platforms | Older cards (Tesla P40 / V100, GTX 10, Radeon VII / MI50, RX 6700 XT, RX 5500 XT); Intel Arc built from source on Linux; Apple Silicon Macs M1–M5 via the Metal engine; AMD Ryzen AI Max; older CPUs without AVX2 (work, but slowly). |
| Your RAM | Take | Why |
|---|---|---|
| 32 GB | Coder | It fits 32 GB and is made for code (with a 24 GB card, Q2_0 and IQ2_XS run too) |
| 48 GB | IQ2_XS (or Q2_0, the fastest) | The larger sizes do not fit |
| 64 GB | IQ2_XS (recommended), or IQ3_XXS / IQ3_S | Every size fits; IQ3_S is the best and the slowest |
| 96 GB or more | IQ3_S, or Unsloth UD-IQ4_XS (~4-bit) | Room for the largest sizes with everything else open |
A note for readers on a 32 GB machine with an RTX 3080: RAM-wise, Coder is your fit — but the graphics card still needs 12 GB or more of VRAM. A 10 GB card is below Strata's stated minimum even if the RAM fits, so check which 3080 you own before downloading 70 GB. If your machine has only 8 to 16 GB of RAM, the project's own advice is blunt: you are outside the useful range for this model and better off with a smaller dense model.
Packages, downloads and installation
There is no app store here and no subscription. The packages are: the engine, the model size you pick, and the API your apps talk to.
| Package | What you get | How to get it |
|---|---|---|
| Strata engine | The inference engine, installer, browser app (Chat, Monitor, About), MCP server, bench tool | Download and unzip the repo, or git clone it; Windows: double-click START-HERE.bat; Linux: run ./setup.sh; Mac: install Command Line Tools once (xcode-select --install), then ./setup.sh (builds the engine in about 7 minutes) |
| Model sizes (installer menu) | Q2_0 (fastest), IQ2_XS, IQ3_XXS, IQ3_S (best quality), Coder (half the experts removed, 91% of the full model's SWE-bench Verified per its authors, weaker outside code), Swift 1.5 (shorter thinking, about the same quality) | Picked in the installer (press Enter for the recommended answer); the model download is about 70 GB and resumes if it stops |
| Unsloth sizes (advanced) | UD-IQ4_XS (~4-bit, 94 GB download); UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL, Q8_0 (4- to 8-bit, 93–95% the same next token as Q8_0; about 88 GB of RAM for UD-Q4_K_XL up to about 136 GB for Q8_0) | With less RAM than the size needs, Strata reads part of it from SSD while answering — much slower (7–8.5 tokens/sec for UD-Q4_K_XL on a 64 GB PC) |
| OrcaRouter Uncensored IQ3_XXS | An uncensored fine-tune | Set up by hand; not in the installer's menu |
| API endpoints | OpenAI-compatible API, Anthropic-compatible API, OpenAI Responses API | Base URL http://127.0.0.1:8080/v1 (any API key and any model name work); Anthropic apps use /v1/messages; Codex CLI uses /v1/responses |
| Coding agents | Cursor, Claude Code, Codex CLI, GitHub Copilot and other OpenAI-style apps | Point the app at the local base URL above, or paste the project's AI-setup prompt into your coding assistant and let it follow docs/AI_SETUP.md (it checks your hardware, picks a fitting size, installs and starts it) |
| Extras | Dockerfile and docker-entrypoint in the repo; multi-GPU setups; bench reports (BENCH.bat / ./bench.sh, 5–10 minutes); updates via UPDATE.bat / ./update.sh without starting the model | Docs live in the repo: docs/INSTALL.md, docs/MODELS.md, docs/TROUBLESHOOTING.md |
Using it day to day: open http://127.0.0.1:8080 in your browser for chat, a live Monitor of model, GPU, CPU and RAM, and an About page with settings and addresses. Thinking is adjustable (off, low, medium, high); off is fastest, high is best for hard questions. Pictures are optional at setup. By default Strata answers one request at a time; version 0.1.45 allows up to eight parallel conversations, with the honest warning that slots steal VRAM from the expert cache, so a single request can actually run slower — worth it for several users or agents at once, not for one chat. From another PC or your phone, run with --host 0.0.0.0 and always set an API key.
What is new in Strata 0.1.45
The latest release (October 2026) keeps the flagship model untouched — answers are byte-identical to 0.1.44.1 across every GPU family tested — and spends its changes elsewhere. Vision arrives for the smaller Qwen3.6-35B-A3B family (setup asks about images, downloads a roughly 0.9 GB image projector, launches with --vision; on NVIDIA cards under 24 GB it uses the CPU encoder to keep VRAM free). Decoding gets up to about 40% faster with new kernels verified against 18,360 real-activation comparisons with zero mismatches. Parallel requests go up to eight. And a set of opt-in STRATA_* variables plus a CUDA-only build for multi-GPU servers cut time-to-first-token from 20.6 seconds to 2.0 seconds on a 4× R9700 rig. Gaps in this release, stated plainly by coverage: Windows hardware, Intel Arc and the RTX 5070 were skipped, and Volta/Pascal support moved to 0.1.46.
Strata: the honest limitations
Strata is the most usable project in this article, but it is not magic. The benchmark table describes the full-precision model, not your quantized local build. The first start freezes your PC for minutes while 35–55 GB loads. The download is about 70 GB. The engine is MIT, but the weights are under the Qwen Community Licence — if you host it as a service or build a commercial product on top, you need a separate licence from Qwen. And one genuine supply-chain quirk: the README offers a prompt that tells your AI assistant to run the setup itself, which commenters compared to piping a script into bash. Read what it does before you paste it.
Related reading: our breakdown of Meta's open 30B agentic model, Muse Glimmer — a dense model that needs 24–32 GB of VRAM, where Strata's mixture-of-experts design needs RAM and disk instead. Also in this series: AnyPS5 and Redox OS.
Is Strata a model or something else?
Strata is an inference engine, not a model. The model it runs is Qwen3.8-Flash-Next (125B language model, about 6B activated per token, plus a 51B n-gram embedding table and a 4B multi-token prediction layer). Confusing the engine with the model is the most common error in coverage.
Can I run Strata on an RTX 3080 with 32 GB of RAM?
RAM-wise, yes — 32 GB fits the Coder size. But Strata's stated minimum is 12 GB of VRAM, so a 10 GB RTX 3080 is below the graphics requirement even though the RAM fits. Check which 3080 you own before downloading about 70 GB. A 12 GB 3080 meets the stated minimum.
Which Strata model size should I pick for my RAM?
32 GB: Coder (with a 24 GB card, Q2_0 and IQ2_XS also run). 48 GB: IQ2_XS or Q2_0, the fastest. 64 GB: IQ2_XS recommended, or IQ3_XXS / IQ3_S, the best quality and the slowest. 96 GB or more: IQ3_S or Unsloth UD-IQ4_XS. If your machine has only 8 to 16 GB of RAM, the project's own advice is that you are outside the useful range for this model — pick a smaller dense model instead.
Sources
- Strata on GitHub: https://github.com/Niko1221/Strata
- Qwen3.8-Flash-Next model card: https://huggingface.co/Qwen/Qwen3.8-Flash-Next
- Strata 0.1.45 release coverage: https://www.linuxcompatible.org/story/strata-0145-opensource-ai-engine-gets-a-speed-and-vision-boost/
- Qwen3.8-Flash-Next architecture and GPU requirements: https://akash.network/the-bid/qwen3-8-flash-next-architecture-gpu-requirements/
- Strata overview: https://startupfortune.com/strata-lets-a-125-billion-parameter-model-run-on-a-gaming-gpu/
Be the first to share a useful question or thought.