Supposedly this is a Kimi k3 rival. Bit of a chonker, especially since they only released bf16 and fp8. So at launch this will be harder to serve than k3. No QAT on q4 means that someone with deep pockets (nvda?) will have to quant it, with plenty of calibration data. Should bring it ~1.3TB, so around k3 size.
License pretty similar to k3 with some caveats. Free to use for internal or <50M$ revenue / year. Limitations above that threshold for serving the model or services targeting coding / productivity agents.
Benchmarks are looking good, trading blows w/ opus4.8 and sol, generally 10-20p under fable. But that's neither here nor there w/ qwen, their benchmark to real world usage correlation has been iffy in the past.
The local model 3.8-27B announced for Friday, same time so ~48 hours from now. That'll be a bit more exciting for a lot more people, since 3.6 was quite good for local inference, and their 3.7-max -> 3.8-max shows a lot of improvement.
Llama.cpp can quantize without special training, but I'm not sure if any special model architecture support is needed to read it in the first place. If it can be converted to gguf at all and you know what tensors to target, it can get the full ternary bonsai treatment today.
NitpickLawyer 14 minutes ago [-]
Sure, but that's for "personal" serving. I meant for 3rd party providers. Usually we get a good indication on what it costs to host this, as the prices settle on open router. That's why I said it's tougher to serve than kimi k3 on launch. As a provider you'd do fp8 if the model creator didn't do QAT on q4, or until someone does a good calibrated nvfp4. And that's usually nvda :)
MrDrMcCoy 4 minutes ago [-]
That makes sense, but your specific phrasing precluded the possibility of non-QAT quantization.
binary132 25 minutes ago [-]
QAT is an optimizing quantization algorithm, not naive quant.
MrDrMcCoy 18 minutes ago [-]
Right, but the way they phrased it suggested that without QAT it could not be quanted at all.
esafak 31 minutes ago [-]
They've got the price and performance right. I would like to see Chinese models improve their reasoning efficiency in the next generation.
jingpostmedia 26 minutes ago [-]
[flagged]
dhx 22 minutes ago [-]
Also of interest: DeepSeek V4-Pro-0813 (1.6T-A49B) benchmark scores have apparently just been announced on the DeepSeek WeChat channel and they're sitting about Fable 5 level.[1]
the a little disappointing part is this is released in BF16. so i suppose no QAT was implemented.
l72 41 minutes ago [-]
> In particular, Qwen3.8-Max is the official version based on Qwen3.8-2.4T-A95B with more features, such as vision input & non-thinking support, 1M context length by default, official built-in tools, etc.
That is unfortunate, that the open weight model doesn't have vision support or the 1M context length...
wren6991 30 minutes ago [-]
People have had surprising success adding vision to open-weight LLMs that ship without it, like DSV4 Flash [1] or GLM-5.2 [2]. Given this model is already vision-trained I expect that approach will work well here.
The 1bit quant model is at an astonishing 397GB with 95B active per MOE. This literally puts Opus 4.5 performance level into a machine a normal person could buy, and still gets usable tokens/second.
The full lossless model BF16 is clocking at 4.9TB. The model card claims the model to be between Opus 4.8 and Fable 5. Again that's astonishing as getting a machine with 7TB RAM (with context + KV cache) is still within the realm of medium size companies.
Bad things: The open source version has its vision capability removed, and the context capped at 250k . I expect someone to bolt a Kimi 2.6 vision tower to it to restore the vision capability (at less performance of course). For context, I played around with extending the context to 600k for Qwen 3.5 397b, and the context remained stable up to around 480k. It'd be interesting to see if the same can be done to Q3.8 .
Also no out of the box DSpark/DFlash support. MTP is present so we should at least get some boost in TP speed.
auspiv 39 minutes ago [-]
Opus 4.5 level of performance is also accessible with deepseek-v4-flash-0731 (0731 being the july 31 update) which is much, much, much smaller. 2x RTX pro 6000 blackwell can run it. 4x can run it very comfortably
guardiangod 33 minutes ago [-]
I am running DS v4 flash 0731 lossless at 80t/s right now. It really is not at Opus 4.5 level (for my workload). I would say it's around 3.7 Sonnet, which is still pretty good, but other models such as GLM 5.2 are still leaps better. Of course I run DSv4 flash over GLM 5.2 for a few very good reasons, but intelligence is not 1 of them.
MrDrMcCoy 27 minutes ago [-]
Despite fitting into VRAM, I can't get DSV4 to run at usable speeds on my AMD hardware. The upcoming qwen3.8 27b greatly excites me, and I hope it can outperform Stepfun 3.7 Flash, which is the best thing I can run today.
Philpax 35 minutes ago [-]
What do you need the extra 2 for? Tensor parallelism?
arjie 32 minutes ago [-]
Longer context and more cache. The problem is that native format with DSpark enabled you have very little room on the VRAM.
Philpax 4 minutes ago [-]
I was under the impression that you could fit the full 1M context within the 192GB VRAM as a result of DeepSeek's various architectural advancements, but I'll grant that DSpark + a larger pool for concurrency may necessitate more VRAM, yes.
ilc 44 minutes ago [-]
To compare a 1 bit quant to the full fat model is misleading.
Honestly this model people at home can tinker with, if you have a big enough Mac. Maybe 4 Strix Halo/DGX Spark, and then at 1 bit quant? Nah.
Use the right sized model, for your hardware. You'll get better results.
guardiangod 42 minutes ago [-]
Extremely large 1 bit models are usually within 50-60% of KV divergence to lossless models. In this case I think the comparison to Opus 4.5 is a fair assessment.
Extremely large models don't suffer as much from quantization due to its weight topology also contains encoded information, so the loss of info from any one weight is somewhat mitigated.
ilc 34 minutes ago [-]
Any one weight, but all of them. And also crushing the architecture itself?
I wouldn't pick up 400gb of hardware to run in that mode. I might try it for fun, but even then you are looking at handling a 95GB active parameter set.
This is NOT a model for most home labs. I'm sure some can and will use it. But most, should steer clear.
ByteWarden 12 minutes ago [-]
More curious about how qwen3.8-27B performs. That's the size that I can run locally.
jephs 15 minutes ago [-]
"QwenSVGBench" elo 1713, pelicanmaxxxing confirmed?
theanonymousone 31 minutes ago [-]
Do we know if AA and DeepSWE benchmarks are on bf16 or fp8 quantisations?
simonw 46 minutes ago [-]
Is this the largest ever open weight model release by parameter count? I think it is.
NitpickLawyer 44 minutes ago [-]
No, Kimi k3 is 2.8T params. This is 2.4T params but ~5TB weights because it was released in bf16 and ~2.5TB for the fp8 version. Kimi k3 launched with QAT 4bit, so ~1.5TB weights.
PunchTornado 52 minutes ago [-]
The card looks almost too good to be true
43 minutes ago [-]
volf_ 56 minutes ago [-]
A ~5TB model.
Rendered at 16:24:02 GMT+0000 (Coordinated Universal Time) with Vercel.
License pretty similar to k3 with some caveats. Free to use for internal or <50M$ revenue / year. Limitations above that threshold for serving the model or services targeting coding / productivity agents.
Benchmarks are looking good, trading blows w/ opus4.8 and sol, generally 10-20p under fable. But that's neither here nor there w/ qwen, their benchmark to real world usage correlation has been iffy in the past.
The local model 3.8-27B announced for Friday, same time so ~48 hours from now. That'll be a bit more exciting for a lot more people, since 3.6 was quite good for local inference, and their 3.7-max -> 3.8-max shows a lot of improvement.
[1] https://www.reddit.com/r/LocalLLaMA/comments/1vmi0fg/deepsee...
https://openrouter.ai/deepseek/deepseek-v4-pro-0813
That is unfortunate, that the open weight model doesn't have vision support or the 1M context length...
[1] https://old.reddit.com/r/LocalLLaMA/comments/1vl6ior/i_gave_...
[2] https://huggingface.co/baseten/GLM-5.2-Vision-NVFP4
The 1bit quant model is at an astonishing 397GB with 95B active per MOE. This literally puts Opus 4.5 performance level into a machine a normal person could buy, and still gets usable tokens/second.
The full lossless model BF16 is clocking at 4.9TB. The model card claims the model to be between Opus 4.8 and Fable 5. Again that's astonishing as getting a machine with 7TB RAM (with context + KV cache) is still within the realm of medium size companies.
Bad things: The open source version has its vision capability removed, and the context capped at 250k . I expect someone to bolt a Kimi 2.6 vision tower to it to restore the vision capability (at less performance of course). For context, I played around with extending the context to 600k for Qwen 3.5 397b, and the context remained stable up to around 480k. It'd be interesting to see if the same can be done to Q3.8 .
Also no out of the box DSpark/DFlash support. MTP is present so we should at least get some boost in TP speed.
Honestly this model people at home can tinker with, if you have a big enough Mac. Maybe 4 Strix Halo/DGX Spark, and then at 1 bit quant? Nah.
Use the right sized model, for your hardware. You'll get better results.
Extremely large models don't suffer as much from quantization due to its weight topology also contains encoded information, so the loss of info from any one weight is somewhat mitigated.
I wouldn't pick up 400gb of hardware to run in that mode. I might try it for fun, but even then you are looking at handling a 95GB active parameter set.
This is NOT a model for most home labs. I'm sure some can and will use it. But most, should steer clear.