Testing Qwen3.8-Flash-Next: Comparison with Production Models and the Importance of Multiple Runs

With the release of the preview version of Qwen3.8-Flash-Next, it has become possible to evaluate the new architecture that will serve as the foundation for the upcoming Qwen4 generation. The model uses a hybrid approach: out of a total of 125 billion parameters, only 6 billion are activated per token, while another 51 billion are offloaded to n-gram embeddings loaded selectively from disk storage without calculations under the classic dense scheme.

Testing was conducted on an RTX PRO 6000 graphics card with an eye toward viability on RTX 5090-class configurations with 32 GB VRAM. In parallel, previous measurement data was corrected: the production server uses the official Qwen3.8-27B-NVFP4A16 with MTP speculation enabled, rather than a third-party test checkpoint.

Benchmark Results

To minimize error margins, each configuration was run three times. The summary results are presented in the table:

Accuracy and Quantization Analysis

Flash-Next and production 27B metrics fall within the statistical margin of error: 0.786 vs 0.769. Furthermore, increasing quantization precision from Q3 to Q4 yielded no score improvement (0.785 vs 0.786), while leading to increased VRAM usage.

The Ornith-35B MoE model trails the leaders by about 0.05 points, but leads in generation speed by more than 3.4x (238 tokens per second vs ~70–79 for competitors). The compact Ornith-9B predictably placed last with a score of 0.590 and higher result variance.

Inference Nuances and Ecosystem Status

During overnight testing, issues arose with the availability of external LLM judge APIs (server overload errors), requiring isolation and reruns of failed tasks.

Currently, running Flash-Next is mainly supported via llama.cpp. Production-grade engines like vLLM and SGLang have not yet received full support for the model's new components—namely the Gated DeltaNet mechanism, sparse attention, and n-gram tables. Due to the lack of mature infrastructure and MTP speculation in production environments, an immediate transition of production services to the new architecture involves trade-offs.