SOURCE UPDATE · 12 OCT 2026

One Ultra, two engines: faster reading or writing?

JV's original controlled comparison, first public on 11 October, used an M5 Ultra 80 GPU cores / 256GB, the same Qwen3.8-Flash-Next IQ4_NL GGUF and Q8_0 MTP head on both engines. The operating system was macOS 27.0; the engines were llama.cpp b11538 and gmlx 0.4.23.

4,096-TOKEN PREFILL · GMLX3,348PP tok/s · llama.cpp 2,150

SHORT-CONTEXT MTP GENERATION · LLAMA.CPP99.9TG tok/s · gmlx 83.6

33,281-TOKEN PROMPT11.3 sgmlx · llama.cpp 18.9 s

The 4K prefill results are built-in benchmarks. MTP-on generation used four prompts, three seeds and 512 requested output tokens per run; the exact generation starting context is unpublished and the author describes the speed as short-context. The 33,281-token prompt was a separate recall test: both engines found two hidden codes. Its elapsed prompt-reading times do not establish TG at 33K. The author also reports MTP-off generation of 57.0 tok/s for llama.cpp via API versus 64.0 for gmlx via its built-in benchmark, so those methods differ. Cache reuse and reasoning-token counts were not reported. A 262K llama.cpp context setting and available memory demonstrate capacity, not verified 262K recall or TG.

12 October: Max 40-core 35B context sweep

A newer original oMLX session, uploaded on 12 October GMT, tests Qwen3.6-35B-A3B-oQ4e-mtp on an M5 Max 40 GPU cores / 128GB. At configured 16K it reports 5,820 PP / 194.6 one-stream TG tok/s; at 195K it reports 1,171 / 53.4. Its six buckets run from 4K to 195K. Recipe: oMLX 0.7.0, MTP adaptive depth 3 and A8 activation on; DFlash, SpecPrefill and TurboQuant KV off. Exact tokenized prompts, TG starting depths, generated lengths, cache reuse and long-context recall are unpublished. The 262K maximum setting is capacity only. These Max measurements remain separate from Ultra results and calculator defaults.

Two new M5 Max source sessions

The original oMLX Max 32-core / 36GB REAP-19B session shows seven configured context buckets. Its 195K result is 1,519 PP / 46.7 one-stream TG tok/s; at 8K it is 4,911 / 104.3. The session's separate eight-stream total is 347.4 TG tok/s, not a per-stream rate. This is an OptiQ 4-bit REAP-19B derivative of Qwen3.6-35B-A3B, not the base checkpoint.

A Max 40-core / 128GB Coder-Next MXFP8 session measures 3,260 PP / 80.0 one-stream TG tok/s at configured 8K. Its separate four-stream whole-device total is 199.4 TG tok/s. Both oMLX sessions show MTP, DFlash and SpecPrefill off in their recipes. DFlash may help in another recipe; these measurements only establish that it was disabled here. Configured buckets are not exact prompt lengths or verified recall tests; output length, exact TG starting depth and per-run cache reuse are unpublished.

Ultra 64-core ThinkingCap: recipe matters

Grok's lead also resolves to original oMLX runs of the ThinkingCap 27B 4-bit DWQ derivative on an Ultra 64-core / 256GB. One configured 32K result reports 1,888 PP / 130.8 TG tok/s with MTP adaptive depth 4 and TurboQuant KV off. A different 32K recipe with MTP depth 3 and TurboQuant KV 8-bit reports 1,847 / 44.1; its 64K session result is 1,534 / 21.9. DFlash and SpecPrefill are off in these recipes. The two 32K runs are not a controlled single-setting comparison; output lengths, cache reuse and exact TG starting depths are unknown. No new payback default is inferred from them.

Explore both charts ↗

一手來源更新 · 2026/10/12

同一台 Ultra、兩個引擎：讀得快，還是寫得快？

JV 於 10 月 11 日首次公開的原作者對照測試，用 M5 Ultra 80 核 GPU／256GB，兩個引擎載入相同的 Qwen3.8-Flash-Next IQ4_NL GGUF 與 Q8_0 MTP head。系統是 macOS 27.0，引擎為 llama.cpp b11538 和 gmlx 0.4.23。

4,096 TOKEN PREFILL · GMLX3,348PP tok/s · llama.cpp 2,150

短 CONTEXT、MTP 生成 · LLAMA.CPP99.9TG tok/s · gmlx 83.6

33,281 TOKEN PROMPT11.3 秒gmlx · llama.cpp 18.9 秒

4K Prefill 是引擎內建 benchmark。開啟 MTP 的生成測試使用四題、三個 seed、每次要求輸出 512 tokens；精確 TG 起始 context 未公開，作者將它視為短 context 測試。33,281-token prompt 是另一項 recall 測試，兩個引擎都找出兩個隱藏代碼；其讀取耗時不能當成從 33K 開始生成的 TG。MTP 關閉時，作者另報 llama.cpp API TG 57.0、gmlx 內建 benchmark 64.0，量測方法不同。Cache reuse 與 reasoning token 數未報。llama.cpp 設定 262K 且記憶體容納得下，只能說明容量，不能代表已驗證 262K recall 或 TG。

10 月 12 日：Max 40 核 35B context sweep

10 月 12 日 GMT 上傳的另一組 oMLX 原始 session，在 M5 Max 40 核 GPU／128GB上測試 Qwen3.6-35B-A3B-oQ4e-mtp。設定 16K 為 PP 5,820／單路 TG 194.6 tok/s；設定 195K 為 1,171／53.4。共六種設定 bucket，範圍 4K–195K。配方為 oMLX 0.7.0、MTP adaptive depth 3 與 A8 activation 開啟；DFlash、SpecPrefill、TurboQuant KV 關閉。精確 tokenized prompt、TG 起始深度、實際輸出長度、cache reuse 與長文 recall 未公開；262K 最大窗口設定僅是容量。這組 Max 數字不與 Ultra 合併，也不放入預設回本試算。

兩組新的 M5 Max 原始 session

oMLX 原始 Max 32 核／36GB REAP-19B session含七種設定 context。195K 結果為 PP 1,519／單路 TG 46.7 tok/s；8K 是 4,911／104.3。另項八路整機合計 TG 347.4，不是每路速度。這是 Qwen3.6-35B-A3B 的 OptiQ 4-bit REAP-19B 衍生 checkpoint，與原版分開看。

另一組Max 40 核／128GB Coder-Next MXFP8 session在設定 8K 測得 PP 3,260／單路 TG 80.0 tok/s；獨立的四路整機合計是 TG 199.4。兩組原始 recipe 的 MTP、DFlash、SpecPrefill 均關閉。DFlash 在其他配方可能有加速效果；這些數字只證明這次沒有啟用。設定 bucket 並非精確 prompt 長度或 recall 驗證；實際輸出長度、精確 TG 起點與每次 cache reuse 未公開。

Ultra 64 核 ThinkingCap：配方很重要

Grok 的線索也追到 ThinkingCap 27B 4-bit DWQ 衍生 checkpoint 的 Ultra 64 核／256GB oMLX 原始頁。一組設定 32K 結果為 PP 1,888／TG 130.8 tok/s，MTP adaptive depth 4、TurboQuant KV 關閉。另一組設定 32K 配方用 MTP depth 3 與 TurboQuant KV 8-bit，得到 1,847／44.1；同 session 的設定 64K 結果為 1,534／21.9。DFlash 和 SpecPrefill 都關閉。兩個 32K 的配方不只一處不同，不能歸因成單一設定的加速；實際輸出長度、cache reuse 和精確 TG 起點仍未知，也沒有拿來當預設回本速度。

查看兩張進展圖 ↗