How do you make a 2.8 trillion parameter model run fast? Recently, teams on both sides of the Pacific fixed their eyes on Kimi K3, the world’s largest open source model by parameter count. With 2.8 trillion total parameters and 104 billion activated, K3 raised the ceiling for open models, but ordinary AI servers struggle to hold it. Kimi officially recommends 64 or more cards, and one insider says an unoptimised supernode may start at only about 10 tokens per second.
On 21 September, Inspur launched the Yuanbrain SD200 Ultra supernode at AICC 2026, tightly coupling 128 domestic AI chips to hold the full 2.8 trillion parameter K3 on a single machine, pushing token generation latency below 5.85 milliseconds for the first time. On 23 September, the overseas TPU inference team Inferact open sourced tpu-megakernels, fitting the entire K3 into one kernel on 16 Google TPU v7 chips with DSpark speculative decoding, reaching 709 tokens per second at low concurrency.
Why is trillion parameter inference slow? The bottleneck is data movement and memory bandwidth, not raw compute. SemiAnalysis estimates one K3 forward pass needs about 1.5 TB of HBM bandwidth, spreads 896 experts across chips, and triggers more than 120 All-to-All communications per pass. Traditional frameworks such as vLLM and TensorRT-LLM launch many kernels, each loading, computing and writing back, leaving tiny bubbles that eat the gap between real and peak speed.
Inferact puts K3’s 92 MoE layers into a single Pallas call, moving the next layer’s weights from HBM to on-chip VMEM while the current layer computes, leaning on TPU v7’s 64 MiB VMEM per TensorCore. Inspur instead fuses many base operators into super operators, cutting operator count about tenfold and lifting inference performance more than threefold. It then added a super operator agent that lets K3 optimise its own operators, raising performance another 50 per cent.
The demand side is exploding. Gartner estimated in March 2026 that one agent workflow consumes 5 to 30 times the tokens of a standard chatbot, and Goldman Sachs projects global token use to grow 24-fold by 2030 to about 120,000 trillion tokens per month. Inspur’s answer is capability compute plus capacity compute: the SD200 Ultra at 5.85 ms per token and 170 tokens per second per user, and the HC2000 multi compute unit lifting token output tenfold per dollar of investment.
China’s edge is system level. The SD200 Ultra uses a 3D HyperMesh architecture across 128 domestic chips, giving 8 TB unified memory and 64 TB RAM with 0.69 microsecond interconnect, the first domestic-chip supernode with GPU memory direct connect. When chip process is capped, system innovation is the most certain catch-up path, and that is why this efficiency war matters most for Chinese AI.
Editor’s note: This is an adapted translation of the original Zhidx report. The full source is at https://aichina.news/news/silicon-valley-throws-out-a-mega-kernel-chinese-team-unveils-super-45o74r (Zhidx via AICHINA.news). It has been trimmed and restructured for readability for an international business audience.