At a media day this week, Iluvatar CoreX put its latest Tianyao 300 next to an Nvidia H200 and let the machines run.
The headline number is narrow but real. In third-party testing completed this year, under a typical prefill-decode disaggregated deployment, the Tianyao 300 reached 1.05 times H200’s end-to-end throughput when normalised to a single card. The tests covered four mainstream models across three scenario types.
That number comes with a specific definition, and the company is careful about it: it reflects system behaviour under particular models and particular deployment conditions, not a general claim of superiority.
The two numbers that actually matter to a buyer
Peak FLOPS has been the metric Chinese AI chips were judged on for years. It is not the metric a buyer uses. What a buyer asks is how many useful tokens per second the box produces, and how long a user waits.
Iluvatar answers both. On GLM-5.3 with a 64K context, holding user experience constant, the Tianyao 300 delivered 1.15 times H200’s MoE throughput. In agent workloads spanning 8K to 128K of context, time to first token came in roughly 10 to 30 per cent lower.
The first figure governs how many requests a system can hold. The second governs how long a person stares at a blank screen. They are different workloads and the company reports them separately, which is a mark of seriousness.
The architecture explains part of it. Tianyao 300 carries 144GB of memory per card on the Tianshu architecture, with the compute organised around attention and mixture-of-experts work. Long context means holding more state and applying more attention pressure. MoE cuts total compute by activating a subset of experts but introduces small matrix multiplications, expert routing and data exchange in exchange.
Iluvatar’s answer is to run matrix operations and vector operations such as softmax in parallel across Tensor and Vector cores, while optimising the small-matrix, low-precision and communication paths that MoE creates. The aim is to cut idle time between stages rather than to raise a peak figure.
Splitting the pipeline further
The company also demonstrated an AFD disaggregation scheme. Standard prefill-decode separation splits the two phases. AFD splits again: the Tianyao 300 handles all of prefill plus the attention portion of decode, while a partner’s XPU handles the feed-forward expert networks. Intermediate activations travel between them and the generation continues.
Iluvatar reports a 60 per cent reduction in decode latency for the configuration shown. Shan Tianyi, who leads AI and accelerated computing, framed it as a structural bet: the AI compute system of the future will not be one chip doing everything, and the value lies in organising heterogeneous parts well.
That is a concession as much as an argument. Specialised inference silicon can beat a general GPU on a fixed workload. Iluvatar’s vice president Zou Xuan told Leiphone the company stays on a general framework with specialised acceleration attached, because models and operators keep changing and a customer who buys a device needs it to keep working after the next architecture shift.
The software admission
The most candid part of the briefing was about deployment. Iluvatar has served more than 380 customers across more than 30 industries. Application migration typically takes under an hour. Performance tuning takes days to a week.
That gap is the whole problem in one sentence. Getting a model to run is quick. Getting it to run well is the expensive part, and it is expensive because it consumes scarce engineers rather than hardware.
The company’s response is to put agents into its own software stack. Skills encode validated procedures, judgement calls and acceptance criteria. Agents call tools, read execution feedback and decide the next step. In a full model-serving deployment, setup time fell from about an hour to about 20 minutes with two human inputs. When a kernel needs rewriting, the system works with the company’s FlyDSL to generate candidates, compile them, check correctness and benchmark on target hardware.
Vice president Shi Jiasheng was careful to say the compiler, runtime, algorithm libraries and performance tools remain the technical foundation. The agents make that accumulated work easier to reach. They do not replace it.
Down to the edge
Iluvatar also showed its Tongyang edge line against Nvidia’s Thor T5000. The TY1200 reached 1.2 times Thor on Qwen3.8-27B and Qwen3.6-35B-A3B, and 1.8 times on Nvidia’s own open Cosmos-R2-8B world model. It provides 300 TOPS of INT8 dense compute, generating a five-second clip at 1056 by 608 in about six minutes entirely on device.
A next-generation TY2000 at 600 TOPS dense is already announced.
The strategic read is that Chinese GPU makers have stopped arguing about whether they can match Nvidia on a benchmark and started arguing about where they sit in a heterogeneous system. That is a more mature position and a harder one, because it concedes that the fight is no longer about silicon.
Editor’s note: This is an adapted translation of the original Leiphone report. It has been trimmed and restructured for readability for an international business audience. The full original (in Chinese) is at https://www.leiphone.com/category/chips/748GRHfg9VPRbp7x.html.