The first week of August was busy in Chinese video generation. On 31 July MiniMax released H3, its first open source multimodal generation model, and ByteDance pushed Seedance 2.5, extending single generation length from 15 seconds to 30. Grand View Research estimates the global AI video market will grow from 4.55 billion dollars in 2025 to 42.29 billion dollars by 2030, a compound annual growth rate of 32.2 per cent.
Into that window, Beijing video generation startup Sand.ai quietly dropped MAGI-2 Preview, the first mixture of experts video generation model at the 100-billion-parameter scale, released and open sourced together.
What was released
The model holds roughly 114 billion total parameters and activates about 6 billion per forward pass. On the Artificial Analysis image to video leaderboard, the one the field watches most closely for updates, MAGI-2 Preview entered the global top ten at number six.
Leaderboard placement and flashy demos are the easy things to show a market. The model’s real value is that it validates a workable path for scaling large mixture of experts video models. A unified single stream handles joint audio and visual modelling, fine-grained mixture of experts expands capacity, and an in-house system keeps 100 billion parameters training stably.
The two walls
China’s AI short drama market passed 22 billion yuan in the first five months of 2026 and may reach 40 billion yuan for the year, with more than 600 million users. Video generation is moving from can it be done to can it be done well and cheaply. Demand growth is pushing base models to their limits, and the models have hit two walls.
The first is capacity. Characters, motion, camera work and scenes keep getting more complex, and dialogue, ambient sound and music now have to be generated in the same pass, so the amount of information the model must hold rises fast.
Token count makes it concrete. In a large language model, 1,000 words of English text corresponds to a little over 1,000 tokens, with Chinese consuming slightly more. Video is another order entirely. A 720p image splits into close to a thousand patch tokens under a visual encoder, and one second of video holds 24 frames. Processing a few seconds means facing hundreds of thousands or even more than a million visual tokens, and adding audio waveforms and text instructions pushes sequence length to hundreds of times a pure text task.
The second wall is cost. That token volume means every forward pass processes far more data than text. The larger the model, the higher the training compute, cross-machine communication and inference bill. Keep scaling a dense model and the parameters activated per generation grow in lockstep. The machine slows down and the output does not necessarily improve in proportion.
Language models already answered the same problem with mixture of experts. The open question was whether the answer transfers to video. Back in November 2025 Sand.ai made what then looked like a non-consensus decision, moving its architecture from dense to mixture of experts. Eight months later, MAGI-2 Preview is the answer.
One stream, not two
MAGI-2 Preview is a unified audio and video generation model, meaning one model produces picture and sound together. It uses a Multi-Head MoE architecture where text, video and audio enter the same Transformer and keep exchanging information at every self-attention layer, so lip movement, expression, motion, dialogue, ambient sound and camera rhythm influence each other from the first step of generation.
Existing unified audio and video models usually take one of two paths. Multi-stream architectures send video and audio into separate network branches and exchange information through cross-attention or a dedicated fusion module. Cascaded designs generate picture and sound separately and synchronise afterwards.
Multi-stream models can be trained end to end, but independent branches, fusion modules and modality-specific compute paths add architectural complexity and produce irregular computation and communication. Cascaded designs need extra interfaces and synchronisation modules, and the audio to visual relationship has to travel through several links before it forms.
Sand.ai chose a single stream, putting text, video and audio into one context so they interact directly in every self-attention layer. Inside the model, shared experts handle features common to all three modalities such as scene semantics and motion trends, while text, video and audio each keep dedicated experts. Everything runs in one generation path, with no separate backbones and no additional cross-modal fusion module, which makes the architecture simpler and cuts end to end latency well below a multi-stream chain.
Why standard MoE breaks on video
Applying mixture of experts to video is not free.
Traditional expert parallelism picks top-K experts for a token and then copies the token to the devices holding those experts. Communication volume grows linearly with the number of activated experts, load imbalance adds scheduling overhead, and once sequence length reaches video scale the gains from sparse computation are easily eaten away.
The deeper problem is the routing target itself. Traditional mixture of experts selects a group of experts for a token’s complete representation, which asks a small number of experts to simultaneously understand semantics, motion, appearance and sound, limiting the model’s ability to differentiate its handling of different information types.
Sand.ai’s earlier work on Multi-Head Latent MoE and Head Parallel supplies the fix. On communication, Head Parallel moves cross-device communication ahead of routing and distributes data by head, so what travels between devices is a fixed-shape head representation rather than dynamically varying expert tokens, and the main communication volume depends only on the input representation.
On routing, Multi-Head MoE changes what gets selected. It splits a token’s 3,072-dimensional hidden representation into 12 subspaces of 256 dimensions each, and every subspace routes independently and picks its own experts.
MAGI-2 Preview follows that line, using Multi-Head MoE across 36 main network layers and filling in shared and modality-specific experts, routing strategy, cross-node Head Parallel, fused compute kernels, mixed precision and activation recomputation around unified audio and video diffusion, turning the whole thing into a trainable system at 100 billion parameters.
The infrastructure nobody sees
Architecture design and stable large-scale training are different problems. Thousands of fine-grained experts produce frequent routing decisions, data reordering and memory movement during training, which places harsh demands on low level operator efficiency, and parameter updates across a huge expert pool challenge optimiser stability.
Sand.ai built its own infrastructure on two levels. On operators, it developed MagiMoE, a high performance mixture of experts kernel library covering routing, expert sorting and expert computation, merging previously separate steps to cut intermediate results and memory movement, and optimising forward and backward passes for the Multi-Head MoE compute pattern. MAGI-2 Preview currently uses the Triton and BF16 path within it, validated at large training scale.
On training, the model uses MagiMuon, a distributed optimiser applying Muon to the main matrix parameters and AdamW to parameters better suited to conventional updates, with parameter organisation and cross-device computation re-adapted for a huge fine-grained expert pool. That hybrid design lets the optimisation method scale stably from small and medium experiments up to a 100-billion-parameter mixture of experts.
The data argument
Scaling laws carry an often-ignored precondition. As parameters grow, the information density of data has to rise with them.
But raising density is not the same as shrinking the dataset. Over-aggressive cleaning strips the model’s coverage of complex motion, unusual camera work, rare subjects and atypical audio, and parameter growth then fails to convert into real-world generation capability.
Sand.ai therefore emphasises scale, diversity and distribution coverage. The focus of data work shifts from deletion to organisation, using more accurate fine-grained annotation to teach the model the correspondence between subject, action, scene, camera, timing, audio and text. Keeping a wider body of valid data suits the scaling phase better than chasing a narrow perfect dataset.
The division of labour between pre-training and post-training changes accordingly. Pre-training covers the data distribution as completely as possible, and post-training concentrates on preference alignment, controllability and product fit. The more complete the base model’s pre-training capability, the less the product side has to patch motion consistency, detail and composition afterwards.
Sand.ai’s own summary is that scale is not capability. Parameter growth is only the starting point. The real challenge is whether a larger expert pool can be filled with sufficiently rich real data, whether a finer routing structure can be guided by precise annotation, and whether denser compute demand can be supported by a stable and efficient low level system. Only when all three advance together does parameter growth mean anything.
MAGI-2 Preview is fully open source, which means subsequent models can continue scaling parameters and data along the same route and explore longer generation, without rebuilding the training system each time.
Editor’s note: This is an adapted translation of the original Zhidx report. It has been trimmed and restructured for readability for an international business audience.