Set a task: you are a forensic assistant and the audio contains several overlapping sound sources. Work out where the child in the recording is.
iFlytek’s SparkAudio 1.0 answered within seconds. Regular mechanical noise from heavy equipment surrounded the child, so a train or a subway car was plausible. What matters is the reasoning path. The model did not seize on the child’s mention of singing and assume a person or an instrument. It weighed the humming, the rhythmic industrial noise and the child’s remark about being on a vehicle, then reasoned toward a setting.
In a second clip, where the prompt barred the model from inferring anything from the spoken content, it still mapped car sounds, machinery and the child’s abstract descriptions of large round vehicles onto a conclusion about heavy traffic nearby.
Why a speech foundation model, and why now
Voice is human communication at its cheapest, and until recently AI treated it as a text problem with an audio front end. That breaks the moment the setting is a car cabin, a pair of smart glasses or a humanoid robot, where the machine has to hear a scene, understand intent and coordinate action while the acoustics shift under it.
Those environments also expose an old trilemma. Accuracy, latency and engineering cost do not all max out at once. iFlytek’s answer is a two-layer structure: a speech foundation model that learns general audio representation, and task-specific models tuned hard against individual jobs.
SparkAudio 1.0 is the base. It accepts text and speech, outputs text, and covers transcription, multilingual translation, multi-dialect recognition, environmental sound recognition, speaker separation, emotion analysis and complex audio question answering. On iFlytek’s published benchmarks it beats several state-of-the-art closed models and matches them on instruction following.
Three techniques doing the work
SemiEnc, a semi-supervised audio decoder, targets weak representation modelling. During pretraining it upgrades the quantiser input from raw acoustic features to reinforced representations extracted by supervised task models, injecting priors from several dimensions and strengthening the encoder through multi-task training. The effect is robustness in recognition and gains across tasks.
S-S2S, dynamic segmented end-to-end modelling, attacks latency. Built on speech-to-speech modelling, it closes the loop between understanding and generation so recognition, comprehension and generation stop waiting on each other in sequence. Incremental streaming allows earlier generation driven by semantics, with dynamic feedback correction, which pulls down response time in simultaneous interpretation and real-time interaction.
CoDec, dual-paradigm cooperative decoding, attacks compute cost. A non-autoregressive CTC decoder produces a full result in parallel. A large-model autoregressive module then uses speculative decoding to decide whether to intervene and where correction should start. Accuracy holds while average latency and cost fall.
The products on top
Three task models sit on the base. Spark-ASR-2.0 handles recognition and goes beyond transcription: it uses context and discourse to output coherent text, stripping filler words and repeated phrases, and normalising numbers, units and punctuation. In test clips mixing times, book titles, web links and mixed Chinese and English, it produced clean prose. It is already integrated into iFlytek Input Method 16.0.
Against the current best available systems, iFlytek reports clear advantages in dialects, high-noise conditions and low volume, with most primary task metrics ahead of the best published results. The architecture behind it combines non-autoregressive recognition, inheriting the audio representation and complex-acoustics modelling of the preview model and scaling through mixture-of-experts and incremental training, with an LLM-enhanced autoregressive path that brings language coverage, context and reasoning.
Spark-LiveTranslation and Spark-Audio-Realtime cover simultaneous interpretation and real-time interaction.
The part that is not in the press release
The company’s framing is a progression from voice interaction to multimodal interaction to embodied multimodal interaction, which places the model on a roadmap toward robots rather than phones.
The forensic demonstration also shows the limits honestly. The model missed a short warning tone in the first clip. It did not combine the two clips into a joint inference, treating each separately. Forensic work demands a standard of rigour that no general model currently meets, and iFlytek presents the demo as evidence of potential rather than readiness.
What it does establish is narrower and more useful. A speech model trained entirely on domestic compute can reason about an acoustic environment rather than only transcribe it, and that capability is the precondition for a machine that operates in a room full of noise without asking the human to repeat themselves.
Editor’s note: This is an adapted translation of the original Zhidx report. It has been trimmed and restructured for readability for an international business audience. The full original (in Chinese) is at https://www.zhidx.com/p/604627.html.