Teaching a humanoid to fold a towel currently requires a robot, an operator and an afternoon. A team at Peking University has proposed deleting the robot from that sentence.
The bottleneck is post-training. Pretraining gives a model a baseline competence, but the ceiling is set afterwards, and for embodied systems that means correction: watching a policy fail and supplying the right action at the point of failure.
The standard method is DAgger, and its most practical form is human-gated DAgger. An operator sits beside a working arm, watches closely, takes over the instant the machine is about to miss a grasp or hit an obstacle, records the correct trajectory, and then restores the scene by hand. A whole afternoon produces very little usable data.
What HIL-UMI does differently
HIL-UMI stands for human-in-the-loop UMI. Older UMI rigs use a handheld gripper with a camera to collect demonstration data cheaply, but the collection is blind: it records positives and cannot tell you where the model would actually struggle.
The Peking University and Qiyuan Robotics team, led by chief scientist Dong Hao, put the model inside the collection loop. The human moves the gripper and does the work, cameras record, and the model receives the same frames and task instruction while predicting, in the background, the next 20 steps it would take. It never controls an arm.
When the human and the model agree, the segment is already learned and does not need recording. When the predicted trajectory diverges sharply from what the human actually did, the system flags it in real time and asks the operator to demonstrate that segment again. The machine’s blind spots are captured as a by-product of ordinary work.
The two mechanisms that make it work
Divergence detection is the first. One situation usually admits several reasonable poses, so the system samples multiple action chunks in parallel and computes an energy distribution rather than a single prediction. A confident model produces a tight distribution. A diffuse distribution that also departs from the human is treated as an out-of-distribution state and triggers targeted collection.
The second is an advantage estimator. Learning to move is not the same as learning to make progress. Hovering with a block in the gripper and placing it precisely on the base are different things, so the estimator scores frames by task progress and trains on advantage-conditioned data, telling the model to learn the corrections that actually advance the task.
The loop closes dynamically. Each round of targeted collection updates the model. The next session triggers fewer alarms, and the divergence points migrate out toward harder edge cases.
What the experiments show
Across four real-arm tasks (folding a towel, tidying a desk, stacking blocks and stamping a document), plain supervised fine-tuning crawled from a 44.3 average task progress score to 58.0 on the same frame budget. HIL-UMI reached 94.8, with the precision-heavy stacking and stamping tasks hitting 100.
Collection efficiency is where the economics change. On the long-horizon desk-tidying task, human-gated DAgger took 412.99 milliseconds per usable frame. HIL-UMI took 73.40 milliseconds, a 5.63-fold improvement, because a human hand moves better than a teleoperated arm.
The commercial implication
Dong’s claim is that pretraining and post-training are not a ratio problem but a phase transition, and the practical consequence of cheaper correction data is distribution.
If a few hundred RMB of handheld hardware and an ordinary person can generate high-quality corrective data, then post-training stops being a laboratory activity. Part-time workers in ordinary homes could perform everyday tasks with the gear, connected to a single cloud model, and whenever the model detects divergence it silently captures that short, high-value differential sample.
Robotics has been short of correction data since the field began. Removing the arm from the loop does not solve the model. It solves the supply of mistakes.
Editor’s note: This is an adapted translation of the original Leiphone report. It has been trimmed and restructured for readability for an international business audience. The full original (in Chinese) is at https://www.leiphone.com/category/ai/XgCxObsqu8pc44px.html.