Apple’s research team has published HALO, a control system that lets a humanoid robot carry out tasks in rooms it has never scanned or mapped. The work is on arXiv under the title Perceptive Humanoid Motion from Simulation, and the hardware used for validation was a Unitree G1.
That last detail is the interesting one. A US company best known for consumer devices chose a Chinese humanoid as the body for a physical AI result, and the system it built runs a Chinese open-weight model on a Mac mini bolted to the robot.
Three layers, three jobs
HALO splits control into task orchestration, motion generation and motion tracking.
The top layer uses a vision language model, by default Alibaba’s Qwen3.6-35B-A3B, running locally on an onboard Mac mini with no network connection required. The system can also call a cloud model through an API when it needs more reasoning. This layer reads the instruction and the robot’s own camera view together. Told to sit down, it first has to work out where sitting is possible. Told to sit on a specific chair with a table in the way, it breaks the job into bending under the obstacle, approaching the chair and lowering itself, and issues one sub-task at a time to the layer below.
The middle layer uses a second, smaller model, Qwen3.5-2B, with a diffusion transformer decoder that plans whole-body motion about five seconds ahead and replans roughly three times a second.
The bottom layer is a tracker trained with reinforcement learning and policy distillation. It runs at 50 hertz, fuses depth images with joint state, and turns planned motion into joint commands.
The numbers that separate a demo from a result
The team picked six real environments that had not been scanned or mapped in advance and ran 360 single-task trials. Average success rate was 91 per cent.
Composite tasks were harder. In one real environment the team ran three multi-step tasks ten times each and averaged 70 per cent. The chain that required bending under an obstacle, walking to a target and pushing a box succeeded only half the time.
Training data came from a synthetic pipeline over 1,000 procedurally generated homes, producing about 300 hours of paired observation, instruction and motion. The pipeline generates a motion sequence from a text instruction, places it in a virtual room, checks geometry to remove foot sliding and floor penetration, rejects configurations where the robot would collide with furniture, then renders the robot’s view with varied lighting, colour and blur.
Compute was 250 Nvidia H100 GPUs for about 48 hours to synthesise the main training set, 64 B200 GPUs to train the motion generator with imitation learning, and 4 H100s to train the tracker.
What still needs a human
HALO is not autonomous in the sense the marketing of this field usually implies. Adding a new physical task still requires a person to write the prompt and design key frames, and some motions need human video or motion capture footage.
The system also relies on vision alone. It cannot directly sense contact state, so tasks that involve sustained physical contact degrade when the object leaves the camera frame or is occluded. The team lists tactile sensing as the next step.
The deeper signal is about where the bottlenecks now sit. The models are open or nearly open. The robots are purchasable. What Apple contributed here is a data engine and a control decomposition that makes an unfamiliar room survivable. That is engineering work, and it is the part that does not transfer by press release.
Editor’s note: This is an adapted translation of the original OFweek report. It has been trimmed and restructured for readability for an international business audience. The full original (in Chinese) is at https://robot.ofweek.com/2026-11/ART-8321205-8110-30710834.html.