126 Pages on Agents That Never Switch Off

Nine universities spent 126 pages arguing that an assistant running in the background is not the same thing as an assistant that is always on. Of 119 systems and products they reviewed, only five research projects reached the top level, and no commercial product did.

Always-On Personal Intelligence survey roadmap overview
The 126-page Always-On Personal Intelligence survey maps a continuing loop across sensing memory and action (Source: 36Kr)

Personal agents are arriving in a dense wave. Overseas labs have shipped Muse, dots, Claude Duet, Grok Bot and Gemini Daily Brief. Some keep working in the cloud after a conversation ends, some deliver a tailored morning brief, some check linked accounts every few hours. Chinese platforms including Tencent, Alibaba and ByteDance have followed with products such as Lingguang 2.0, LightVela and Qwen Go. On hardware, RayNeo and Rokid glasses advertise all-day capture and memory, and embodied intelligence is heading the same way, with frontier models becoming the high-level brain of a robot that knows the household.

The signal is that the second half of this race is about systems architecture, not a single model or feature. Perception, memory and action have to coordinate and pass information to each other, without the user prompting, while the user is absent. That raises three questions: what counts as always-on, how does an assistant get better with use, and how far are current systems from either.

A definition built on a continuing loop

Twenty-three researchers from Nanyang Technological University, UC Berkeley, Princeton, Stanford, the National University of Singapore, KAUST, Dartmouth, the University of North Carolina at Chapel Hill and Northwestern University reviewed 632 papers and 119 research systems and products across wearables, digital assistants and embodied intelligence.

Always-on in software engineering simply means powered up and available. Applied to a personal assistant it has no settled meaning, and it is routinely mixed up with proactive agents, personal agents and long-horizon agents. The survey defines it through three functional modules and six directed information flows, then sets out a definition, a framework, six levels and a roadmap. Its end state is that the user steers and the assistant runs.

Today’s agents can take work off a person’s desk, but a human still has to stay inside the loop, issuing instructions and granting permission step by step. In the survey’s final form, the user sets direction and boundaries, and the assistant decides when to observe and when to act, improves its own way of working from experience, and stops only to ask for authorisation on actions that need it.

Two properties have to hold together. Operational Continuity means the assistant decides whether and when to act without a new request, and waiting is itself a legitimate decision. State Durability means personal state survives across interactions, including preferences and promises made but not yet delivered, so that a task can be fulfilled long after the conversation that created it ended. Together they form the Continuing Loop. The everyday example is a request to remind the user to send a report after a meeting ends: the assistant has to notice when the meeting finishes, choose the moment, and hold the open commitment in between.

Always-On Personal Intelligence framework diagram with sensing memory and action
Three modules and six coupling channels form the continuing loop, under shared conditions and an implementation layer (Source: 36Kr)

By that standard, several familiar designs fall short. An assistant with long-term memory that waits for a prompt has state durability only. One that initiates conversation but retains nothing has operational continuity only. An alarm or cron script runs all day but makes no decision, firing on a preset trigger. Long-horizon agents are a different category again: they finish a given complex task across sessions, while an always-on assistant has to support goals and needs that keep changing.

Self-evolution is not remembering more

The survey’s central claim is that the continuing loop is the foundation and self-evolution is the direction. Storing a preference that the user likes reminders after returning to their desk is just updating a record. Noticing that such reminders are routinely dismissed and rewriting the rule so reminders wait until the user is back at the desk changes the mechanism, and with it the handling of every later input.

Self-evolution therefore has to be traceable, inspectable and reversible. Each rule revision needs a documented basis, a check before it takes effect and a way to roll it back. Proving the capability requires a full evidence chain from what was done, through the result, to the revised rule being used later.

The framework breaks the assistant into Sensing, Memory and Action inside one loop. Each module has both a capability question and a selection question, and choosing not to look, not to update or not to speak is also a decision. Six directed coupling channels carry information between the modules. Outside the loop sit Shared Conditions set by the user and the physical world, covering privacy, permission, safety, resources, availability and uncertainty: in a confidential meeting the assistant may listen but must not store or repeat what it heard. An Implementation layer combines the Model with Scaffolding, meaning the runtime harness, storage and tool and API interfaces that connect model output to devices, apps and data.

Three settings, one shared bottleneck

Wearable assistants face a trade between capture cost and future memory, since more camera and microphone use means more power and compute, yet what is captured now determines what can be recalled later. They also have to decide when to speak and who should hear, in rooms where other people never consented to being recorded.

Digital assistants compete with themselves for compute, with background checks, memory maintenance and user responses drawing on the same budget. They also misuse stale information: when an interface, task or permission changes, carrying forward old context does harm. The authors note that Muse combines scheduled and event-driven tasks with cross-session memory, while OpenAI’s dots keeps working on its own cloud computer between conversations, deciding when to pause and when to wake.

Embodied assistants carry the highest cost of error, because a collision cannot be undone. They pay for post-action verification and have to judge whether a helpful act was actually helpful, using memory of the user’s reaction and what happened next.

HALO oversight results across 40 always-on agent systems
HALO audit of 40 systems shows weak reporting on the memory-to-sensing and action-to-memory channels (Source: 36Kr)

Three hidden failure modes

Applying the same five questions to each module, what options exist, what constrains them, what policy is used, what other modules can supply and how success is measured, produces three findings. First, missed observations are worse than wrong ones, because an assistant cannot notice what it never saw and so never corrects its state. Low frame rates on power-constrained wearables let brief actions slip between frames, while digital assistants miss a cancelled booking or a rescheduling email they did not re-read. Most evaluations test whether observed content was handled correctly, not whether the assistant looked at the right thing or missed something.

Second, errors compound into what the paper calls a memory trap. A false memory can be reused many times before a user notices, and it distorts how later feedback is interpreted, so every attempted self-correction preserves or amplifies the original mistake. No system in the review showed long-term immunity. Keeping memory correct over time is harder than retrieving it.

Third, outcomes arrive late, so help can become a burden. A sent message is not a completed task, a thoughtful reminder today can be an irritation tomorrow, and a suggestion accepted now may not help in the long run. If a system restarts or moves across devices after the originating conversation has gone, the authors note that benchmarks offer almost nothing on avoiding both missed and duplicated promises.

Where 119 systems actually sit

The roadmap runs from L0 to L5. At L3 the assistant decides for itself whether to act or wait without a request, and the continuing loop exists. At L5 it revises its own mechanisms inside user-set boundaries. A background recorder waiting to be queried and a timer that fires without judging are both L2. The real dividing line is whether the assistant itself decides to put the result in front of the user.

Of 119 systems and products, 52 do not qualify as always-on and 67 clear the L3 threshold. Only five research efforts meet L5, with every mechanism change either explicitly approved by the user or locked inside a preset safe range, and no product reaches it. The survey places Muse and Claude Duet at L3, and dots, Grok Bot, OpenClaw and Hermes Agent at L4. Ratings are based on public material, so each level is a lower bound on what the system may actually do.

HALO and the two weakest channels

The team open sourced HALO, Harness for Always-On Loop Oversight, to audit both coupling channels and the self-evidence chain. For each item it asks whether the design is reported in the paper or public code, and whether cutting that channel or evidence step would change the output. Read Mode uses three large models as a jury, voting on whether each channel and step is documented. Run Mode is an A/B test: run the system as is, then cut one channel or evidence step under identical inputs and model, and have a blinded model judge whether the output changed.

Read Mode reviewed 40 works and found a strong bias toward action. Sensing-to-memory and memory-to-action were each reported in 37 works, action-to-sensing in 28, and memory-to-sensing in only 17. On the feedback chain, 25 works reported an action-to-memory channel but only 15 retained the result of the action. Along the five-step self-evidence chain, 24 kept action records, 16 kept outcomes, 18 linked outcomes to decisions, five actually revised a mechanism and four showed the revised mechanism being used later.

Run Mode was tested on AiGet, an open source smart-glasses assistant, and EAIA, an email assistant. Some channels collapsed the output as soon as they were cut, while others were redundant and only revealed an effect when two were cut together. Across both modes, memory-to-sensing and action-to-memory are the field’s weakest channels, and their absence hides the evidence a system needs to correct itself. If a stale memory misleads sensing, a rescheduling email looks like noise and the assistant misses it.

For closed products the team built a zero-intrusion MCP plug-in. HALO is attached like any other tool, with no vendor access to internals, and the assistant reports each action, a new email seen, a memory consulted, a reminder sent, together with the observations and memories it relied on. HALO infers which channels are carrying information, how far the evidence chain runs and which level the record can support. Teams with source code can use an adapter to mark the code behind each channel, cut it and rerun the test.

Fifteen open questions and one hard rule

Four groups of 15 open questions cover the long gaps with no prompts, lawful overreach and memory poisoning while the user is absent, proving self-improvement in a life that cannot be replayed, and evaluating months of live use where each intervention changes what the user does next. One direction is to re-check permissions at the moment an action takes effect, because a sandbox cannot stop misuse of a legitimate account and killing a process cannot recall a sent email.

The survey states one safety rule plainly: a constraint that a self-evolution mechanism can rewrite cannot constrain self-evolution. The final boundary must be set by the user and stored somewhere the assistant cannot change it. The measure of a personal assistant used to be how well it answered when asked. With always-on intelligence, a second measure appears: what it does when nobody asks.

Lead author and project lead is Shuai Liu, a first-year doctoral student at NTU’s S-Lab, with Professor Ziwei Liu as supervisor and corresponding author. An advisory group includes Gedas Bertasius, Manling Li, Jiajun Wu, Mengdi Wang and Dawn Song. The conclusion invokes Mark Weiser’s 1991 line that the most profound technologies are those that disappear.

Editor’s note: This is an adapted translation of the original 36Kr report. It has been trimmed and restructured for readability for an international business audience. The full original (in Chinese) is at https://m.36kr.com/p/4072062269952903.

Leave a comment