SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience

Abstract

Repurposing large vision-language models (LVLMs) as computer use agents (CUAs) has led to substantial breakthroughs, primarily driven by human-labeled data. However, these models often struggle with novel and specialized software, particularly in scenarios lacking human annotations. To address this challenge, we propose SEAgent, an agentic self-evolving framework enabling CUAs to autonomously evolve through interactions with unfamiliar software. Specifically, SEAgent empowers computer-use agents to autonomously master novel software environments via experiential learning, where agents explore new software, learn through iterative trial-and-error, and progressively tackle auto-generated tasks organized from simple to complex. To achieve this goal, we design a World State Model for step-wise trajectory assessment, along with a Curriculum Generator that generates increasingly diverse and challenging tasks. The agent's policy is updated through experiential learning, comprised of adversarial imitation of failure actions and Group Relative Policy Optimization (GRPO) on successful ones. Furthermore, we introduce a specialist-to-generalist training strategy that integrates individual experiential insights from specialist agents, facilitating the development of a stronger generalist CUA capable of continuous autonomous evolution. This unified agent ultimately achieves performance surpassing ensembles of individual specialist agents on their specialized software. We validate the effectiveness of SEAgent across five novel software environments within OS-World. Our approach achieves a significant improvement of 23.2% in success rate, from 11.3% to 34.5%, over a competitive open-source CUA, i.e., UI-TARS.

Author abstract · arXiv abstract · v2

Publication

International Conference on Machine Learning (ICML), 2026

Paper and resources

• Paper · arXiv v2 (PDF)
• arXiv:2508.04700
• Google Scholar
• Code · SunzeY/SEAgent

Research topics

SEAgent · Computer-use agents · Autonomous learning · Experiential learning · World State Model · Curriculum generation · GRPO · Specialist-to-generalist transfer · OS-World

Research problem and approach

Human-labeled trajectories are scarce for novel software. SEAgent generates experience through interaction, assesses trajectories with a World State Model and progressively increases task difficulty. arXiv abstract · v2

Main contributions

• Combines adversarial imitation of failed actions with GRPO on successful trajectories. arXiv abstract · v2
• Transfers experience from specialist agents into a stronger generalist computer-use agent. arXiv abstract · v2

Method comparison

arXiv abstract · v2

Selected results

• Averaged over three runs on the five reported OSWorld applications, SEAgent reaches 32.2% success with specialist agents, 30.6% with general RL, and 34.5% after specialist-to-generalist training. The UI-TARS-7B-DPO baseline scores 11.3%; these figures refer to the evaluated application set. Table 2 · five-application OSWorld evaluation · arXiv v2
• With the World State Model, GRPO reaches 34.8% VSCode success and adding adversarial imitation raises it to 37.7%. The corresponding SFT settings score 23.2% without and 30.4% with adversarial imitation. Table 3 · VSCode ablation · arXiv v2

Cite this paper

Zeyi Sun, Ziyu Liu, Yuhang Zang, Yuhang Cao, Xiaoyi Dong, Tong Wu, Dahua Lin, Jiaqi Wang. SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience. International Conference on Machine Learning (ICML), 2026.

@inproceedings{arxiv250804700,
  title     = {{SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience}},
  author    = {Zeyi Sun and Ziyu Liu and Yuhang Zang and Yuhang Cao and Xiaoyi Dong and Tong Wu and Dahua Lin and Jiaqi Wang},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2026},
  url       = {https://icml.cc/virtual/2026/poster/65711}
}