Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live iOS Apps
Reported by Apple Machine Learning Research1 source ↓
Apple researchers propose an online reinforcement-learning framework that trains UI navigation agents on live iOS apps. They report gains over supervised and offline training baselines, including nearly sevenfold improvement in Android out-of-distribution generalization for an 8B agent.
Key points
- The framework trains agents directly on live iOS apps using separate planning and UI-grounding components
- Apple researchers report nearly sevenfold better Android out-of-distribution generalization for their 8B agents than the SFT baseline
- The agents outperformed Gemini-3.5-Flash and tied Gemini-3.1-Pro on real iOS apps, according to the researchers
- Evaluations included UINavBench, iOSWorld and AndroidWorld
Details
The framework combines an LLM judge’s step-level feedback on progress with a terminal reward for task completion. The researchers use Proximal Policy Optimization with step-level advantage estimation and introduce GiGPPO, a value-estimation method that draws on rewards from visually similar states across concurrent trajectories.
Apple’s researchers report that the agents outperformed supervised fine-tuning and terminal-only reinforcement-learning baselines on evaluations including UINavBench, iOSWorld and AndroidWorld, and converged faster during training. They report that 8B agents achieved nearly seven times the SFT baseline’s out-of-distribution generalization on Android; on real iOS apps, the agents outperformed Gemini-3.5-Flash and tied Gemini-3.1-Pro. The article does not provide benchmark scores or information about public availability.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh