[Paper & Models] Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent
Reported by r/LocalLLaMA2 sources ↓
A paper introduces Gamer, an agent for long-horizon interaction in 2D interfaces and 3D simulated worlds. Its authors say they trained it on tens of thousands of hours of keyboard-and-mouse interaction data.
Key points
- Training data spans simulated games and embodied environments
- The authors report competitive results on 2D game benchmarks and performance comparable to skilled players in several 3D open-world games
- The paper and model resources are publicly listed on arXiv, GitHub and Hugging Face
Sources
Details
The Gamer framework combines perception, memory management and multi-step reasoning, with a shared training and deployment pipeline for 2D interfaces and 3D simulated environments. Its authors say the training process includes continued pre-training on keyboard-and-mouse interaction data, sparse-thinking instruction tuning and multi-turn reinforcement learning.
The authors report that Gamer generalizes across multiple challenging 3D open-world games and achieves competitive results on conventional 2D game benchmarks. The article provides no benchmark scores or independent evaluations.
Written from coverage as of