Tencent’s GameGen-O looks like the first step towards an AI game engine [Video]

Just two weeks ago, Google researchers revealed the results of their experiments recreating Doom with AI using GameNGen. We were impressed, but it didn’t exactly make it look like we’d be seeing AI game generation anytime very soon. But now the Chinese tech company Tencent has dropped a research paper that appears to lay the foundations for AI game engines.

Tencent’s GameGen-O is described as the first diffusion transformer model tailored for the generation of open-world video games. The paper says that it allows high-quality, open-domain generation by simulating characters, dynamic environments, complex actions and events.

According to the paper on GitHub, GameGen-O involves data collection and processing from scratch from “over a hundred next-generation open-world games”, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. A two-stage training process consists of foundation model pretraining via text-to-video and video continuation, followed by instruction tuning using a trainable InstructNet, which …