Google researchers recreate gameplay from shooter “Doom” using artificial intelligence (AI), producing playable gameplay at 20 frames per second on a single chip using a diffusion model on a neural network, reaching new possibilities for AI integration into the video game industry.
“We present GameNGen, the first game engine powered entirely by a neural model that enables real-time interaction with a complex environment over long trajectories at high quality,” the researchers wrote in their paper, published on arXiv. “While not an exact simulation, the neural model is able to perform complex game state updates, such as tallying health and ammo, attacking enemies, damaging objects, opening doors, and persist the game state over long trajectories.”
The researchers noted that “human raters are only slightly better than random chance at distinguishing short clips of the game from clips of the simulation.”
How it Works
The neural network runs on Google’s custom-built Tensor Processing Unit (TPU) and efficiently handles “Doom’s” intricate 3D environments and fast-paced action. Unlike traditional game engines that rely on software manually coded to manage game states and render graphics, GameNGen autonomously simulates the entire game environment using an AI-driven generative diffusion model.
Researchers Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter have utilized Stable Diffusion v1.4 to create GameNGen by processing both its previous frames and the player’s current input to produce new frames in the game world. They introduced Gaussian noise to the original frames and incentivized the model to “correct” these frames sequentially, a technique crucial for ensuring stable, long-term rendering from the model.
It still requires human input as gameplay recordings were fed into the GameNGen, similar to how generative AI that generates static images is being trained from vaguely sourced data online.
Beyond Gaming: Broader Implications
The amalgamation of AI with video games has long been desired by game studios seeking innovative ways to speed up the complex process of creating a game through a traditional game engine. Now, about 60% of them are already incorporating AI into their art and level design, writing, automated playtesting, and animation workflow.
The implications of GameNGen extend beyond gaming. Its capabilities suggest potential applications in virtual reality, autonomous vehicles, and smart cities, where real-time simulations are crucial for training, testing, and operational management.
For example, autonomous vehicles need to simulate numerous driving scenarios to navigate safely, a task an AI-driven engine like GameNGen could handle with high fidelity and real-time processing.
Currently, GameNGen is trained only for “Doom,” which has relatively simple systems and mechanics. Developing a more general-purpose AI game engine that can handle multiple titles would be challenging, especially for graphically intensive modern games that require substantial computational power.
“GameNGen answers one of the important questions on the road towards a new paradigm for game engines, one where games are automatically generated, similarly to how images and videos are generated by neural models in recent years,” states the research team. They believe that AI-powered engines could significantly reduce both development time and costs, potentially democratizing game creation and enabling smaller studios and individual creators to produce complex, interactive experiences that were previously unimaginable.
