Google AI watches and learns how to play open-world video games

Google DeepMind artificial intelligence models can play a variety of open-world video games, including: no man's skycould be a step toward generally intelligent AI that operates in the physical world just like humans do by simply watching videos from a screen.

Playing video games has long been used as a way to test advances in AI systems, such as Google DeepMind's virtual chess and AI mastery of Go, but these games have obvious winners and losers. Training AI to be successful is relatively easy. they.

Open world games with more abstract objectives and irrelevant information that can be ignored. Mine Craft, AI systems are more difficult to crack. The sheer number of options available in these games brings them closer to everyday life, making them an important stepping stone toward training AI agents that can perform tasks in the real world, such as controlled robots and artificial general intelligence. It is considered.

Now, researchers at Google DeepMind have developed an AI called Scalable Instructable Multiworld Agent (SIMA). It allows you to play nine different video games and virtual environments, never seen before, using only the video feed from the game.This also includes space exploration no man's sky,problem solving tear down and action packed goat simulator 3.

“This is actually the interface that humans use to interact with computers, and it's a very general-purpose interface,” said DeepMind's Frederic Besse.

SIMA can perform approximately 600 tasks in 10 seconds or less that are common to a variety of games, such as moving, using objects, navigating menus, and more. You can also perform more unique tasks such as flying spaceships and mining resources.

Besse and his colleagues trained SIMA to use existing video and image recognition models to interpret game video data and map what's happening in the video to specific tasks. To provide this information, researchers asked pairs of people to play a video game together, with one person looking at the screen and asking the other person what movements he or she would make. , and also had people reflect on their gameplay and describe the mouse and keyboard movements they performed in the game. their game action. This allowed SIMA to see how people's movements were related to tasks.

When SIMA was trained on eight games, the researchers discovered that SIMA was able to play a ninth game that had never been seen before. However, it fell short of human-level performance. The researchers used an iterative training method to train the AI ​​on eight games and test it on a ninth to ensure it could play games it had never seen before.

Felipe Meneguzzi from the University of Aberdeen in the UK says that while estimating a variety of games is an important step towards generalist AI agents, SIMA can currently only perform a relatively limited set of games that do not require long-term planning. Only short tasks. Performing a wider range of complex tasks will become more difficult, he says.

“For a company like DeepMind, it's worth remembering that this research isn't really about gaming, it's about robotics,” says Michael Cook of King's College London. “Navigating a 3D environment is a means to an end, and these companies are hard at work developing AI systems that can perceive and act on the world. Therefore, this will have a huge impact on video games. I don't think it will give us much, but there may be many unknown effects on our lives in the real world.”