Google introduced Scalable Instructable Multiworld Agent (SIMA), a generalist AI agent for virtual environments. Bots understand natural language commands and can execute them throughout the game environment. This release marks a shift from AI agents playing a specific game to being able to play any game.
Bots can operate in various virtual environments. (Image source: Google).
Main highlights
- Google is partnering with game developers to train its AI.
- The goal is not to win a single game, but to navigate different games.
- The researchers also developed a custom environment to train the bot.
Google introduced Scalable Instructable Multiworld Agent (SIMA), an AI bot that can follow natural language instructions in a variety of gaming environments. Google's Deepmind previously developed an AI agent aimed at beating video games and other humans in games in multiplayer settings. SIMA is a generalist AI for virtual environments that focuses on interpreting and following various instructions rather than winning a single game.
Google researchers partnered with video game studios to train their AI on actual released games.These games are included satisfaction, goat simulator 3, Valheim, No Man's Sky, Wobly Life, Decomposition and Hydronia. Agents can interact with menu options, mine resources, craft items, drive vehicles, and shoot down asteroids in these virtual environments. Google engineers also used a “research environment” in Unity. This includes new environments they have developed. construction lab.
How SIMA was trained
SIMA was trained on two datasets. One approach was to record a human giving instructions to another human and performing those instructions in a video game. Another approach he took was for humans to play the game and provide instructions in the form of voice commands, causing actions within the game to occur. Together, the two data sets provided SIMA with the information it needed to understand how natural language commands were tied to gameplay behavior.
Google aims for high-level strategic planning
The AI doesn't need access to the game's source code or specific APIs. SIMA controls the game using keyboard and mouse input, the same interface that humans use. SIMA has the potential to interact with any virtual environment. SIMA can now follow simple instructions such as “climb the ladder,'' “move left,'' “mine resources,'' and “open the map.'' In the future, Google plans to improve SIMA so that it can follow instructions that require high-level strategic planning and perform multiple subtasks, such as “mining resources and building bases.”
