Google Demos a New AI That Can Play Video Games with You

The Google DeepMind team has developed an AI agent called SIMA (Scalable Instructable Multiworld Agent) that carries out tasks in a wide variety of video games. What is unique about this development is that for the first time, a general AI agent can follow user instructions to perform a variety of tasks in a wide range of virtual games, just like humans.

The AI agent has not been trained on an individual video game limited to that environment. Instead, SIMA is trained on several video games which makes it a general, versatile, and multi-world agent. Google DeepMind’s SIMA demonstrates that it can understand unseen gaming worlds and execute actions like humans.

Google partnered with eight gaming studios to build the SIMA agent. It’s trained on nine 3D video games which include No Man’s Sky by Hello Games, Teardown by Tuxedo Labs, Valheim, and more. The agent learns new skills from each environment like mining resources, navigating the world, flying a spaceship, etc. The model is not designed to get the highest score in the game, instead, it can be a companion who can accomplish tasks based on your instructions.

Further, Google DeepMind says that SIMA has been trained on images and natural language instructions. It can control the gameplay using keyboard and mouse outputs. It has acquired almost 600 skills so from navigating, to object interaction, and opening the map, it can do almost everything in a virtual environment within 10 seconds. The team wants to make AI agents that can plan and perform actions.

In comparison to specialized agents trained on an individual game, SIMA outperforms all of them. You can read the technical report on SIMA by the Google DeepMind team to understand how the agent works. While it’s an AI agent performing tasks in a virtual world, the application of such generalized AI agents can be of immense value in robotics, marking a paradigm shift in the real world.