OpenAI, the San Francisco non-profit research organization has released Neural MMO – a massive multiagent environment for reinforcement learning.
The advances in Deep Reinforcement Learning lead to the creation of a number of virtual environments for training RL agents. However, different tasks and problems require different virtual simulations i.e. environments for training these agents.
The Neural MMO environment is suited for large-scale multiagent interaction in an open-ended manner. In order to achieve this, researchers from OpenAI designed and built the simulation taking into account scaling, efficiency, and expansion.
The environment contains two core objects: players(agents) and servers (environment). The players (or agents) can join an available server, which contains an automatically generated tile-based game map of configurable size. These tiles can be traversable like “forest” or “grass” or non-traversable such as water or stone.
Neural MMO allows researchers to examine large-scale multiagent behavior and interaction. OpenAI trained an AI system using Neural MMO by rewarding agents for their lifetime. One of the key findings was that agents that interacted more became better at certain tasks.
OpenAI says that Neural MMO environment was inspired by the human game genre MMORPGs – Massive Multiplayer Online Role-Playing Games.
“The emergence of complex life on Earth is often attributed to the arms race that ensued from a huge number of organisms all competing for finite resources.”, mention researchers in the official paper.
According to them, the inclusion of many agents and species leads to better exploration, divergent niche formation, and greater overall competence.
Neural MMO is available at Github. It can support up to 100 concurrent servers with 128 agents each. More about the environment and the experiments conducted by researchers at OpenAI can be read in the paper.