Crowd simulation by deep reinforcement learning

Abstract

The simulation of realistic virtual crowds has been an active research area in several domains, including film industry, video games, engineering, and psychology. The main aspect of visual crowd is the ability of having agents capable of navigating to their goal by avoiding collisions with obstacles and other agents. The authoring part, which is the ways of building and controlling the simulations, is also a key point. During the years, multiple approaches and methods have been utilized for crowd simulation, with remarkable results. With the rise of deep learning methods, reinforcement learning has shown great results in sequential decision-making problems, and it would not be long until it was also applied to virtual crowds. Reinforcement learning is able to produce simulations with complex scenarios by using simple reward functions that control the navigation and behavior of the agents. Thus, this work explores various methods which focusing on how reinforcement learning can be applied to the area of crowd simulation. Specifically, two RL-based approaches are presented, a microscopic which focuses on bringing diversity among agents' behaviors and enables the real-time modification of them, and a mesoscopic approach which concentrates on creating large-scale simulations by combining a list of available components.