Heterogeneous Crowd Simulation Using Parametric Reinforcement Learning

Show simple item record

dc.contributor.author Hu, Kaidong
dc.contributor.author Haworth, Brandon
dc.contributor.author Berseth, Glen
dc.contributor.author Pavlovic, Vladimir
dc.contributor.author Faloutsos, Petros
dc.contributor.author Kapadia, Mubbasir
dc.date.accessioned 2023-05-28T15:24:04Z
dc.date.available 2023-05-28T15:24:04Z
dc.date.copyright 2021 en_US
dc.date.issued 2021-12-29
dc.identifier.citation K. Hu, B. Haworth, G. Berseth, V. Pavlovic, P. Faloutsos, & M. Kapadia. (2023). Heterogeneous Crowd Simulation Using Parametric Reinforcement Learning. IEEE Transactions on Visualization and Computer Graphics, 29(4), 2036–2052. https://doi.org/10.1109/TVCG.2021.3139031 en_US
dc.identifier.uri https://doi.org/10.1109/TVCG.2021.3139031
dc.identifier.uri http://hdl.handle.net/1828/15134
dc.description.abstract Agent-based synthetic crowd simulation affords the cost-effective large-scale simulation and animation of interacting digital humans. Model-based approaches have successfully generated a plethora of simulators with a variety of foundations. However, prior approaches have been based on statically defined models predicated on simplifying assumptions, limited video-based datasets, or homogeneous policies. Recent works have applied reinforcement learning to learn policies for navigation. However, these approaches may learn static homogeneous rules, are typically limited in their generalization to trained scenarios, and limited in their usability in synthetic crowd domains. In this article, we present a multi-agent reinforcement learning-based approach that learns a parametric predictive collision avoidance and steering policy. We show that training over a parameter space produces a flexible model across crowd configurations. That is, our goal-conditioned approach learns a parametric policy that affords heterogeneous synthetic crowds. We propose a model-free approach without centralization of internal agent information, control signals, or agent communication. The model is extensively evaluated. The results show policy generalization across unseen scenarios, agent parameters, and out-of-distribution parameterizations. The learned model has comparable computational performance to traditional methods. Qualitatively the model produces both expected (laminar flow, shuffling, bottleneck) and unexpected (side-stepping) emergent qualitative behaviours, and quantitatively the approach is performant across measures of movement quality. en_US
dc.description.sponsorship The research was supported in part by the Murray Postdoctoral Fellowship, NSERC Create DAV, Ontario Research Foundation (Grant No. RE08-054), NSERC Discovery [funding reference number RGPIN-2021-03541], and NSF awards: IIS-1703883, S&AS-1723869, IIS-1955404, IIS1955365, RETTL-2119265, and EAGER-212211 en_US
dc.language.iso en en_US
dc.publisher IEEE Transactions on Visualization and Computer Graphics en_US
dc.subject Multi-agent navigation en_US
dc.subject crowd simulation en_US
dc.subject reinforcement learning en_US
dc.subject parametric policy learning en_US
dc.title Heterogeneous Crowd Simulation Using Parametric Reinforcement Learning en_US
dc.type Postprint en_US
dc.description.scholarlevel Faculty en_US
dc.description.reviewstatus Reviewed en_US

Files in this item

This item appears in the following Collection(s)

Show simple item record

Search UVicSpace


My Account