Predicting Other's Minds Using Model-based Reinforcement Learning

Abstract

Most of the studies on model-based reinforcement learning (RL) have been confined to single agent learning scenarios. This impedes us from fully exploring the nature and potential of model-based RL. In this study, we design a simple model-based RL agent capable of performing multi-agent tasks and evaluate the ability of model-based RL using various types of iterative Keynesian beauty contests. We examined its characteristics from various perspectives, including dominance in the competition as a performance measure, evolutionary process as an adaptation measure, the rate of convergence to the Nash equilibrium as a basic measure of effectiveness, and the degree of cooperation as an alternative measure of it. Simulations showed that the model-based RL agent outperforms other types of agents, including rule-based methods and model-free RL, by all measures. © 2022 IEEE.