Tuning the Hyperparameters of Anytime Planning: A Metareasoning Approach with Deep Reinforcement Learning
In Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS), 2022
Cite: Bhatia, A., Svegliato, J., Nashed, S. B., & Zilberstein, S. (2022). Tuning the Hyperparameters of Anytime Planning: A Metareasoning Approach with Deep Reinforcement Learning. In Proceedings of the International Conference on Automated Planning and Scheduling, 32(1), 556-564. https://ojs.aaai.org/index.php/ICAPS/article/view/19842
TL;DR: Deep RL to determine optimal stopping point and hyperparameters of anytime algorithms at runtime to optimize utility of the final solution. Good results on Anytime A* search algorithm and RRT* motion planning algorithm.
Abstract
Anytime planning algorithms often have hyperparameters that can be tuned at runtime to optimize their performance. While work on metareasoning has focused on when to interrupt an anytime planner and act on the current plan, the scope of metareasoning can be expanded to tuning the hyperparameters of the anytime planner at runtime. This paper introduces a general, decision-theoretic metareasoning approach that optimizes both the stopping point and hyperparameters of anytime planning. We begin by proposing a generalization of the standard meta-level control problem for anytime algorithms. We then offer a meta-level control technique that monitors and controls an anytime algorithm using deep reinforcement learning. Finally, we show that our approach boosts performance on a common benchmark domain that uses anytime weighted A* to solve a range of heuristic search problems and a mobile robot application that uses RRT* to solve motion planning problems.