IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL

International Joint Conference on Artificial Intelligence (IJCAI)


We propose a novel framework to identify subgoals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor, 2016) which maximizes empowerment – the ability to reliably reach a diverse set of states and show how to identify sub-goals as states with high necessary option information through an information theoretic regularizer. Despite being discovered without explicit goal supervision, our subgoals provide better exploration and sample complexity on challenging grid-world navigation tasks compared to supervised counterparts in prior work.

Latest Publications

Boosted Dense Retriever

Patrick Lewis, Barlas Oğuz, Wenhan Xiong, Fabio Petroni, Wen-tau Yih, Sebastian Riedel

NAACL - 2022