Towards Efficient Representation and Future-Context Transformers for Goal-Conditioned Planning

Learning task-agnostic policies is essential for versatile AI agents. This talk first highlights limitations of current latent representations in capturing environment geometry, and introduces Plannable Continuous Latent States (PcLast) — combining temporal contrastive loss with multi-step inverse dynamics to build robust representations despite exogenous noise. It then examines the limitations of decision transformers in goal-conditioned tasks with changing world configurations, and improves decision-making by integrating implicit local dynamics through short-term imagined future rollouts as context — the Future Context Decision Transformer.