Learning unsupervised disentangled skill latents to adapt unseen task and morphological modifications

Learning adaptable policies in the absence of explicit reward signals is a challenging problem in reinforcement learning. We propose an algorithm that disentangles the morphology-aware variables from skill latent space for adapting quickly to unseen morphological changes. Through several task learning experiments using MuJoCo Ant environments, we demonstrate that the agent can perform zero-shot inference and adapt to mild modification in morphology within an expected performance range. Furthermore, in the case of severe unseen morphological damage to the agent's body, with the help of add-on training steps, we can subjoin additional value incorporated into the disentangled latent space without catastrophically destroying the pre-trained network. We observe that the proposed separable-skill based method outperforms prior evolutionary meta-learning-based approaches, and the presented approach opens up research direction toward reinforcement learning for open-world novelty. Our source code is available at:https://github.com/boratw/sd4m.