Shared-unique Features and Task-aware Prioritized Sampling on Multi-task Reinforcement Learning

We observe that current state-of-the-art (SOTA) methods suffer from theperformance imbalance issue when performing multi-task reinforcement learning(MTRL) tasks. While these methods may achieve impressive performance onaverage, they perform extremely poorly on a few tasks. To address this, wepropose a new and effective method called STARS, which consists of two novelstrategies: a shared-unique feature extractor and task-aware prioritizedsampling. First, the shared-unique feature extractor learns both shared andtask-specific features to enable better synergy of knowledge between differenttasks. Second, the task-aware sampling strategy is combined with theprioritized experience replay for efficient learning on tasks with poorperformance. The effectiveness and stability of our STARS are verified throughexperiments on the mainstream Meta-World benchmark. From the results, our STARSstatistically outperforms current SOTA methods and alleviates the performanceimbalance issue. Besides, we visualize the learned features to support ourclaims and enhance the interpretability of STARS.