TY - JOUR
T1 - Robust Contrastive Learning With Dynamic Mixed Margin
AU - So, Junhyuk
AU - Lim, Yongtaek
AU - Kim, Yewon
AU - Oh, Changdae
AU - Song, Kyungwoo
N1 - Publisher Copyright:
© 2013 IEEE.
PY - 2023
Y1 - 2023
N2 - One of the promising ways for the representation learning is contrastive learning. It enforces that positive pairs become close while negative pairs become far. Contrastive learning utilizes the relative proximity or distance between positive and negative pairs. However, contrastive learning might fail to handle the easily distinguished positive-negative pairs because the gradient of easily divided positive-negative pairs comes to vanish. To overcome the problem, we propose a dynamic mixed margin (DMM) loss that generates the augmented hard positive-negative pairs that are not easily clarified. DMM generates hard positive-negative pairs by interpolating the dataset with Mixup. Besides, DMM adopts the dynamic margin incorporating the interpolation ratio, and dynamic adaptation improves representation learning. DMM encourages making close for positive pairs far away, whereas making a little far for strongly nearby positive pairs alleviates overfitting. Our proposed DMM is a plug-and-play module compatible with diverse contrastive learning loss and metric learning. We validate that the DMM is superior to other baselines on various tasks, video-text retrieval, and recommender system task in unimodal and multimodal settings. Besides, representation learned from DMM shows better robustness even if the modality missing occurs that frequently appears on the real-world dataset. Implementation of DMM at downstream tasks is available here: https://github.com/teang1995/DMM
AB - One of the promising ways for the representation learning is contrastive learning. It enforces that positive pairs become close while negative pairs become far. Contrastive learning utilizes the relative proximity or distance between positive and negative pairs. However, contrastive learning might fail to handle the easily distinguished positive-negative pairs because the gradient of easily divided positive-negative pairs comes to vanish. To overcome the problem, we propose a dynamic mixed margin (DMM) loss that generates the augmented hard positive-negative pairs that are not easily clarified. DMM generates hard positive-negative pairs by interpolating the dataset with Mixup. Besides, DMM adopts the dynamic margin incorporating the interpolation ratio, and dynamic adaptation improves representation learning. DMM encourages making close for positive pairs far away, whereas making a little far for strongly nearby positive pairs alleviates overfitting. Our proposed DMM is a plug-and-play module compatible with diverse contrastive learning loss and metric learning. We validate that the DMM is superior to other baselines on various tasks, video-text retrieval, and recommender system task in unimodal and multimodal settings. Besides, representation learned from DMM shows better robustness even if the modality missing occurs that frequently appears on the real-world dataset. Implementation of DMM at downstream tasks is available here: https://github.com/teang1995/DMM
KW - Multimodal learning
KW - contrastive learning
KW - recommender system
KW - retrieval
KW - robustness
KW - video representation
UR - http://www.scopus.com/inward/record.url?scp=85162706506&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85162706506&partnerID=8YFLogxK
U2 - 10.1109/ACCESS.2023.3286931
DO - 10.1109/ACCESS.2023.3286931
M3 - Article
AN - SCOPUS:85162706506
SN - 2169-3536
VL - 11
SP - 60211
EP - 60222
JO - IEEE Access
JF - IEEE Access
ER -