Learning to Find Unpaired Cross-Spectral Correspondences

Somi Jeong, Seungryong Kim, Kihong Park, Kwanghoon Sohn

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)


We present a deep architecture and learning framework for establishing correspondences across cross-spectral visible and infrared images in an unpaired setting. To overcome the unpaired cross-spectral data problem, we design the unified image translation and feature extraction modules to be learned in a joint and boosting manner. Concretely, the image translation module is learned only with the unpaired cross-spectral data, and the feature extraction module is learned with an input image and its translated image. By learning two modules simultaneously, the image translation module generates the translated image that preserves not only the domain-specific attributes with separate latent spaces but also the domain-agnostic contents with feature consistency constraint. In an inference phase, the cross-spectral feature similarity is augmented by intra-spectral similarities between the features extracted from the translated images. Experimental results show that this model outperforms the state-of-the-art unpaired image translation methods and cross-spectral feature descriptors on various visible and infrared benchmarks.

Original languageEnglish
Article number8721725
Pages (from-to)5394-5406
Number of pages13
JournalIEEE Transactions on Image Processing
Issue number11
Publication statusPublished - 2019 Nov

Bibliographical note

Funding Information:
Manuscript received October 15, 2018; revised March 25, 2019 and May 11, 2019; accepted May 13, 2019. Date of publication May 24, 2019; date of current version August 22, 2019. This work was supported by the National Research Foundation of Korea (NRF), Ministry of Science and ICT, through the R&D Program for Advanced Integrated-intelligence for Identification (AIID), under Grant NRF-2018M3E3A1057289. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Jocelyn Chanussot. (Corresponding author: Kwanghoon Sohn.) S. Jeong, K. Park, and K. Sohn are with the School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, South Korea (e-mail: somijeong@yonsei.ac.kr; khpark7727@yonsei.ac.kr; khsohn@yonsei.ac.kr).

Publisher Copyright:
© 1992-2012 IEEE.

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Learning to Find Unpaired Cross-Spectral Correspondences'. Together they form a unique fingerprint.

Cite this