A collaborative cpu vector offloader: Putting idle vector resources to work on commodity processors

Youngbin Son, Seokwon Kang, Hongjun Um, Seokho Lee, Jonghyun Ham, Donghyeon Kim, Yongjun Park

Research output: Contribution to journalArticlepeer-review


Most modern processors contain a vector accelerator or internal vector units for the fast computation of large target workloads. However, accelerating applications using vector units is difficult because the underlying data parallelism should be uncovered explicitly using vector-specific instructions. Therefore, vector units are often underutilized or remain idle because of the challenges faced in vector code generation. To solve this underutilization problem of existing vector units, we propose the Vector Offloader for executing scalar programs, which considers the vector unit as a scalar operation unit. By using vector masking, an appropriate partition of the vector unit can be utilized to support scalar instructions. To efficiently utilize all execution units, including the vector unit, the Vector Offloader suggests running the target applications concurrently in both the central processing unit (CPU) and the decoupled vector units, by offloading some parts of the program to the vector unit. Furthermore, a profile-guided optimization technique is employed to determine the optimal offloading ratio for balancing the load between the CPU and the vector unit. We implemented the Vector Offloader on a RISC-V infrastructure with a Hwacha vector unit, and evaluated its performance using a Polybench benchmark set. Experimental results showed that the proposed technique achieved performance improvements up to 1.31× better than the simple, CPU-only execution on a field programmable gate array (FPGA)-level evaluation.

Original languageEnglish
Article number2960
JournalElectronics (Switzerland)
Issue number23
Publication statusPublished - 2021 Dec 1

Bibliographical note

Funding Information:
This research was supported by the National Research Foundation of Korea (NRF) grants (2020M3H6A1085498, 2020M3H6A1085535), by the MOTIE (Ministry of Trade, Industry, and Energy (10080613), and by the Institute of Information and Communications Technology Planning and Evaluation (IITP) grant (No.2020-0-01373), funded by the Korean government (MSIT).

Publisher Copyright:
© 2021 by the authors. Licensee MDPI, Basel, Switzerland.

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Signal Processing
  • Hardware and Architecture
  • Computer Networks and Communications
  • Electrical and Electronic Engineering


Dive into the research topics of 'A collaborative cpu vector offloader: Putting idle vector resources to work on commodity processors'. Together they form a unique fingerprint.

Cite this