Overviewing AI-Dedicated Hardware for On-Device AI in Smartphones

Machine learning applications for smartphones, such as camera applications, and automatic speech recognition, require a real-time response of deep learning processing. Although one inference processing of deep learning requires more than a billion operations, the development of hardware and deep learning has allowed on-device real-time deep learning processing in smartphones. The performance of smartphone neural processing unit (NPU) has been doubling every year since NPU was integrated into smartphone application processors in 2017. As of 2021, dozens of on-device deep learning applications have been employed in a smartphone. This chapter aims to provide an overview of the history of progress in achieving on-device inference in the smartphone industry. This chapter describes the development and overview of system-on-chip technology and single instruction multiple data architecture such as central processing unit, graphics processing unit, digital signal processor, and NPU, to achieve on-device artificial intelligence used in smartphones. We will also briefly introduce the architectures and features of NPUs widely used in the mobile phone industry. In addition, it introduces strategies such as quantization, pruning, and compression that can increase computational efficiency.