HarvNet: Resource-Optimized Operation of Multi-Exit Deep Neural Networks on Energy Harvesting Devices

Optimizing deep neural networks (DNNs) running on resource-constrained devices, such as energy harvesting sensor devices, poses unique challenges due to the limited memory and varying energy conditions. Existing efforts have shown that deploying a multi-exit network mitigates the problem by allowing tradeoffs between accuracy and computational complexity. However, previous works did not fully consider two essential requirements: optimized neural architecture and optimized inference policy. In this paper, we present HarvNet, which comprises two complementary techniques for generating and operating a multi-exit network for energy harvesting devices. First, we provide a neural architecture search scheme, HarvNAS, which configures the best multi-exit architecture while meeting memory and energy constraints. Second, HarvSched learns and constructs the best progressive inference policy with different energy constraints by considering runtime factors, such as the harvesting status and the energy storage level. We implemented HarvNAS using the TensorFlow framework and then implemented and evaluated HarvSched on an MSP430-based sensor device. The evaluation showed that HarvNAS generated a model with up to 2.6%p higher accuracy while saving up to 70% of memory compared to the existing technique, and HarvSched enabled zero-downtime operation of the generated model.