Vision-based virtual vibration sensor using error calibration convolutional neural network with signal augmentation

In this study, we propose a methodology for using image-based virtual vibration sensors to replace acceleration sensors in the vibration monitoring of targets. Most image-based vibration-monitoring techniques typically involve the installation of a non-contact-capture camera at a location unaffected by disturbances to photograph the outside of a target. However, these methods have several drawbacks, including the additional cost of installing sensors and the need to select appropriate installation locations. The proposed virtual vibration sensor addresses these limitations by combining signal extraction based on a contact-capture camera installed inside the target with a convolutional neural network for error calibration. By employing a camera using the contact-capture method, the installed cameras can be reused to integrate the functions of other sensors and reduce the total number of sensors required. Moreover, in the proposed method, a calibration convolutional neural network (CCNN) improves the limited accuracy of contact-capture cameras. Furthermore, the calibration performance of the CCNN was improved via signal augmentation even with the availability of limited data. The experimental results obtained using a quadcopter platform demonstrated the improved accuracy and effectiveness of the proposed virtual sensor compared to that of the actual sensors.