HCI Based In-Cabin Monitoring System for Irregular Situations with Occupants Facial Anonymization

Absence of human in-charge in fully autonomous vehicles (FAVs) impose multi-pronged in-cabin monitoring in real-time to monitor any irregular situation. Moreover, occupant’s visual information (such as video, images) transmission from in-cabin of vehicle to the data center and control-room is required for various in-cabin monitoring tasks. However, this information transfer may cause a substantial threat to the occupants. Though applying some patch on face of the occupants protects personal information however, simultaneously it deteriorates the important facial information which is very crucial in monitoring various tasks such as emotion detection. Therefore, a human-computer-interaction (HCI) is required to manage this trade-off between facial information and facial anonymization without any important information loss. Also, this HCI based in-cabin monitoring system should be implicit to retain the users trust in the intelligent transportation system (ITS). In this paper, we have proposed generative adversarial network (GAN) based in-cabin monitoring approach for emotion detection in FAV. In this method, we proposed to generate an artificial (virtual) face having real-facial expressions to provide facial anonymity to the occupants while monitoring FAV cabin. Therefore, this method provides anonymous facial information which is essentially required in detection and monitoring tasks to avoid irregular situations. We have used both publicly available and our own in-cabin dataset to assess our proposed approach. Our experiments show satisfactory results.