Estimating Heart Rate from Facial Images using Artificial Intelligence

2019-10-31

Wearable heart rate monitoring devices are no longer a novelty, but have you ever heard of a system that can determine heart rate solely based on facial images? Recently, researchers from the Chinese Academy of Sciences published a preprint paper on Arxiv.org, introducing an end-to-end trainable heart rate estimator called RhythmNet. It utilizes AI and an optical technology that detects changes in blood volume in the skin tissue to address challenges related to head movement and variations in lighting. As explained by the researchers in the paper, the basis for HR estimation using photoplethysmography (PPG) is the periodic variation in light absorption by the skin due to changes in blood volume pulse (BVP). When chromophores like hemoglobin in the microvessels of the dermis and subcutaneous layers absorb disproportionate amounts of light, there is a subtle color change when blood is pumped into the underlying veins and arteries. Although these changes are imperceptible to the naked eye, they can be easily captured by the RGB sensors embedded in wearable devices. RhythmNet consists of multiple components, with the facial detector being the most crucial one. This detector locates more than 81 facial landmarks based on a person's facial video. Subsequently, another separate component performs alignment and skin segmentation to remove the eye region and other non-facial areas, generating spatiotemporal maps from video frames with a 0.5-second interval to represent the heart rate signal. Finally, these spatiotemporal maps are fed into a machine learning model, which, after training, can predict heart rate based on the maps and calculate the estimated number of heartbeats per minute. Through scientific statistical methods, the average number of heartbeats per minute is determined. To train RhythmNet, the team created a large multimodal database called VIPL-HR1, which includes 2378 visible light videos and 752 near-infrared videos, involving a total of 107 subjects. These video clips were captured using webcams, infrared sensors, and smartphones, providing a diverse range of data. The researchers evaluated RhythmNet on two widely used databases, MAHNOB-HCI and MMSE-HR, as well as their own database. They reported that for the majority of samples tested in VIPL-HR1 (71%), RhythmNet had a heart rate estimation error of less than 5 beats per minute. Additionally, the error rates for MAHNOB-HCI and MMSE-HR did not exceed 8.28 beats per minute, surpassing the historical records in the field. "Heart rate is an important physiological signal that reflects a person's physical and emotional state. Traditional heart rate measurement typically relies on contact monitoring devices, which can be inconvenient and uncomfortable. Our proposed system addresses this issue in experiments," the paper stated. Finally, the team expressed their desire to develop a superior heart rate estimation model using transfer learning and multitask learning techniques. They also plan to further investigate the applicability of RhythmNet to other physiological measurement tasks, such as respiratory rate and blood pressure measurement through video.

RECENT AI NEWS

RECENT AI TOOLS