Illuminance Color Independent in Remote Photoplethysmography for Pulse Rate Variability and Respiration Rate Measurement

— Remote photoplethysmography (rPPG) is now becoming a new trend method to measure human physiological parameters. Especially due to it noncontact measurement which safe dan suitable to use in this new era condition. Pulse rate variability (PRV) and respiration rate (RR) included as parameters can be measured by using rPPG. PRV and RR are used to measure both physical and psychological wellness of the subject. However, current performance challenges in rPPG algorithm in measuring PRV and RR are illuminance invariant and motion. Especially in different light condition which represent real-life environment, signal-to-noise ratio (SNR) will be affected and directly reduce the measurement accuracy. Therefore in this study, we develop rPPG algorithm and then investigate the performance rPPG in different illuminance scenarios. We perform PRV and RR measurement under each scenario. On this study, for the pulse signal extraction, we were using algorithm is based on the modification of plane orthogonal-to-skin (POS) algorithm. While, for respiration signal extraction is done in CIE Lab color space. Our experimental results show the mean absolute error (MAE) of each measured parameters are 3.25 BPM and 2 BPM for PRV and RR respectively compared with clinical apparatus. The proposed method proved to be more reliable to use in real environments measurement. However, limitation of our proposed algorithm is still running in offline mode, hence for the future we want try to make our algorithm run in real time.


I. INTRODUCTION
A regular physiological parameter check can improve the quality of life and guide the person to have better habit.Especially our social health status after Covid-19, where individual have to check their health condition in any place.Not only physical health affected by Covid-19, but also psychological of individual that may lead to stress, anxiety, and depression [1].Heart rate variability (HRV) and respiration rate (RR) is a physiological parameter used as the indicator of a health status, wellness, and physical condition [2].Traditionally, the acquisition of HRV and RR is done by using electrocardiogram (ECG).However, since the ECG sensors (electrodes) are needed to place contacted to the skin, with potentially spread virus or bacteria and may cause the discomfort to the user [3].Therefore, several method based on non-contact system developed to overcome the limitation of contact-based system.One reliable contactless method to sense human vital signs parameter is using camera as the sensor and well-known as remote photoplethysmography (rPPG).Similarly, to contact-based photoplethysmography (cPPG), rPPG also refer to an electro-optic technique for noninvasively measuring the tissue blood volume pulses in the microvascular tissue bed underneath the skin [4].
Different rPPG method has been proposed to deal with PRV measurement.Started in 2011, Poh et al. [18] presented a multiparameter physiological measurement including PRV parameter by using independent component analysis (ICA) method.The comparison performance between ICA and principle component analysis (PCA) for PRV measurement were investigated by Alghoul [15] in 2017. in 2018 a study about PRV was done by A. M. Rodríguez [4], he proposed a system for PRV measurement used in application including motion activity.And most recent study were done by Yu SG., et al. [16] and M. M. Shoushan, et al. [17].
For RR measurement, various algorithms have been presented as well.Most of the studies are focusing on ROI selection (e.g.face and chest) [19] and color space used in signal extraction (e.g.visible light and NIR) [20].The recent study was done by Sanyal [21], the work present RR measurement in Hue color space.
Currently, there are two major challenges from using rPPG method for PRV and RR measurement, those are invariant color illumination and subject's motion.The previous studies investigated only focus on subject's motion problem and using static light source in their experiment.In fact, the invariant light source will make the physiological signal has low signal-to-ration (SNR).Fig. 1 shows the relationship between relative amplitude between rPPG signal and luminance spectrum modeled by Hulsbusch [22] and derived by Gastel [23].In a short, the rPPG signal quality depends on the incoming wavelength spectra.
The goal of this paper is to investigate our developed rPPG algorithm to measure PRV and RR in different illuminance scenarios.For the hardware cost, we use a personal computer (PC) as processing unit and one consumer grade RGB webcam as the sensor.All processing stages will be run in offline performance.We believe in the future, our system improves the usability of rPPG in different environmental condition.Moreover, our algorithms are applicable for application with high dynamic illuminance condition such as driving activity.

II. MATERIAL AND METHOD
Illustration of our used method is shown in Fig. 2, the substantial of rPPG algorithm consists of spatial (image) processing and temporal (signal) processing.The spatial processing has main function to select and track the region of interest (ROI) and then convert the 2 dimensional data (pixels) into 1 dimensional signal.Since the converted signal still raw and usually has low SNR, temporal processing employed to improve the SNR there are 4 stages in the temporal processing including sliding window, band pass filtering, moving average and peak detection.Finally, after clean signal obtained the parameter calculation can be done.

A. ROI Selection and Tracking
We select forehead-to-cheek area as the ROI, based on our preliminary analysis stage those area has good SNR for both PRV and RR parameter.The area formed for ROI is 120 x 80 pixels.Kanade-Lucas-Tomasi (KLT) tracking algorithm [24] was used to track the ROI.

B. Intensity Averaging
To convert the pixels data into 1 dimension signal, we simply employ intensity averaging for each R, G, and B channel.Our operation in this stage was using Equation (1).Where Intensityn (xi,yj) is the intensity of each color channel located in (xi,yj), and n is the current frame number.

C. Signal Extraction
We decided to use different signal extraction algorithm for PR and RR parameter.For PR signal we were extracted by using plane-orthogonal-to-skin (POS) algorithm.The algorithm was proposed by Wang [7] which demonstrated robust to both motion and light source.The core of POS algorithm is using the plane orthogonal to the skin tone in the temporally normalized RGB space.As our best knowledge, no study investigates the usage of POS algorithm for PRV measurement.The detail of algorithm can see thoroughly from his paper [7].In the other hand, we use CIE Lab color space instead of RGB color space.The idea is to separate diffused and specular reflection from incoming luminance.Specular reflection is the component from surface of the skin Initialize: H = zeros(1 ,N) 2: For n = 1,2,…,N do 3: 6: % 0.222 0.07 0.71 .Cn (3) 7:

D. Sliding Window
After signal from full video frames were extracted, for further processing we applied sliding window processing.The window lengths of two parameters were also different.For PR signal, 5 second window and 1 second overlap were used since the fast changing in PR.While for RR parameter had longer window with 8 second length and 2 second overlap.Since we proceed all the process in offline mode the length of sliding window was not significantly different in result.However, to give better understanding what our algorithms perform in real-time we set as mentioned before.

E. Band Pass Filter
Afterwards, the signal filtering process is applied to remove the redundant noises.Because PR and RR signal are located in different bandwidths, we construct two bandpass Butterworth coefficient filters with different cutoff frequencies for pulse rate signal [0.4,2.5] Hz and respiratory rate signal [0.1, 0.5] Hz.The raw POS signal will be filtered with PR filter, and raw signal from CIE Lab will be filtered with RR filter.We set our filter parameter similar to related studies [25] - [27].

F. Moving Average
Due to non-ideal IIR filter performance, there are still some noises left on the both PR and RR signals.To avoid false positives PR and RR signal peak detection [28] - [30], we simply employ moving average operation using Equation ( 8) Where SMA is the output signal, yn is the input signal from IIR filter, n presents as an average order, which is set as 10 here.Fig. 3 shows the PR final signal after this stage with (a), (b), (c), and (d) are the signal formed from different illumination color and also (e) is the spectrogram between signals from proposed method and the reference device.Fig. 4 is the example of RR signal from red illumination with (a) is comparison signal using CIE Lab algorithm and RGB color space and also (b) is the spectrogram from L* channel signal with the reference signal.Both PR and RR signal need a peak detection algorithm to determine the parameter for further processing.We use peak detection algorithm based on study [31].Fig. 5 depicts the peak detection flow used in this study.In the beginning of the test, the Flag is set to false.When the slope value is larger than the threshold (Th), the Flag is set to true and this slope is recorded in the buffer.While the Flag is true and the slope value is smaller than or equal to 0, a peak index (peak position) is recorded.The maximum slope value from the buffer will be used to update the new threshold as adaptive thresholding and then clean the buffer to restart the peak detection.A peak-topeak interval (PPI) check is applied after peak detection to prevent noise contaminations.When the PPI is in the requested range, a peak is detected.

H. Parameter Calculation
For PRV parameter, we calculated both time and frequency domain.The detail equation used for PRV parameter presented by Table 1 where NN is same with peak-to-peak interval (PPI) from peak detection stage.The time domain parameter presented by Equation ( 9) -( 11) and frequency domain presented by Equation ( 12) - (14).
In other side, for respiration rate parameter we use operation from Equation (15).Where PPI(RR) is peak-topeak interval from respiration rate signal.

III. RESULT AND DISCUSSION
To evaluate the performance of our method, we built benchmark dataset.The video sequence was recorded twice from 5 health subjects.Hardware cost for the experiment were a low cost Logitech C920 pro HD RGB camera (setting in 480 x 640 pixels resolution, 8 bit depth, and run on average 30 fps), Philips Hue 7 W LED lamp (smartphone controllable to change the LED color), a laptop with an Intel Core i5 processor and the ground-truth data was recorded from contact-based PPG DASH 3000.All analyses and implementation were developed offline in MATLAB.
The experimental setup and procedures and the illumination exposure to the subject presented in Fig. 6.Each subject was asked to stand 50 cm in front of a desk with camera.The LED lamp was positioned around 1 m side from camera.Each video had same procedure and 4 minute overall long, for every 1 minute the illumination color changed (white-red-green-blue).Also, we close the window and the door to avoid another light source (e.g.sun light or lamp from other room).All subject need to control their breath corresponding to our scenario.Therefore, all of subject would have same respiration rate pattern.The breath procedure in 1 minute describe as follow: first 20 second breath for every 5 second (4 breath), then hold breath for 10 second, after that breath for every 2 second in 10 second span (5 breath), then hold breath again for 10 second and the next 10 second breath every 5 second (2 breath).This 1 minute breath procedure was repeated for every LED color illumination.
We used standard evaluation metrics: mean absolute error (MAE) from Equation ( 17) and root mean square error (RMSE) from Equation (18) for each PRV and RR parameter.For PRV parameter we done fair comparison with recently related method [15], the same processing technique from reference study was applied to our dataset.
The results from PRV and RR measurement are presented in Table 2 and Table 3 respectively.From the result, our proposed method has better performance and compared to related work at some parameters.To be noted, we also performed experiment in different light source condition to represent real condition of some applications, this experiment scheme is only few study to attempt.Furthermore, we tested the respiration rate measurement with scheme represent no breath, normal breath and fast breath also only few study to conduct this scenario.
The reason our proposed method is better from related study is because we use better rPPG algorithm to extract pixel image for PR and RR signal (POS algorithm currently one of the best rPPG method).Also we use correct spatial processing to deal with both PR and RR signal.For PR measurement, in a harmony with study [7] ICA method including in [15] is still not capable to give better result in different luminance color.And also this algorithm is very sensitive to large motion distortion.ICA will provide better result if only under good illumination and less motion noise, because the key of ICA algorithm is estimation (in PR frequency band) and component selection.For RR measurement case, most of rPPG method are detect movement caused by respiration and then calculate the RR.In this case, one of best accurate method to detect motion is by employing L* of CIELab color space.Based on our result, we outperformed blue channel from RGB color space which most study used this channel to compute their RR.Blue channel will have huge distortion if the illumination color are different.In contrast with blue channel method, L* channel are more illuminance color independent due to brightness component representation of color.As for the difference occurs between our proposed method and reference device were because the change of illumination color simultaneously (camera need to stable 1-2 second), ambient light that still captured by camera, and also motion noise.The frame rate of the camera had slightly drop when low brightness illumination applied (blue and red).When that happen our filter will not work optimally.In additional to be consideration, the frame rate of camera need to be fast enough.A clinical medical equipment are usually runs in more than 250 Hz of sampling rate frequency.
The limitations of our method, for current version follows: our algorithm are still run in offline mode.However, based on complexity of the algorithm we use, real-time performance is possible to made.The parts having more computational effort shown in Equation ( 2) and Equation (7) where we need to allocate amount memory in both process and take ceareffully if data complexity is exponentially growth.Our algorithm is less robust for respiration rate measurement if subjects do more movements.We had not test our algorithm in different movement when measurement taken.But we found short-time spike on our signal when more motion happened.
Our plan for future work, we will build application for our rPPG method which can work in real-time that will work in different light condition and for different kind of biometric from people of worldwide.We also strongly want to fulfill our desire to build an API framework, so our application can run in different platform (e.g.android, apple, Linux, and etc.).We also will try to verify our algorithm in different activities that include different movements (e.g.fitness, exercise activity).

IV. CONCLUSION
This paper presented robust rPPG algorithm to extract pulse rate variability (PRV) parameters and respiration rate (RR) in different illumination color and skin type.POS-based algorithm is utilized to convert frames pixels into pulse rate signal, while CIELab-based algorithm covers the RR signal conversion.We had done tested our method to 15 different subjects with various illumination scenarios.In the result section shown, proposed rPPG method is able to outperform most recent related study in state of the art of PRV and respiration rate measurement.And for RR measurement we also achieve performance compare to the conventional method.Our main focus for future work, we want to develop an application using our method runs in real-time and different platform.

Fig. 3 Final
Fig. 3 Final PR signal in different illumination color: white (black signal), red, green, blue, respectively and PR Spectrogram proposed method vs reference

Fig. 6
Fig. 6 Experimental setup and procedure