Biometric Authentication based on Liveness Detection Using Face Landmarks and Deep Learning Model

Ooi Jie - Tunku Abdul Rahman University of Management and Technology, Malaysia
Lim Ming - Tunku Abdul Rahman University of Management and Technology, Malaysia
Tan Wee - Tunku Abdul Rahman University of Management and Technology, Malaysia

Citation Format:



This paper describes the approach to active liveness detection of the face using facial features and movements. The project aims to create a better method for detecting liveness in real-time on an application programming interface (API) server. The project is built using Python programming with the computer vision libraries OpenCV, dlib and MediaPipe and the deep learning library Tensorflow. There are five modules in active liveness detection progress related to different parts or movements on the face: headshakes, nodding, eye blinks, smiles, and mouths. The functionality of modules runs through face landmarking through dlib and MediaPipe and detection of face features through Tensorflow Convolutional Neural Network (CNN) trained in two different approaches: smile detection and eye-blink detection. The result of implementing face landmarking shows an accurate result through the pre-trained model of MediaPipe and the pre-trained parameter of the dlib 68 landmarking model. And more than 90% classification model accuracy in precision, recall, and f1-score for both trained CNNs in detecting smiles and eyes blinking through the Scikit-Learn classification report. In addition, the prototype API is also implemented using the Python RESTful API library, FastAPI, to test the detection functionality in the prototype Android application. The prototype result is outstanding, as the model excellently requests and retrieves from the API server. The possible research path gives the success of real-time detection on API servers for easy implementation of liveness detection on low-spec client devices.


Liveness detection; face landmarking; perspective-n-point problem; deep learning; computer vision; face motion.

Full Text:



Mordor-Intelligence, "Facial Recognition Market - Growth, Trends, Covid-19 Impact, and Forecasts (2023 - 2028),", [Online]. Available:

D. Sharma and A. Selwal, "A survey on face presentation attack detection mechanisms: hitherto and future perspectives," Multimed Syst, vol. 29, no. 3, pp. 1527–1577, 2023, doi: 10.1007/s00530-023-01070-5.

J. K. Khan and D. Upadhyay, "Security issues in face recognition," 2014 5th International Conference - Confluence The Next Generation Information Technology Summit (Confluence), Sep. 2014, doi:

R. Singh, A. Agarwal, M. Singh, S. Nagpal, and M. Vatsa, "On the Robustness of Face Recognition Algorithms Against Attacks and Bias," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 09, pp. 13583–13589, Apr. 2020, doi:

M. C. Gentile, D. Danks, and M. Harrell, "Case Study: Does Facial Recognition Tech Enhance Security?," Harvard Business Review. Nov. 2022. [Online]. Available:

L. Li, P. L. Correia, and A. Hadid, "Face recognition under spoofing attacks: countermeasures and research directions," IET Biom, vol. 7, no. 1, pp. 3–14, Jan. 2018, doi:

S. Kumar, S. Singh, and J. Kumar, "A comparative study on face spoofing attacks," IEEE Xplore. pp. 1104–1108, May 2017. doi:

A. Hadid, "Face Biometrics Under Spoofing Attacks: Vulnerabilities, Countermeasures, Open Issues, and Research Directions," 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Jun. 2014, doi:

P. Kavitha and K. Vijaya, "A Study on Spoofing Face Detection System," 2017. [Online]. Available:

E. A. Raheem and Ahmad, "Statistical Analysis of Image Quality Measures for Face Liveness Detection," Lecture notes in electrical engineering, vol. 547, Jan. 2019, doi:

Y. Li, Y. Li, Q. Yan, H. Kong, and R. H. Deng, "Seeing Your Face Is Not Enough," Computer and Communications Security, Oct. 2015, doi:

D. Garud and S. S. Agrwal, "Face liveness detection," IEEE Xplore. pp. 789–792, Sep. 2016. doi:

J. Yang, Z. Lei, S. Liao, and S. Z. Li, "Face Liveness Detection with Component Dependent Descriptor," IEEE Xplore. pp. 1–6, Jun. 2013. doi:

A. Ali, F. Deravi, and S. Hoque, "Liveness Detection Using Gaze Collinearity," Oct. 2012. [Online]. Available:

L. Wang, X. Ding, and C. Fang, "Face live detection method based on physiological motion analysis," Tsinghua Sci Technol, vol. 14, no. 6, pp. 685–690, Dec. 2009, doi:

M. Jabberi, A. Wali, B. B. Chaudhuri, and A. M. Alimi, "68 landmarks are efficient for 3D face alignment: what about more?," Multimed Tools Appl, Apr. 2023, doi:

S. Hangaragi, T. Singh, and N. N, "Face Detection and Recognition Using Face Mesh and Deep Neural Network," Procedia Comput Sci, vol. 218, pp. 741–749, 2023, doi:

V. Bazarevsky, Y. Kartynnik, A. Vakunov, K. Raveendran, and M. Grundmann, "BlazeFace: Sub-millisecond Neural Face Detection onMobile GPUs," arXiv:1907.05047 [cs], Jul. 2019, [Online]. Available:

P. Shaha, U. Sharma, and K. Pawar, "Face Recognition Technology," International Journal of Research in Engineering, Science and Management, vol. 1, no. 9, pp. 149–151, Sep. 2018, [Online]. Available:

S. Chakraborty and D. Das, "An Overview of Face Liveness Detection," International Journal on Information Theory, vol. 3, no. 2, pp. 11–25, Apr. 2014, doi:

S. Li, X. Dong, Y. Shi, B. Lu, L. Sun, and W. Li, "Multi-angle head pose classification with masks based on color texture analysis and stack generalization," Concurr Comput, vol. 35, no. 18, 2023, doi: 10.1002/cpe.6331.

C. Gao, X. Li, F. Zhou, and S. Mu, "Face liveness detection based on the improved CnN with context and texture information," Chinese Journal of Electronics, vol. 28, no. 6, pp. 1092–1098, 2019, doi: 10.1049/cje.2019.07.012.

J. Whitehill, G. Littlewort, I. Fasel, M. Bartlett, and J. Movellan, "Toward Practical Smile Detection," IEEE Trans Pattern Anal Mach Intell, vol. 31, no. 11, pp. 2106–2111, Nov. 2009, doi:

I. Grishchenko and V. Bazarevsky, "MediaPipe Holistic — Simultaneous Face, Hand and Pose Prediction, on Device," Google AI Blog. Dec. 2020. [Online]. Available:

L. Fraiture, "A History of the Description of the Three-Dimensional Finite Rotation," Journal of The Astronautical Sciences, vol. 57, no. 1–2, pp. 207–232, Jan. 2009, doi:

X. Zhao, S. Sulaiman, L. Chen, M. Dong, Y. Duo, and H. Song, "Continuity Rotation Representation for Head Pose Estimation without Keypoints," in ACM International Conference Proceeding Series, 2023, pp. 358–363. doi: 10.1145/3594315.3594341.

F. Rocca, M. Mancas, and B. Gosselin, "Head Pose Estimation by Perspective-n-Point Solution Based on 2D Markerless Face Tracking," Springer eBooks, vol. 136, pp. 67–76, Jul. 2014, doi:

E. Garea-Llano and A. Morales-Gonzalez, "Framework for biometric iris recognition in video, by deep learning and quality assessment of the iris-pupil region," J Ambient Intell Humaniz Comput, vol. 14, no. 6, pp. 6517–6529, 2023, doi: 10.1007/s12652-021-03525-x.

V. Panwar and Pooja, "A Review on Iris Recognition System using Machine and Deep Learning," in 3rd IEEE 2022 International Conference on Computing, Communication, and Intelligent Systems, ICCCIS 2022, 2022, pp. 857–866. doi: 10.1109/ICCCIS56430.2022.10037643.

N. K. Singh, S. Mishra, and A. Bhardwaj, "Eye Blinking Detection Test," in Proceedings - 2021 3rd International Conference on Advances in Computing, Communication Control and Networking, ICAC3N 2021, 2021, pp. 1734–1736. doi: 10.1109/ICAC3N53548.2021.9725633.

R. Rao and V. N. Hedge, "Recognition and Classification of Smiles using Computer Vision," in 2022 1st International Conference on Artificial Intelligence Trends and Pattern Recognition, ICAITPR 2022, 2022. doi: 10.1109/ICAITPR51569.2022.9844198.

L. Ruan, Y. Han, J. Sun, Q. Chen, and J. Li, "Facial expression recognition in facial occlusion scenarios: A path selection multi-network," Displays, vol. 74, 2022, doi: 10.1016/j.displa.2022.102245.

P. Bansal and A. Ouda, "Study on Integration of FastAPI and Machine Learning for Continuous Authentication of Behavioral Biometrics," in 2022 International Symposium on Networks, Computers and Communications (ISNCC), 2022, pp. 1–6. doi: 10.1109/ISNCC55209.2022.9851790.

P. Bansal and A. Ouda, "Study on Integration of FastAPI and Machine Learning for Continuous Authentication of Behavioral Biometrics," in 2022 International Symposium on Networks, Computers and Communications, ISNCC 2022, 2022. doi: 10.1109/ISNCC55209.2022.9851790.

K. Cao, Y. Liu, G. Meng, and Q. Sun, "An Overview on Edge Computing Research," IEEE Access, vol. 8, pp. 85714–85728, 2020, doi:


  • There are currently no refbacks.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

JOIV : International Journal on Informatics Visualization
ISSN 2549-9610  (print) | 2549-9904 (online)
Organized by Society of Visual Informatocs, and Institute of Visual Informatics - UKM and Soft Computing and Data Mining Centre - UTHM
W :
E :,,

View JOIV Stats

Creative Commons License is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.