Selecting Control Menu on Electric Wheelchair Using Eyeball Movement for Difable Person

Fitri Utaminingrum - Brawijaya University, Malang, Indonesia
I Komang Somawirata - National Institute of Technology Malang, Malang, Indonesia
Gusti Pengestu - Bina Nusantara University, Jakarta, Indonesia
Tipajin Thaipisutikul - Mahidol University, Thailand
Timothy Shih - National Central University, Taoyuan, Taiwan

Citation Format:



Each country's number of people with disabilities and strokes increases yearly. Hand defects and stroke make them have limitations in doing activities. It caused their hand has paralyzed. Hence, they find it difficult to do daily activities, such as running a wheelchair, choosing a menu on the screen display, and so on. One solution offered is utilizing eye movement as a navigation tool that can replace the role of the user's hand, so they can run a wheelchair independently or choose a menu selection on display by themselves through the movement of their eyes. Detection of eyeball movements in this study only utilizes a camera as a sensor mounted in front of the user. So that it is more practical and easier to use than if we have to pair an electrooculography sensor in the area around the user's eyes. This research proposed a new approach to detect the five gazes (upward, downward, leftward, rightward, and forward) of the eyeball movements by using Backpropagation Neural Network (BPNN) and Dynamic Line Sector Coordinate (DLSC). Line Sector Coordinate is used to detect the eyeball movement based on the pupil coordinate position. The eyeball movement direction was analyzed from four lengths of a line. Our proposed method can detect five gaze directions that can be used for selecting four menus on the display monitor. The mean accuracy of our proposed method to detect eye movements for each gaze is 88.6%.


Disabilities, Detection; Eyeball Movements; Eyeball Movements; Dynamic Line Sector Coordinate

Full Text:



B. Vijayalaxmi, C. Anuradha, K. Sekaran, M. N. Meqdad, and S. Kadry, "Image processing based eye detection methods a theoretical review," Bull. Electr. Eng. Informatics, vol. 9, no. 3, pp. 1189–1197, Feb. 2020, doi: 10.11591/eei.v9i3.1783.

Vijayalaxmi and Sreehari, "Knowledge based template for Eye detection," in National Conference on Microwave, Antenna and Signal Processing, 2011, pp. 90–94.

S. S, K. N, and S. Natarajan, "Feed Forward Neural Network Based Eye Localization and Recognition Using Hough Transform," Int. J. Adv. Comput. Sci. Appl., vol. 2, no. 3, 2011, doi: 10.14569/IJACSA.2011.020318.

F. Wadehn, T. Weber, D. J. Mack, T. Heldt, and H.-A. Loeliger, "Model-Based Separation, Detection, and Classification of Eye Movements," IEEE Trans. Biomed. Eng., vol. 67, no. 2, pp. 588–600, Feb. 2020, doi: 10.1109/TBME.2019.2918986.

B. Yan, T. Pei, and X. Wang, "Wavelet Method for Automatic Detection of Eye-Movement Behaviors," IEEE Sens. J., vol. 19, no. 8, pp. 3085–3091, Apr. 2019, doi: 10.1109/JSEN.2018.2876940.

F. Sadikoglu and S. Uzelaltinbulat, “Biometric Retina Identification Based on Neural Network,†Procedia Comput. Sci., vol. 102, pp. 26–33, 2016, doi: 10.1016/j.procs.2016.09.365.

J. Rose, Y. Liu, and A. Awad, "Biometric Authentication Using Mouse and Eye Movement Data," in 2017 IEEE Security and Privacy Workshops (SPW), May 2017, pp. 47–55, doi: 10.1109/SPW.2017.18.

S. Plesnick, D. Repice, and P. Loughnane, "Eye-controlled wheelchair," in 2014 IEEE Canada International Humanitarian Technology Conference - (IHTC), Jun. 2014, pp. 1–4, doi: 10.1109/IHTC.2014.7147553.

F. Utaminingrum, P. P. Adikara, Y. A. Sari, D. Syauqy, and A. G. Hapsani, "Left-Right Head Movement for Controlling Smart Wheelchair By Using Centroid Coordinates Distance," J. Theor. Appl. Inf. Technol., vol. 96, no. 19, pp. 2852–2861, 2018.

B. W. Miller, "Using Reading Times and Eye-Movements to Measure Cognitive Engagement," Educ. Psychol., vol. 50, no. 1, pp. 31–42, Jan. 2015, doi: 10.1080/00461520.2015.1004068.

A. Siripitakchai, S. Phimoltares, and A. Mahaweerawat, "EYE-CAPTCHA: An enhanced CAPTCHA using eye movement," in 2017 3rd IEEE International Conference on Computer and Communications (ICCC), Dec. 2017, pp. 2120–2126, doi: 10.1109/CompComm.2017.8322911.

C. Atmaji, A. E. Putra, and A. Hanif, "Sliding window method for eye movement detection based on electrooculogram signal," in 2018 International Conference on Information and Communications Technology (ICOIACT), Mar. 2018, pp. 628–632, doi: 10.1109/ICOIACT.2018.8350779.

A. Bulling, J. A. Ward, H. Gellersen, and G. Tröster, "Eye Movement Analysis for Activity Recognition Using Electrooculography," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 4, pp. 741–753, Apr. 2011, doi: 10.1109/TPAMI.2010.86.

K. Arai, "Mobile Phone Operations using Human Eyes Only and its Applications," Int. J. Adv. Comput. Sci. Appl., vol. 9, no. 3, 2018, doi: 10.14569/IJACSA.2018.090322.

F. Utaminingrum, M. A. Fauzi, Y. A. Sari, R. Primaswara, and S. Adinugroho, "Eye Movement as Navigator for Disabled Person," in Proceedings of the 2016 International Conference on Communication and Information Systems - ICCIS '16, 2016, pp. 1–5, doi: 10.1145/3023924.3023926.

S. N. Patel and V. Prakash, "Autonomous camera based eye controlled wheelchair system using raspberry-pi," in 2015 International Conference on Innovations in Information, Embedded and Communication Systems (ICIIECS), Mar. 2015, pp. 1–6, doi: 10.1109/ICIIECS.2015.7192876.

R. P. Prasetya and F. Utaminingrum, "Triangle similarity approach for detecting eyeball movement," in 2017 5th International Symposium on Computational and Business Intelligence (ISCBI), Aug. 2017, pp. 37–40, doi: 10.1109/ISCBI.2017.8053540.

C. Yaiprasert, "Artificial Intelligence for Para Rubber Identification Combining Five Machine Learning Methods," Karbala Int. J. Mod. Sci., vol. 7, no. 4, Dec. 2021, doi: 10.33640/2405-609X.3154.

G. Pangestu, F. Utaminingrum, and F. A. Bachtiar, "Eyeball Movement Detection System using Corner Triangle Similarity, Na ̈ıve Bayes, and Ear Approach," Int. J. Adv. Soft Comput. its Appl., vol. 11, pp. 1–14, 2019.

M. S. Islam, M. Ahsan Ullah, and J. Prakash Dhar, "An imperceptible & robust digital image watermarking scheme based on DWT, entropy and neural network," Karbala Int. J. Mod. Sci., vol. 5, no. 1, Mar. 2019, doi: 10.33640/2405-609X.1068.

P. Viola and M. J. Jones, "Robust real-time face detection," Int. J. Comput. Vis., vol. 57, no. 2, pp. 137–154, 2004.

F. N. Ibrahim, Z. M. Zin, and N. Ibrahim, "Eye Center Detection Using Combined Viola-Jones and Neural Network Algorithms," in 2018 International Symposium on Agent, Multi-Agent Systems and Robotics (ISAMSR), Aug. 2018, pp. 1–6, doi: 10.1109/ISAMSR.2018.8540543.

A. Priadana and M. Habibi, "Face Detection using Haar Cascades to Filter Selfie Face Image on Instagram," in 2019 International Conference of Artificial Intelligence and Information Technology (ICAIIT), Mar. 2019, pp. 6–9, doi: 10.1109/ICAIIT.2019.8834526.

M. S. Uddin and A. Y. Akhi, "Horse Detection Using Haar Like Features," Int. J. Comput. Theory Eng., vol. 8, no. 5, pp. 415–418, Oct. 2016, doi: 10.7763/IJCTE.2016.V8.1081.

W. R. Gowers, "The Movements of the Eyelids," J. R. Soc. Med., vol. 62, no. 1, pp. 429–440, 2015.

H. Wang, "Linear Algebra Online Interactive Guiding Innovation Based on Big Data and Eye Movement Monitoring," in 2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS), May 2021, pp. 924–927, doi: 10.1109/ICICCS51141.2021.9432351.

S. Habib, I. Khan, S. Aladhadh, M. Islam, and S. Khan, "External Features-Based Approach to Date Grading and Analysis with Image Processing," Emerg. Sci. J., vol. 6, no. 4, pp. 694–704, 2022.

F. Laurene, Fundamentals of Neural Network, Architectures, Algorithm And Applications. New Jersey: Prentice Hall, Upper Sadle River, 1994.

G. Pangestu, F. Utaminingrum, and F. Bachtiar, "Eye State Recognition Using Multiple Methods for Applied to Control Smart Wheelchair," Int. J. Intell. Eng. Syst., vol. 12, no. 1, pp. 232–241, Feb. 2019, doi: 10.22266/ijies2019.0228.23.

M. Singh, P. Jain, and S. Chopra, "Eye movement detection for wheelchair control application," in 2015 International Conference on Electrical, Electronics, Signals, Communication and Optimization (EESCO), Jan. 2015, pp. 1–5, doi: 10.1109/EESCO.2015.7253877.