A Conceptual Framework for Personalized Early Prediction of Asthma Exacerbation Attacks Using Proximal Policy Optimization

Dahiru Aliyu - Universiti Teknologi PETRONAS, Seri Iskandar, 32610, Malaysia
Emelia Patah Akhir - Universiti Teknologi PETRONAS, Seri Iskandar, 32610, Malaysia
Nurul Osman - Universiti Teknologi PETRONAS, Seri Iskandar, 32610, Malaysia
Saidu Yahaya - Universiti Teknologi PETRONAS, Seri Iskandar, 32610, Malaysia
Shamsudden Adamu - Universiti Teknologi PETRONAS, Seri Iskandar, 32610, Malaysia
Hussaini Mamman - Universiti Teknologi PETRONAS, Seri Iskandar, 32610, Malaysia


Citation Format:



DOI: http://dx.doi.org/10.62527/joiv.8.4.2944

Abstract


Asthma, a chronic respiratory ailment affecting millions worldwide, presents significant challenges due to the unpredictable nature of exacerbation episodes. Existing methodologies struggle to accurately predict exacerbations individually, particularly across diverse patient demographics. This paper introduces an innovative conceptual framework for the early prediction of asthma exacerbations, leveraging advanced reinforcement learning (RL) techniques, specifically proximal policy optimization, along with patient-specific data and environmental factors. The primary goal is to revolutionize asthma management by providing customized predictions and tailored reward mechanisms that enable proactive interventions and optimize resource allocation. The framework comprises critical components such as patient profiling through a mobile application, trigger identification, a RL-based predictive model, an early warning mechanism, and a personalized reward scheme. Data for patient profiling is gathered through a mobile application, which includes medical history, demographics, symptoms, and triggers. Profiling forms the foundation for the prediction model, enabling it to identify subtle patterns and anticipate exacerbation events more accurately and efficiently. The significant contributions of this research include offering a novel approach by incorporating custom reward functions to enhance learning across heterogeneous patient populations, tailoring interventions to unique triggers and symptom presentations, and addressing challenges associated with patient diversity. By addressing the limitations of existing methodologies and offering a comprehensive solution, this research promises significant improvements in asthma care and healthcare delivery, paving the way for future advancements in personalized medicine and predictive healthcare systems.

Keywords


Asthma exacerbation; Attacks; Personalized; Prediction; Proximal policy optimization; Reinforcement learning

Full Text:

PDF

References


O. Enilari and S. Sinha, “The global impact of asthma in adult populatio,” Ann. Glob. Heal., vol. 85, no. 1, pp. 1–7, 2019, doi: 10.5334/aogh.2412.

D. Serebrisky and A. Wiznia, “Pediatric asthma: A global epidemic,” 2019, Ubiquity Press. doi: 10.5334/aogh.2416.

O. Zhang, L. L. Minku, and S. Gonem, “Detecting asthma exacerbations using daily home monitoring and machine learning,” J. Asthma, vol. 58, no. 11, pp. 1518–1527, 2021, doi: 10.1080/02770903.2020.1802746.

K. C. H. Tsang, H. Pinnock, A. M. Wilson, and S. Ahmar Shah, “Application of Machine Learning to Support Self-Management of Asthma with mHealth,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, 2020, pp. 5673–5677. doi: 10.1109/EMBC44109.2020.9175679.

J. G. Zein, C. P. Wu, A. H. Attaway, P. Zhang, and A. Nazha, “Novel Machine Learning Can Predict Acute Asthma Exacerbation,” Chest, vol. 159, no. 5, pp. 1747–1757, 2021, doi: 10.1016/j.chest.2020.12.051.

H. Pinnock, M. Noble, D. Lo, K. McClatchey, V. Marsh, and C. Y. Hui, “Personalised management and supporting individuals to live with their asthma in a primary care setting,” Expert Rev. Respir. Med., vol. 17, no. 7, pp. 577–596, Jul. 2023, doi: 10.1080/17476348.2023.2241357.

G. Wang and V. M. McDonald, “Contemporary Concise Review 2020: Asthma,” Respirology, vol. 26, no. 8, pp. 804–811, 2021, doi: 10.1111/resp.14099.

A. R. Kahkoska et al., “Individualized interventions and precision health: Lessons learned from a systematic review and implications for analytics-driven geriatric research,” J. Am. Geriatr. Soc., vol. 71, no. 2, pp. 383–393, 2023, doi: 10.1111/jgs.18141.

S. Xiong, W. Chen, X. Jia, Y. Jia, and C. Liu, “Machine learning for prediction of asthma exacerbations among asthmatic patients: a systematic review and meta-analysis,” BMC Pulm. Med., vol. 23, no. 1, 2023, doi: 10.1186/s12890-023-02570-w.

N. L. Lugogo et al., “A Predictive Machine Learning Tool for Asthma Exacerbations: Results from a 12-Week, Open-Label Study Using an Electronic Multi-Dose Dry Powder Inhaler with Integrated Sensors,” J. Asthma Allergy, vol. 15, pp. 1623–1637, 2022, doi: 10.2147/JAA.S377631.

C. Niu et al., “Evaluation of Risk Scores to Predict Pediatric Severe Asthma Exacerbations,” J. Allergy Clin. Immunol. Pract., vol. 9, no. 12, pp. 4393-4401.e8, 2021, doi: 10.1016/j.jaip.2021.08.030.

M. Lovrić, I. Banić, E. Lacić, K. Pavlović, R. Kern, and M. Turkalj, “Predicting treatment outcomes using explainable machine learning in children with asthma,” Children, vol. 8, no. 5, 2021, doi: 10.3390/children8050376.

J. L. M. Amaral, A. J. Lopes, J. Veiga, A. C. D. Faria, and P. L. Melo, “High-accuracy detection of airway obstruction in asthma using machine learning algorithms and forced oscillation measurements,” Comput. Methods Programs Biomed., vol. 144, pp. 113–125, 2017, doi: 10.1016/j.cmpb.2017.03.023.

S. Im et al., “Real-time counting of wheezing events from lung sounds using deep learning algorithms: Implications for disease prediction and early intervention,” PLoS One, vol. 18, no. 11 November, 2023, doi: 10.1371/journal.pone.0294447.

P. N. Pfeiffer et al., “Mobile health monitoring to characterize depression symptom trajectories in primary care,” J. Affect. Disord., vol. 174, pp. 281–286, 2015, doi: 10.1016/j.jad.2014.11.040.

M. B. M. Shdaifat, R. A. Khasawneh, and Q. Alefan, “Clinical and economic impact of telemedicine in the management of pediatric asthma in Jordan: a pharmacist-led intervention,” J. Asthma, vol. 59, no. 7, pp. 1452–1462, 2022, doi: 10.1080/02770903.2021.1924774.

S. D. Bennett and R. Shafran, “Adaptation, personalization and capacity in mental health treatments: A balancing act?,” Curr. Opin. Psychiatry, vol. 36, no. 1, pp. 28–33, 2023, doi: 10.1097/YCO.0000000000000834.

D. A. Aliyu et al., “Optimization Techniques for Asthma Exacerbation Prediction Models: A Systematic Literature Review,” IEEE Access, vol. 12, no. August, pp. 110862–110890, 2024, doi: 10.1109/ACCESS.2024.3440502.

A. Hodkinson et al., “Self-management interventions to reduce healthcare use and improve quality of life among patients with asthma: Systematic review and network meta-analysis,” BMJ, vol. 370, p. m2521, Aug. 2020, doi: 10.1136/bmj.m2521.

A. Licari et al., “Asthma endotyping and biomarkers in childhood asthma,” Pediatr. Allergy, Immunol. Pulmonol., vol. 31, no. 2, pp. 44–55, Jun. 2018, doi: 10.1089/ped.2018.0886.

E. Herrera-Luis et al., “Multi-ancestry genome-wide association study of asthma exacerbations,” Pediatr. Allergy Immunol., vol. 33, no. 6, 2022, doi: 10.1111/pai.13802.

N. Nakwan and K. Suansan, “In-hospital Mortality and Hospital Outcomes among Adults Hospitalized for Exacerbations of Asthma and COPD in Southern Thailand (2017-2021): A Population-Based Study,” Chinese Med. Sci. J., vol. 38, no. 3, pp. 228–234, 2023, doi: 10.24920/004252.

T. Y. Lee et al., “Individualised risk prediction model for exacerbations in patients with severe asthma: Protocol for a multicentre real-world risk modelling study,” BMJ Open, vol. 13, no. 3, 2023, doi: 10.1136/bmjopen-2022-070459.

R. T. Bhowmik and S. P. Most, “A Personalized Respiratory Disease Exacerbation Prediction Technique Based on a Novel Spatio-Temporal Machine Learning Architecture and Local Environmental Sensor Networks,” Electron., vol. 11, no. 16, 2022, doi: 10.3390/electronics11162562.

J. Bridge, J. D. Blakey, and L. J. Bonnett, “A systematic review of methodology used in the development of prediction models for future asthma exacerbation,” BMC Med. Res. Methodol., vol. 20, no. 1, pp. 1–13, 2020, doi: 10.1186/s12874-020-0913-7.

G. Racine, A. Forget, G. Moullec, T. Jiao, L. Blais, and C. Lemiere, “Predictors of Asthma Control and Exacerbations: A Real-World Study,” J. Allergy Clin. Immunol. Pract., vol. 9, no. 7, pp. 2802-2811.e2, Jul. 2021, doi: 10.1016/j.jaip.2021.04.049.

A. Tiotiu, I. Ioan, N. Wirth, R. Romero-Fernandez, and F.-J. González-Barcala, “The Impact of Tobacco Smoking on Adult Asthma Outcomes,” Int. J. Environ. Res. Public Health, vol. 18, no. 3, p. 992, Jan. 2021, doi: 10.3390/ijerph18030992.

F. Yang et al., “Factors Associated with Frequent Exacerbations in the UK Severe Asthma Registry,” J. Allergy Clin. Immunol. Pract., vol. 9, no. 7, pp. 2691-2701.e1, Jul. 2021, doi: 10.1016/j.jaip.2020.12.062.

S. Graff et al., “Clinical and biological factors associated with irreversible airway obstruction in adult asthma,” Respir. Med., vol. 175, p. 106202, Dec. 2020, doi: 10.1016/j.rmed.2020.106202.

R. Abrahamsen, G. F. Gundersen, M. V. Svendsen, G. Klepaker, J. Kongerud, and A. K. M. Fell, “Possible risk factors for poor asthma control assessed in a cross-sectional population-based study from Telemark, Norway,” PLoS One, vol. 15, no. 5, p. e0232621, May 2020, doi: 10.1371/journal.pone.0232621.

S. Soremekun et al., “Asthma exacerbations are associated with a decline in lung function: A longitudinal population-based study,” Thorax, vol. 78, no. 7, pp. 643–652, 2023, doi: 10.1136/thorax-2021-217032.

K. Lisspers et al., “Developing a short-term prediction model for asthma exacerbations from Swedish primary care patients’ data using machine learning - Based on the ARCTIC study,” Respir. Med., vol. 185, 2021, doi: 10.1016/j.rmed.2021.106483.

R. J. B. Loymans et al., “Identifying patients at risk for severe exacerbations of asthma: Development and external validation of a multivariable prediction model,” Thorax, vol. 71, no. 9, pp. 838–846, 2016, doi: 10.1136/thoraxjnl-2015-208138.

R. Howard, M. Rattray, M. Prosperi, and A. Custovic, “Distinguishing Asthma Phenotypes Using Machine Learning Approaches,” Curr. Allergy Asthma Rep., vol. 15, no. 7, 2015, doi: 10.1007/s11882-015-0542-0.

S. H. Khaleefah, S. A. Mostafa, S. S. Gunasekaran, U. F. Khattak, M. A. Jubair, and R. Afyenni, “Optimizing Smart Power Grid Stability Based on the Prediction of a Deep Learning Model,” JOIV Int. J. Informatics Vis., vol. 8, no. 3, 2024, [Online]. Available: http://joiv.org/index.php/joiv/article/view/2758

S. K. Mohamed, N. A. Sakr, and N. A. Hikal, “A Review of Breast Cancer Classification and Detection Techniques,” Int. J. Adv. Sci. Comput. Eng., vol. 3, no. 3, pp. 128–139, 2021, doi: 10.62527/ijasce.3.3.55.

Y. Liu, P. H. C. Chen, J. Krause, and L. Peng, “How to Read Articles That Use Machine Learning: Users’ Guides to the Medical Literature,” JAMA - J. Am. Med. Assoc., vol. 322, no. 18, pp. 1806–1816, 2019, doi: 10.1001/jama.2019.16489.

G. H. Tison, J. Zhang, F. N. Delling, and R. C. Deo, “Automated and Interpretable Patient ECG Profiles for Disease Detection, Tracking, and Discovery,” Circ. Cardiovasc. Qual. Outcomes, vol. 12, no. 9, 2019, doi: 10.1161/CIRCOUTCOMES.118.005289.

S. E. Awan, F. Sohel, F. M. Sanfilippo, M. Bennamoun, and G. Dwivedi, “Machine learning in heart failure: Ready for prime time,” Curr. Opin. Cardiol., vol. 33, no. 2, pp. 190–195, 2018, doi: 10.1097/HCO.0000000000000491.

X. lu Xiong, R. xin Zhang, Y. Bi, W. hong Zhou, Y. Yu, and D. long Zhu, “Machine Learning Models in Type 2 Diabetes Risk Prediction: Results from a Cross-sectional Retrospective Study in Chinese Adults,” Curr. Med. Sci., vol. 39, no. 4, pp. 582–588, 2019, doi: 10.1007/s11596-019-2077-4.

D. Spathis and P. Vlamos, “Diagnosing asthma and chronic obstructive pulmonary disease with machine learning,” Health Informatics J., vol. 25, no. 3, pp. 811–827, 2019, doi: 10.1177/1460458217723169.

S. Saglani and A. Custovic, “Childhood asthma: Advances using machine learning and mechanistic studies,” Am. J. Respir. Crit. Care Med., vol. 199, no. 4, pp. 414–422, 2019, doi: 10.1164/rccm.201810-1956CI.

T. Goto, C. A. Camargo, M. K. Faridi, B. J. Yun, and K. Hasegawa, “Machine learning approaches for predicting disposition of asthma and COPD exacerbations in the ED,” Am. J. Emerg. Med., vol. 36, no. 9, pp. 1650–1654, 2018, doi: 10.1016/j.ajem.2018.06.062.

S. J. Patel, D. B. Chamberlain, and J. M. Chamberlain, “A Machine Learning Approach to Predicting Need for Hospitalization for Pediatric Asthma Exacerbation at the Time of Emergency Department Triage,” Acad. Emerg. Med., vol. 25, no. 12, pp. 1463–1470, 2018, doi: 10.1111/acem.13655.

J. Zhang, Z. Zhang, S. Han, and S. Lü, “Proximal policy optimization via enhanced exploration efficiency,” Inf. Sci. (Ny)., vol. 609, pp. 750–765, 2022, doi: 10.1016/j.ins.2022.07.111.

C. Y. Tang, C. H. Liu, W. K. Chen, and S. D. You, “Implementing action mask in proximal policy optimization (PPO) algorithm,” ICT Express, vol. 6, no. 3, pp. 200–203, 2020, doi: 10.1016/j.icte.2020.05.003.

M. Kraft et al., “Patient characteristics, biomarkers and exacerbation risk in severe, uncontrolled asthma,” Eur. Respir. J., vol. 58, no. 6, 2021, doi: 10.1183/13993003.00413-2021.

Y. Gu, Y. Cheng, K. Yu, and X. Wang, “Anti-Martingale Proximal Policy Optimization,” IEEE Trans. Cybern., vol. 53, no. 10, pp. 6421–6432, 2023, doi: 10.1109/TCYB.2022.3170355.

H. Zenati, A. Bietti, M. Martin, E. Diemert, and J. Mairal, “Counterfactual Learning of Stochastic Policies with Continuous Actions: from Models to Offline Evaluation,” arXiv Mach. Learn., pp. 1–20, 2020, doi: https://doi.org/10.48550/arXiv.2004.11722.

Y. Sun, X. Yuan, W. Liu, and C. Sun, “Model-Based Reinforcement Learning via Proximal Policy Optimization,” Proc. - 2019 Chinese Autom. Congr. CAC 2019, pp. 4736–4740, 2019, doi: 10.1109/CAC48633.2019.8996875.

Y. Meng, S. Kuppannagari, R. Kannan, and V. Prasanna, “PPOAccel: A High-Throughput Acceleration Framework for Proximal Policy Optimization,” IEEE Trans. Parallel Distrib. Syst., vol. 33, no. 9, pp. 2066–2078, 2022, doi: 10.1109/TPDS.2021.3134709.

A. K. Sharma, S. Saini, P. Chhabra, S. K. Chhabra, C. Ghosh, and P. Baliyan, “Air pollution and weather as the determinants of acute attacks of asthma: Spatiotemporal approach,” Indian J. Public Health, vol. 64, no. 2, pp. 124–129, 2020, doi: 10.4103/ijph.IJPH_135_19.

J. Finkelstein and I. cheol Jeong, “Machine learning approaches to personalize early prediction of asthma exacerbations,” Ann. N. Y. Acad. Sci., vol. 1387, no. 1, pp. 153–165, Jan. 2017, doi: 10.1111/nyas.13218.

H. Hairani, T. Widiyaningtyas, and D. Dwi Prasetya, “Addressing Class Imbalance of Health Data: A Systematic Literature Review on Modified Synthetic Minority Oversampling Technique (SMOTE) Strategies,” JOIV Int. J. Informatics Vis., vol. 8, no. 3, pp. 1310–1318, 2024.

N. Aqilah, M. I. Jaya, I. Dyah, and T. H. Rassem, “Comparative Analysis of Imputation Methods for Enhancing Predictive Accuracy in Data Models,” JOIV Int. J. Informatics Vis., vol. 8, no. September, pp. 1271–1276, 2024.

M. Sulistiyono, L. A. Wirasakti, and Y. Pristyanto, “The Effect of Adaptive Synthetic and Information Gain on C4.5 and Naive Bayes in Imbalance Class Dataset,” Int. J. Adv. Sci. Comput. Eng., vol. 4, no. 1, pp. 1–11, 2022, doi: 10.30630/ijasce.4.1.70.

J. Castner, C. R. Jungquist, M. J. Mammen, J. J. Pender, O. Licata, and S. Sethi, “Prediction model development of women’s daily asthma control using fitness tracker sleep disruption,” Hear. Lung, vol. 49, no. 5, pp. 548–555, Sep. 2020, doi: 10.1016/j.hrtlng.2020.01.013.

V. Plevris, G. Solorzano, N. P. Bakas, and M. E. A. Ben Seghier, “Investigation of Performance Metrics in Regression Analysis and Machine Learning-Based Prediction Models,” in World Congress in Computational Mechanics and ECCOMAS Congress, CIMNE, 2022. doi: 10.23967/eccomas.2022.155.

S. V. Razavi-Termeh, A. Sadeghi-Niaraki, and S. M. Choi, “Asthma-prone areas modeling using a machine learning model,” Sci. Rep., vol. 11, no. 1, pp. 1–16, 2021, doi: 10.1038/s41598-021-81147-1.

S. V. Razavi-Termeh, A. Sadeghi-Niaraki, and S. M. Choi, “Effects of air pollution in Spatio-temporal modeling of asthma-prone areas using a machine learning model,” Environ. Res., vol. 200, 2021, doi: 10.1016/j.envres.2021.111344.

R. Khasha, M. M. Sepehri, and S. A. Mahdaviani, “An ensemble learning method for asthma control level detection with leveraging medical knowledge-based classifier and supervised learning,” J. Med. Syst., vol. 43, no. 6, p. 158, Jun. 2019, doi: 10.1007/s10916-019-1259-8.

Q. Do, A. Doig, T. C. Son, and J. Chaudri, “Portugal predicting lung healthiness risk scores to identify probability of an asthma attack,” Procedia Comput. Sci., vol. 160, pp. 424–431, 2019, doi: 10.1016/j.procs.2019.11.071.

Q. Do, S. Tran, and A. Doig, “Reinforcement Learning Framework to Identify Cause of Diseases-Predicting Asthma Attack Case,” Proc. - 2019 IEEE Int. Conf. Big Data, Big Data 2019, pp. 4829–4838, 2019, doi: 10.1109/BigData47090.2019.9006407.

M. Seo et al., “Combining individual patient data from randomized and non-randomized studies to predict real-world effectiveness of interventions,” Stat. Methods Med. Res., vol. 31, no. 7, pp. 1355–1373, Jul. 2022, doi: 10.1177/09622802221090759.

C. Boe, K. Ng, S. Haw, P. Naveen, and E. Abdulwahab, “An Automated Face Detection and Recognition for Class Attendance,” JOIV Int. J. Informatics Vis., vol. 8, no. September, pp. 1146–1153, 2024.

M. Perkonigg, J. Hofmanninger, and G. Langs, “Continual Active Learning for Efficient Adaptation of Machine Learning Models to Changing Image Acquisition,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12729 LNCS, 2021, pp. 649–660. doi: 10.1007/978-3-030-78191-0_50.

Q. Do, “REINFORCEMENT LEARNING FRAMEWORK TO IDENTIFY CAUSE OF DISEASES AN EXAMPLE OF PREDICTING ASTHMA ATTACK Quan Do A dissertation submitted to the Graduate School in partial fulfillment of the requirements for the degree PhD in Interdisciplinary Major : Interd,” 2019.

A. Budiarto, K. C. H. Tsang, A. M. Wilson, A. Sheikh, and S. A. Shah, “Machine Learning–Based Asthma Attack Prediction Models From Routinely Collected Electronic Health Records: Systematic Scoping Review,” Jmir Ai, vol. 2, p. e46717, Dec. 2023, doi: 10.2196/46717.

L. Zhou, S. Sun, H. Fu, and P. X. K. Song, “Subgroup-Effects Models for the Analysis of Personal Treatment Effects,” Ann. Appl. Stat., vol. 16, no. 1, pp. 80–103, Mar. 2022, doi: 10.1214/21-AOAS1503.

P. Liu, J. Li, and M. R. Kosorok, “Change plane model averaging for subgroup identification,” Stat. Methods Med. Res., vol. 32, no. 4, pp. 773–788, Apr. 2023, doi: 10.1177/09622802231154327.

E. T. Alharbi, F. Nadeem, and A. Cherif, “Predictive models for personalized asthma attacks based on patient’s biosignals and environmental factors: a systematic review,” BMC Med. Inform. Decis. Mak., vol. 21, no. 1, 2021, doi: 10.1186/s12911-021-01704-6.

H. Suresh, J. J. Gong, and J. V. Guttag, “Learning tasks for multitask learning: Heterogenous patient populations in the ICU,” Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., pp. 802–810, 2018, doi: 10.1145/3219819.3219930.

M. Binjubeir, M. A. Ismail, S. Kasim, H. Amnur, and Defni, “Big healthcare data: Survey of challenges and privacy,” Int. J. Informatics Vis., vol. 4, no. 4, pp. 184–190, 2020, doi: 10.30630/joiv.4.4.246.