A Systematic Literature Review on Characteristics Influencing Software Reliability

Lehka Subramanium - Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang, Selangor, Malaysia
Saadah Hassan - Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang, Selangor, Malaysia
Mohd. Hafeez Osman - Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang, Selangor, Malaysia
Hazura Zulzalil - Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang, Selangor, Malaysia


Citation Format:



DOI: http://dx.doi.org/10.62527/joiv.8.4.3665

Abstract


Reliability, as a non-functional requirement, is a crucial aspect that refers to the system's ability to perform its intended functions consistently and without failure over an extended period. It is essential in designing and implementing software systems, as it affects software quality. Maintaining software reliability is a significant challenge, as it is directly impacted by factors such as the complexity of the software design, the amount of code, and the measures taken to secure the system from unauthorized use. There are significant growing appeals for predicting reliability to account for risks. Research on reliability risk assessment has a long tradition; unfortunately, comprehensible reliability characteristics are still vague when determining potential risks. Clearly defining, prioritizing, and addressing reliability characteristics is essential for delivering reliable, high-quality software that meets user needs and business goals. The ignorance and lack of comprehensive reliability characteristics have evolved into inaccurate risk assessment, triggering malfunctions in the operational environment. Comprehensive characteristics are key elements to predict and estimate software reliability. The reliability characteristics could determine the precise objective of reliability efforts. This systematic literature review aims to identify the key characteristics influencing software reliability, the potential risks associated with these characteristics, and the metrics used to measure and assess them. Thirty-one research articles related to research questions have been reviewed. The findings indicate that comprehensive reliability characteristics could identify, classify, and prioritize potential risks, improving current metrics. It can be concluded that the accurate potential reliability risk can demonstrate the consequence of failure.

Keywords


Reliability; characteristics; risk; metrics; systematic literature review

Full Text:

PDF

References


L. Subramanium, S. Hassan, H. Zulzalil, and M. H. Osman, “Identification of Emergent Properties Occurrences Factors in System-of-Systems,” 2023 IEEE Int. Conf. Comput. ICOCO 2023, pp. 71–76, 2023, doi: 10.1109/icoco59262.2023.10397923.

V. K. Singh, R. A. Khan, and S. W. Abbas Rizvi, “Revisiting Software Reliability Engineering with Fuzzy Techniques,” pp. 1037–1042, 2016.

A. Mohan and S. K. Jha, “Predicting and accessing reliability of components in component based software development,” 2019 Int. Conf. Intell. Comput. Control Syst. ICCS 2019, no. Iciccs, pp. 1110–1114, 2019, doi: 10.1109/iccs45141.2019.9065290.

C. Haritha Madhav and K. S. Vipin Kumar, “A method for predicting software reliability using object oriented design metrics,” 2019 Int. Conf. Intell. Comput. Control Syst. ICCS 2019, no. Iciccs, pp. 679–682, 2019, doi: 10.1109/iccs45141.2019.9065541.

A. A. Baybulatov and G. Promyslov, “A Metric for the IACS Availability Risk Assessment,” Proc. - 2022 Int. Russ. Autom. Conf. RusAutoCon 2022, pp. 750–754, 2022, doi:10.1109/rusautocon54946.2022.9896250.

A. Jatain and Y. Mehta, “Metrics and models for Software Reliability: A systematic review,” Proc. 2014 Int. Conf. Issues Challenges Intell. Comput. Tech. ICICT 2014, pp. 210–214, 2014, doi:10.1109/icicict.2014.6781281.

O. Stover, P. Karve, and S. Mahadevan, “Reliability and risk metrics to assess operational adequacy and flexibility of power grids,” Reliab. Eng. Syst. Saf., vol. 231, no. November 2022, p. 109018, 2023, doi:10.1016/j.ress.2022.109018.

C. Wang and A. Mosleh, “Qualitative-Quantitative Bayesian Belief Networks for reliability and risk assessment,” 2010 Proceedings - Annual Reliability and Maintainability Symposium (RAMS), pp. 1–5, Jan. 2010, doi: 10.1109/rams.2010.5448022.

T. Hovorushchenko, “The software emergent properties and them reflection in the non-functional requirements and quality models,” Proc. Int. Conf. Comput. Sci. Inf. Technol. CSIT 2015, no. September, pp. 146–153, 2015, doi: 10.1109/stc-csit.2015.7325454.

D. G. Lubas, “Department of defense system of systems reliability challenges,” Proc. - Annu. Reliab. Maintainab. Symp., pp. 1–6, 2017, doi: 10.1109/ram.2017.7889676.

L. Fan and Z. Ma, “Tendency analysis of software reliability engineering,” ICRMS’2011 - Saf. First, Reliab. Prim. Proc. 2011 9th Int. Conf. Reliab. Maintainab. Saf., pp. 771–773, 2011, doi:10.1109/icrms.2011.5979369.

S. Jayatilleka, “Intersection of systems and reliability engineering during new product development process,” Proc. - Annu. Reliab. Maintainab. Symp., vol. 2020-Janua, 2020, doi:10.1109/rams48030.2020.9153653.

F. M. Safie, R. G. Stutts, and Z. Huang, “Reliability and probabilistic risk assessment - How they play together,” Proc. - Annu. Reliab. Maintainab. Symp., vol. 2015-May, pp. 1–5, 2015, doi:10.1109/rams.2015.7105058.

J. Luo, H. Li, and S. Wang, “A quantitative reliability assessment and risk quantification method for microgrids considering supply and demand uncertainties Loss of Energy Expected Loss of Load Expected Loss of Power Supply Probability,” Appl. Energy, vol. 328, no. September, p. 120130, 2022, doi: 10.1016/j.apenergy.2022.120130.

J. Ai, W. Su, and F. Wang, “Software Reliability Evaluation Method Based on a Software Network,” Proc. - 29th IEEE Int. Symp. Softw. Reliab. Eng. Work. ISSREW 2018, pp. 136–137, 2018, doi:10.1109/issrew.2018.00-15.

A. Boranbayev, S. Boranbayev, and A. Nurusheva, “Development of a software system to ensure the reliability and fault tolerance in information systems based on expert estimates,” 2018, doi:10.1007/978-3-030-01057-7_68.

R. Mijumbi, K. Okumoto, A. Asthana, and J. Meekel, “Recent Advances in Software Reliability Assurance,” Proc. - 29th IEEE Int. Symp. Softw. Reliab. Eng. Work. ISSREW 2018, pp. 77–82, 2018, doi:10.1109/issrew.2018.00-27.

Y. Liu, M. Lu, and B. Xu, “Software reliability case development method based on software reliability characteristic model and measures of defect control,” Proc. IEEE Int. Conf. Softw. Eng. Serv. Sci. ICSESS, vol. 0, pp. 1–6, 2016, doi:10.1109/icsess.2016.7883004.

Y. Zhao, J. Gong, Y. Hu, Z. Liu, and L. Cai, “Analysis of quality evaluation based on ISO/IEC SQuaRE series standards and its considerations,” Proc. - 16th IEEE/ACIS Int. Conf. Comput. Inf. Sci. ICIS 2017, pp. 245–250, 2017, doi: 10.1109/icis.2017.7960001.

M. A. Al Imran, S. P. Lee, and M. A. M. Ahsan, “Measuring impact factors to achieve conflict-free set of quality attributes,” 2017 IEEE 8th Control Syst. Grad. Res. Colloquium, ICSGRC 2017 - Proc., no. August, pp. 174–178, 2017, doi: 10.1109/icsgrc.2017.8070590.

H. Al-Kilidar, K. Cox, and B. Kitchenham, “The use and usefulness of the ISO/IEC 9126 quality standard,” 2005 Int. Symp. Empir. Softw. Eng. ISESE 2005, pp. 126–132, 2005, doi:10.1109/isese.2005.1541821.

J. Eckhardt, A. Vogelsang, and D. M. Fernández, “Are ‘Non-functional’ Requirements really Non-functional? An Investigation of Non-functional Requirements in Practice,” Lect. Notes Informatics (LNI), Proc. - Ser. Gesellschaft fur Inform., vol. P-267, pp. 105–106, 2016.

S. Zhu, M. Lu, and B. Xu, “Software Reliability Case Development Method Based on the 4+1 Principles,” Proc. - 12th Int. Conf. Reliab. Maint. Safety, ICRMS 2018, pp. 197–202, 2018, doi:10.1109/icrms.2018.00045.

F. P. Juniawan et al., “E-Voting Software Quality Analysis with McCall’s Method,” 2020 8th Int. Conf. Cyber IT Serv. Manag. CITSM 2020, pp. 7–11, 2020, doi: 10.1109/citsm50537.2020.9268854.

J. Zhang, “Field Product Reliability Risk Assessment,” Proc. - Annu. Reliab. Maintainab. Symp., vol. 2022-Janua, pp. 1–6, 2022, doi:10.1109/rams51457.2022.9894003.

N. Singh and K. Tyagi, “Important factors for estimating reliability of SOA,” Conf. Proceeding - 2015 Int. Conf. Adv. Comput. Eng. Appl. ICACEA 2015, pp. 381–386, 2015, doi:10.1109/icacea.2015.7164734.

A. Quyoum, M.-U.-D. Dar, and S. M. K. Quadri, “Improving Software Reliability using Software Engineering Approach- A Review,” Int. J. Comput. Appl., vol. 10, no. 5, pp. 41–47, 2010, doi: 10.5120/1474-1990.

B. Kitchenham, “Procedures for Performing Systematic Reviews,” Empir. Softw. Eng., vol. 33, no. 2004, pp. 1–26, 2004.

M. Lepmets, E. Ras, and A. Renault, “A quality measurement framework for IT services,” Proc. - 2011 Annu. SRII Glob. Conf. SRII 2011, pp. 767–774, 2011, doi: 10.1109/srii.2011.84.

S. Yin, Q. Shi, Y. Wang, and C. Chen, “Summary of software reliability Research,” IOP Conf. Ser. Mater. Sci. Eng., vol. 1043, no. 5, 2021, doi: 10.1088/1757-899X/1043/5/052039.

X. Pan and M. Zhang, “Quality and Reliability Improvement Based on the Quality Function Deployment Method,” Proc. - 12th Int. Conf. Reliab. Maint. Safety, ICRMS 2018, pp. 38–42, 2018, doi:10.1109/icrms.2018.00018.

J. Klohoker, “A risk informed approach to reliability requirements tailoring,” Proc. - Annu. Reliab. Maintainab. Symp., vol. 2019-Janua, pp. 1–4, 2019, doi: 10.1109/rams.2019.8768994.

M. Radu, “Reliability and fault tolerance analysis of FPGA platforms,” 2014 IEEE Long Isl. Syst. Appl. Technol. Conf. LISAT 2014, pp. 1–4, 2014, doi: 10.1109/lisat.2014.6845211.

P. Garraghan et al., “Emergent Failures: Rethinking Cloud Reliability at Scale,” IEEE Cloud Comput., vol. 5, no. 5, pp. 12–21, 2018, doi:10.1109/mcc.2018.053711662.

J. Wang, “Model of Open Source Software Reliability with Fault Introduction Obeying the Generalized Pareto Distribution,” Arab. J. Sci. Eng., vol. 46, no. 4, pp. 3981–4000, 2021, doi: 10.1007/s13369-021-05382-4.

P. S. Sabnis, S. Joshi, and J. Naveenkumar, “A Study on Machine Learning Techniques based Software Reliability Assessment,” 4th Int. Conf. Inven. Res. Comput. Appl. ICIRCA 2022 - Proc., no. Icirca, pp. 687–692, 2022, doi: 10.1109/icirca54612.2022.9985530.

E. Bagheri and F. Ensan, “Reliability estimation for component-based software product lines,” Can. J. Electr. Comput. Eng., vol. 37, no. 2, pp. 94–112, 2014, doi: 10.1109/cjece.2014.2323958.

Y. Li, W. Wang, and X. Leng, “A Mission Reliability Method ( MRM ) for Risk Management in the Development of Materiel System,” p. 5, 2010.

E. Cota, “Adjusting reliability predictions for risk,” 2017 Annual Reliability and Maintainability Symposium (RAMS), pp. 1–5, 2017, doi: 10.1109/ram.2017.7889717.

Q. Li, L. Luo, and J. Wang, “Accelerated reliability testing approach for high-reliablity software based on the reinforced operational profile,” 2013 IEEE Int. Symp. Softw. Reliab. Eng. Work. ISSREW 2013, pp. 337–342, 2013, doi: 10.1109/issrew.2013.6688917.

B. Cukic and J. Dong, “Availability Monitor for a Software Based System,” Proc. IEEE Int. Symp. High Assur. Syst. Eng., pp. 321–328, 2007, doi: 10.1109/hase.2007.49.

L. C. Hao, L. J. Wu, R. Yan, X. Y. Han, and L. L. Tang, “Research on Software Reliability Index Allocation Method Based on Network Architecture,” Proc. 2019 Int. Conf. Qual. Reliab. Risk, Maintenance, Saf. Eng. QR2MSE 2019, no. Qr2mse, pp. 551–556, 2019, doi:10.1109/qr2mse46217.2019.9021200.

S. M. Yacoub and H. H. Ammar, “A methodology for architecture-level reliability risk analysis,” IEEE Trans. Softw. Eng., vol. 28, no. 6, pp. 529–547, 2002, doi: 10.1109/tse.2002.1010058.

L. Chang, X. Song, and L. Zhang, “Uncertainty-oriented reliability and risk-based output control for complex systems with compatibility considerations,” Inf. Sci. (Ny)., vol. 606, pp. 512–530, 2022, doi:10.1016/j.ins.2022.05.068.

C. Ji, D. Wu, D. Cheng, and Z. Shen, “Software-hardware interdependent reliability assessment technique for software-intensive complex systems,” ICRMS 2014 - Proc. 2014 10th Int. Conf. Reliab. Maintainab. Saf. More Reliab. Prod. More Secur. Life, pp. 493–500, 2014, doi: 10.1109/icrms.2014.7107246.

K. Okumoto, A. Asthana, and R. Mijumbi, “BRACE: Cloud-based software reliability assurance,” Proc. - 2017 IEEE 28th Int. Symp. Softw. Reliab. Eng. Work. ISSREW 2017, pp. 57–60, 2017, doi:10.1109/issrew.2017.48.

V. Gaur, O. P. Yadav, G. Soni, and A. P. S. Rathore, “A Review of Metrics, Algorithms and Methodologies for Network Reliability,” IEEE Int. Conf. Ind. Eng. Eng. Manag., pp. 1129–1133, 2019, doi:10.1109/ieem44572.2019.8978688.

J. Ludwig, S. Xu, and F. Webber, “Compiling static software metrics for reliability and maintainability from GitHub repositories,” 2017 IEEE Int. Conf. Syst. Man, Cybern. SMC 2017, vol. 2017-Janua, pp. 5–9, 2017, doi: 10.1109/smc.2017.8122569.