An Investigation of the Student Learning Satisfaction Model for User Story Learning in Software Engineering Course

Muhammad Ihsan Zul - Politeknik Caltex Riau, Pekanbaru, Indonesia
Suhaila Mohd. Yasin - Universiti Tun Hussein Onn Malaysia, Batu Pahat, Johor, Malaysia
Dadang Syarif Sihabudin Sahid - Politeknik Caltex Riau, Pekanbaru, Indonesia


Citation Format:



DOI: http://dx.doi.org/10.62527/joiv.8.4.3089

Abstract


Software engineering courses are essential for students to become professional software engineers. These courses expose them to their first user stories (US). Despite extensive studies on US-related issues, quality remains the most prominently discussed topic. Therefore, it is essential to investigate US education in higher education to produce qualified software practitioners. In the educational context, such investigations are typically measured using the learning satisfaction approach. This study aims to investigate the suitability of the learning satisfaction model in software engineering courses, specifically in the US context. Subsequently, the study will identify opportunities for improving US teaching methods. The applied learning satisfaction model consists of four factors: perceived ease of use, perceived usefulness, learning motivation, and learning satisfaction. These factors are derived by combining the Technology Acceptance Model (TAM) and Learning Motivation Theory. The study employs Confirmatory Factor Analysis (CFA) using the partial least squares structural equation modelling (PLS-SEM). The measurement model and model evaluation fit stages are used to assess the suitability of the implemented learning satisfaction model. The structural model examines opportunities for improving the US teaching method based on the identified factors. The study involves 142 software engineering students as respondents. The results indicate that the current model requires adjustments in indicators and model fit, particularly SRMR and NFI, to align with the study. Regarding learning enhancement, the factors of perceived ease of use and perceived usefulness suggest that improvements in US teaching methods are necessary to increase student learning satisfaction in US learning.

Keywords


user story; learning satisfaction; confirmatory factor analysis; PLS-SEM

Full Text:

PDF

References


J. Patton and P. Economy, User Story Mapping: Discover The Whole Story, Build The Right Product, 1st ed. O’Reilly Media, Inc., 2014.

M. Cohn, User Stories Applied For Agile Software Development. Addison-Wesley, 2008.

A. Amna and G. Poels, “Ambiguity in user stories: A systematic literature review,” May 01, 2022, Elsevier B.V. doi: 10.1016/j.infsof.2022.106824.

F. Dalpiaz, I. van der Schalk, S. Brinkkemper, F. B. Aydemir, and G. Lucassen, “Detecting terminological ambiguity in user stories: Tool and experimentation,” Inf Softw Technol, vol. 110, pp. 3–16, Jun. 2019, doi: 10.1016/j.infsof.2018.12.007.

X. Xu, Y. Dou, L. Qian, J. Jiang, K. Yang, and Y. Tan, “Quality improvement method for high-end equipment’s functional requirements based on user stories,” Advanced Engineering Informatics, vol. 56, Apr. 2023, doi: 10.1016/j.aei.2023.102017.

J. Jia, X. Yang, R. Zhang, and X. Liu, “Understanding software developers’ cognition in agile requirements engineering,” Sci Comput Program, vol. 178, no. 0, pp. 1–19, Jun. 2019, doi: 10.1016/j.scico.2019.03.005.

M. Urbieta, L. Antonelli, G. Rossi, and J. C. S. do Prado Leite, “The impact of using a domain language for an agile requirement management,” Inf Softw Technol, vol. 127, Nov. 2020, doi: 10.1016/j.infsof.2020.106375.

G. B. Herwanto, G. Quirchmayr, and A. M. Tjoa, “PrivacyStory: Tool Support for Extracting Privacy Requirements from User Stories,” in Proceedings of the IEEE International Conference on Requirements Engineering, Melbourne: IEEE Computer Society, Aug. 2022, pp. 264–265. doi: 10.1109/RE54965.2022.00036.

J. F. Hair and M. Sarstedt, “Factors versus Composites: Guidelines for Choosing the Right Structural Equation Modeling Method,” Project Management Journal, vol. 50, no. 6, pp. 619–624, Dec. 2019, doi: 10.1177/8756972819882132.

P.-C. Muñoz-Carril, N. Hernández-Sellés, E.-J. Fuentes-Abeledo, and M. González-Sanmamed, “Factors influencing students’ perceived impact of learning and satisfaction in Computer Supported Collaborative Learning,” Comput Educ, vol. 174, p. 104310, Dec. 2021, doi: 10.1016/j.compedu.2021.104310.

H. Toring et al., “Evaluation of students’ satisfaction toward an adopted learning management system at Indiana Aerospace University: A structural equation modeling approach,” Asia Pacific Management Review, vol. 28, no. 3, pp. 336–346, Sep. 2023, doi: 10.1016/j.apmrv.2022.12.002.

C.-H. Huang, “Using PLS-SEM Model to Explore the Influencing Factors of Learning Satisfaction in Blended Learning,” Educ Sci (Basel), vol. 11, no. 5, p. 249, May 2021, doi: 10.3390/educsci11050249.

B. W. Gao, J. Jiang, and Y. Tang, “The effect of blended learning platform and engagement on students’ satisfaction—— the case from the tourism management teaching,” J Hosp Leis Sport Tour Educ, vol. 27, p. 100272, Nov. 2020, doi: 10.1016/j.jhlste.2020.100272.

C.-H. Huang, “The Influence of Self-Efficacy, Perceived Usefulness, Perceived Ease of Use, and Cognitive Load on Students’ Learning Motivation, Learning Attitude, and Learning Satisfaction in Blended Learning Methods,” in 2020 3rd International Conference on Education Technology Management, New York, NY, USA: ACM, Dec. 2020, pp. 29–35. doi: 10.1145/3446590.3446595.

L. Kim, P. Pongsakornrungsilp, S. Pongsakornrungsilp, T. Cattapan, and N. Nantavisit, “Determinants of perceived e-learning usefulness in higher education: A case of Thailand,” Innovative Marketing, vol. 18, no. 4, pp. 86–96, Nov. 2022, doi: 10.21511/im.18(4).2022.08.

A. A. Daneji, A. F. M. Ayub, and M. N. M. Khambari, “The effects of perceived usefulness, confirmation and satisfaction on continuance intention in using massive open online course (MOOC),” Knowledge Management & E-Learning: An International Journal, pp. 201–214, Jun. 2019, doi: 10.34105/j.kmel.2019.11.010.

D. S. S. Sahid, “Learner Behavior in e-Learning as a Multicriteria Attribute based on Perspective of Flow Experience,” International Journal of Advanced Computer Science and Applications, vol. 11, no. 12, 2020, doi: 10.14569/IJACSA.2020.0111235.

F. D. Davis, “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology,” MIS Quarterly, vol. 13, no. 3, p. 319, Sep. 1989, doi: 10.2307/249008.

H. S. Siregar, “Perceived Usefulness and Perceived Ease of Use of Online Learning for Islamic Religious Education Teacher,” Jurnal Pendidikan Islam, vol. 9, no. 1, pp. 93–106, Jun. 2023, doi: 10.15575/jpi.v0i0.25518.

N. Nuryakin, N. L. P. Rakotoarizaka, and H. G. Musa, “The Effect of Perceived Usefulness and Perceived Easy to Use on Student Satisfaction The Mediating Role of Attitude to Use Online Learning,” Asia Pacific Management and Business Application, vol. 011, no. 03, pp. 323–336, Apr. 2023, doi: 10.21776/ub.apmba.2023.011.03.5.

D. S. Wahyuni, G. Ariadi, N. Sugihartini, and I. N. E. Mertayasa, “The Impact of Achievement Motivation on Educational Compatibility mediated by Perceived Ease of Use,” in Proceedings of the 5th International Conference on Vocational Education and Technology, IConVET 2022, 6 October 2022, Singaraja, Bali, Indonesia, EAI, 2023. doi: 10.4108/eai.6-10-2022.2327429.

T. Thi Uyen Nguyen, P. Van Nguyen, H. Thi Ngoc Huynh, G. Q. Truong, and L. Do, “Unlocking e-government adoption: Exploring the role of perceived usefulness, ease of use, trust, and social media engagement in Vietnam,” Journal of Open Innovation: Technology, Market, and Complexity, vol. 10, no. 2, p. 100291, Jun. 2024, doi: 10.1016/j.joitmc.2024.100291.

G.-A. Olger, T.-A. Fiorela, C.-Q. Giuliana, D.-P. Aleixandre, S.-V. Leslie, and C.-H. Sandra, “Ease of use and perceived usefulness and its influence on motivation, collaboration and behavioral intention in university students in times of Covid-19,” in 2022 XVII Latin American Conference on Learning Technologies (LACLO), IEEE, Oct. 2022, pp. 1–6. doi: 10.1109/LACLO56648.2022.10013372.

W.-H. Huang, “Evaluating learners’ motivational and cognitive processing in an online game-based learning environment,” Comput Human Behav, vol. 27, no. 2, pp. 694–704, Mar. 2011, doi: 10.1016/j.chb.2010.07.021.

J. T. Nagy, “Evaluation of Online Video Usage and Learning Satisfaction: An Extension of the Technology Acceptance Model,” The International Review of Research in Open and Distributed Learning, vol. 19, no. 1, Feb. 2018, doi: 10.19173/irrodl.v19i1.2886.

S. Ramli, N. Razali, and N. Abdullah, “The Satisfaction Level Of Education Technology On Students’ Learning Process,” in Proceedings of the Proceedings of the First International Conference on Technology Management and Tourism, ICTMT, 19 August, Kuala Lumpur, Malaysia, EAI, 2020. doi: 10.4108/eai.19-8-2019.2293786.

W.-W. Chua and Y.-L. Ling, “Students motivation as predictors of learning satisfaction in a synchronous hybrid learning space,” ATTARBAWIY: Malaysian Online Journal of Education, vol. 6, no. 2, pp. 71–83, Dec. 2022, doi: 10.53840/attarbawiy.v6i2.143.

W. S. Chen and A. Y. Tat Yao, “An Empirical Evaluation of Critical Factors Influencing Learner Satisfaction in Blended Learning: A Pilot Study,” Universal Journal of Educational Research, vol. 4, no. 7, pp. 1667–1671, Jul. 2016, doi: 10.13189/ujer.2016.040719.

M. Q. Melchor and C. P. Julián, “The Impact of the Human Element in the Information Systems Quality for Decision Making and User Satisfaction,” Journal of Computer Information Systems, vol. 48, no. 2, pp. 44–52, 2008, doi: 10.1080/08874417.2008.11646008.

S. A. Nikou and A. A. Economides, “Mobile-Based Assessment: Integrating acceptance and motivational factors into a combined model of Self-Determination Theory and Technology Acceptance,” Comput Human Behav, vol. 68, pp. 83–95, Mar. 2017, doi: 10.1016/j.chb.2016.11.020.

C.-H. Huang, “Explore the Effects of Usefulness and Ease of Use in Digital Game-Based Learning on Students’ Learning Motivation, Attitude, and Satisfaction,” in 5th EAI International Conference Design, Learning, and Innovation, 2021, pp. 26–39. doi: 10.1007/978-3-030-78448-5_2.

I. F. Rachmi, F. R. Asta, and N. D. Kartiko, “The Effects of Perceived Ease of Use, Perceived Usefulness, and Computer Self-Efficacy on e-Nofa Application User Satisfaction,” in E3S Web of Conferences, T. N. Mursitama, Noerlina, E. Sitepu, and F. T. Basaria, Eds., May 2023, p. 04017. doi: 10.1051/e3sconf/202338804017.

D. Apriandi, H. Retnawati, and A. Maman Abadi, “Construct Validity and Reliability of the Learning Motivation Questionnaire,” TEM Journal, pp. 1494–1499, Nov. 2022, doi: 10.18421/TEM114-09.

E. Lameier, L. Reinerman-Jones, G. Matthews, E. Biddle, and M. Boyce, “Motivational Assessment Tool (MAT): Enabling Personalized Learning to Enhance Motivation,” in Intelligent Tutoring System, 2018, pp. 88–98. doi: 10.1007/978-3-319-91464-0_9.

C. Z. Guo and W. C. Wu, “The Influence of Major Satisfaction on Learning Engagement of Agriculture-Related Vocational Colleges in China: Taking Learning Motivation as a Mediating Variable,” Journal of Higher Education Theory and Practice, vol. 23, no. 3, Feb. 2023, doi: 10.33423/jhetp.v23i3.5859.

W. G. Cochran, Sampling Techniques, 3rd Edition, 3rd ed. John Wiley and Sons, 1977.

R. V. Krejcie and D. W. Morgan, “Determining Sample Size for Research Activities,” Educ Psychol Meas, vol. 30, no. 3, pp. 607–610, Sep. 1970, doi: 10.1177/001316447003000308.

R. C. MacCallum, K. F. Widaman, K. J. Preacher, and S. Hong, “Sample Size in Factor Analysis: The Role of Model Error,” Multivariate Behav Res, vol. 36, no. 4, pp. 611–637, Oct. 2001, doi: 10.1207/S15327906MBR3604_06.

O. Bolarinwa, “Principles and methods of validity and reliability testing of questionnaires used in social and health science researches,” Nigerian Postgraduate Medical Journal, vol. 22, no. 4, p. 195, 2015, doi: 10.4103/1117-1936.173959.

J. Smith-Merry, “Evidence-based policy, knowledge from experience and validity,” Evidence & Policy, vol. 16, no. 2, pp. 305–316, May 2020, doi: 10.1332/174426419X15700265131524.

D. George and P. Mallery, IBM SPSS Statistics 23 Step by Step. Routledge, 2016. doi: 10.4324/9781315545899.

K. K.-K. Wong, “Partial Least Squares Structural Equation Modeling (PLS-SEM) Techniques Using SmartPLS,” Marketing Bulletin, 2013.

D. Russo and K.-J. Stol, “PLS-SEM for Software Engineering Research,” ACM Comput Surv, vol. 54, no. 4, pp. 1–38, May 2022, doi: 10.1145/3447580.

J. F. Hair, J. J. Risher, M. Sarstedt, and C. M. Ringle, “When to use and how to report the results of PLS-SEM,” European Business Review, vol. 31, no. 1, pp. 2–24, Jan. 2019, doi: 10.1108/EBR-11-2018-0203.

Pavlou and Fygenson, “Understanding and Predicting Electronic Commerce Adoption: An Extension of the Theory of Planned Behavior,” MIS Quarterly, vol. 30, no. 1, p. 115, 2006, doi: 10.2307/25148720.

M. Sarstedt, C. M. Ringle, and J. F. Hair, “Partial Least Squares Structural Equation Modeling,” in Handbook of Market Research, Cham: Springer International Publishing, 2017, pp. 1–40. doi: 10.1007/978-3-319-05542-8_15-1.

W. W. Chin, The partial least squares approach to structural equation modeling. Modern methods for business research, 1998.

J. F. Hair, W. C. Black, B. J. Babin, and R. E. Anderson, Multivariate Data Analysis (7th ed.), 7th ed. Pearson Education, 2009.

J. Cohen, Statistical Power Analysis for the Behavioral Sciences. Routledge, 1998. doi: 10.4324/9780203771587.

B. G. Tabachnick and L. S. Fidell, Using Multivariate Statistics, 7th ed. Northridge USA: Pearson Education, Inc., 2019.

L. Hu and P. M. Bentler, “Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives,” Struct Equ Modeling, vol. 6, no. 1, pp. 1–55, Jan. 1999, doi: 10.1080/10705519909540118.

K. Schermelleh-Engel, H. Moosbrugger, and H. Müller, “Evaluating the Fit of Structural Equation Models: Tests of Significance and Descriptive Goodness-of-Fit Measures,” Methods of Psychological Research, vol. 8, no. 2, pp. 23–74, 2003.

P. M. Bentler and G. D. Bonett, “Significance Tests and Goodness of Fit in the Analysis of Covariance Structures,” Psychol Bull, vol. 88, no. 3, pp. 588–606, 1980.

G. James, D. Witten, T. Hastie, and R. Tibshirani, An Introduction to Statistical Learning: with Applications in R. Springer Publishing Company, Incorporated, 2014.

S. Menard, Applied Logistic Regression Analysis. 2455 Teller Road, Thousand Oaks California 91320 United States of America : SAGE Publications, Inc., 2002. doi: 10.4135/9781412983433.

M. J. Lachowicz, K. J. Preacher, and K. Kelley, “A novel measure of effect size for mediation analysis.,” Psychol Methods, vol. 23, no. 2, pp. 244–261, Jun. 2018, doi: 10.1037/met0000165.

S. Ogbeibu, C. J. C. Jabbour, J. Gaskin, A. Senadjki, and M. Hughes, “Leveraging STARA competencies and green creativity to boost green organisational innovative evidence: A praxis for sustainable development,” Bus Strategy Environ, vol. 30, no. 5, pp. 2421–2440, Jul. 2021, doi: 10.1002/bse.2754.