VALIDITY OF ENGAGEMENT INSTRUMENT DURING ONLINE LEARNING IN MATHEMATICS EDUCATION
DOI:
https://doi.org/10.22437/jiituj.v8i2.34453Keywords:
Confirmatory Factor Analysis, Engagement, Exploratory Factor Analysis, Rasch Analysis, Z GenerationAbstract
The current study aimed to assess the validity of the engagement instrument during online learning in mathematics education. This study used a survey research design as its approach. The current research participants were 203 Generation Z students in West Nusa Tenggara Barat, Indonesia. Convenience sampling techniques were used to assess who had completed the online survey. Three procedures were used to analyze the data in this research: exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and Rasch analysis. EFA revealed that the online engagement instrument had two sub-components: behavioral engagement and emotional engagement. At the same time, the CFA results showed that the model fit indices established the first- and second-order model's two-factor structure. Finally, the results showed that the online engagement instrument’s item and person reliability were good. The findings indicate a potential for enhancement even though the Rasch analysis largely supported the results of EFA and CFA. The current research’s novelty is that it provides a valid and reliable instrument to assess student`s engagement during online learning in a mathematical education context. Using the current instrument can ensure the accuracy, reliability, and credibility of research on student engagement during online learning in mathematics education.
Downloads
References
Adnan, M., & Anwar, K. (2020). Online learning amid the COVID-19 pandemic: Students' perspectives. Journal of Pedagogical Sociology and Psychology, 1(2), 45–51. https://doi.org/10.33902/jpsp.2020261309.
Aljaberi, M. A., et al. (2022). Rasch modeling and multilevel confirmatory factor analysis for the usability of the impact of event scale-revised (IES-R) during the COVID-19 pandemic. Healthcare, 10(10), 1858. https://doi.org/10.3390/healthcare10101858.
Asia, N., Kinda, J., & Edwards, J. (2024). Development of learning video media in biology subjects. Tekno - Pedagogi : Jurnal Teknologi Pendidikan, 14(2), 9-14. https://doi.org/10.22437/teknopedagogi.v14i2.37486.
Asrial, A., Syahrial, S., Kurniawan, D. A., Aldila, F. T., & Iqbal, M. (2023). Implementation of web-based character assessment on students' character outcomes: A review on perception and gender. JOTSE: Journal of Technology and Science Education, 13(1), 301-328. https://doi.org/10.3926/jotse.1564.
Asrial, A., Syahrial, S., Sabil, H., Ryan, J., Lamb, M., Cox, R., Emmitt, M., & Rahmi, R. (2024). Science teacher’s book: Analyzing elementary school students’ character values. Jurnal Ilmiah Ilmu Terapan Universitas Jambi, 8(1), 180–187. https://doi.org/10.22437/jiituj.v8i1.32766.
Asrial, A., Syahrial, S., Kurniawan, D. A., Putri, F. I., Perdana, R., Rahmi, R., Susbiyanto, S., & Aldila, F. T. (2024). E-Assessment for Character Evaluation in Elementary Schools. Qubahan Academic Journal, 4(3), 806-822. https://doi.org/10.48161/qaj.v4n3a595.
Azis, I., & Clefoto, M. (2024). Improving learning discipline: The effect of self-management ability on students in mathematics subjects. Interval: Indonesian Journal of Mathematical Education, 2(1), 8-14. https://doi.org/10.37251/ijome.v2i1.982.
Baah, R., Konovalov, O., & Tenzin, S. (2024). Effectiveness of e-assessment in science learning: Improving the quality and efficiency of assessment in the digital era. Integrated Science Education Journal, 5(2), 74-81. https://doi.org/10.37251/isej.v5i2.960.
Bagdi, H., Bulsara, H. P., Sankar, D., & Sharma, L. (2023). The transition from traditional to digital: factors that propel Generation Z’s adoption of online learning. International Journal of Educational Management, 37(3), 695-717. https://doi.org/10.1108/IJEM-01-2023-0003.
Bagriacik Yilmaz, Y., & Banyard, P. (2020). Engagement in distance education settings: A trend analysis. Turkish Online Journal of Distance Education. 101–120. https://doi.org/10.17718/tojde.690362.
Bravini, E., Giordano, A., Sartorio, F., Ferriero, G., & Vercelli, S. (2016). Rasch analysis of the Italian lower extremity functional scale: Insights on dimensionality and suggestions for an improved 15-item version. Clinical Rehabilitation, 31(4), 532–543. https://doi.org/10.1177/0269215516647180.
Boone, W., Staver, J., & Yale, M.S. (2014). Rasch analysis in the human sciences. Springer.
Buelow, J. R., Barry, T. A., & Rich, L. E. (2018). Supporting learning engagement with online students. Online Learning, 22(4), 313–340. https://doi.org/10.24059/olj.v22i4.1384.
Cacheiro-Gonzalez, M. L., Medina-Rivilla, A., Dominguez-Garrido, M. C., & Medina-Dominguez, M. (2019). The learning platform in distance higher education: Students' perceptions. Turkish Online Journal of Distance Education, 20(1), 71–95. https://doi.org/10.17718/tojde.522387.
Cadiz, M. C. D., Manuel, L. a. F., Reyes, M. M., Natividad, L. R., & Ibarra, F. P. (2024). Technology Integration in Philippine higher education: A content-based bibliometric analysis. Jurnal Ilmiah Ilmu Terapan Universitas Jambi, 8(1), 35–47. https://doi.org/10.22437/jiituj.v8i1.31807.
Callahan, C. M., Azano, A. P., Park, S., Brodersen, A. V., Caughey, M., Bass, E. L., & Amspaugh, C. M. (2020). Validation of instruments for measuring affective outcomes in gifted education. Journal of Advanced Academics, 31(4), 470–505. https://doi.org/10.1177/1932202x20929963.
Csobanka, Z. E. (2016). The Z generation. Acta Technologica Dubnicae, 6(2), 63–76. https://doi.org/10.1515/atd-2016-0012.
Dogan, U. (2014). Validity and reliability of student engagement scale. Bartin Universitesi Egitim Fakultesi Dergisi, 3(2), 390. https://doi.org/10.14686/BUEFAD.201428190.
Dumford, D., & Miller, A. L. (2018). Online learning in higher education: Exploring advantages and disadvantages for engagement. Journal of Computing in Higher Education, 30(3), 452–465. https://doi.org/10.1007/s12528-018-9179-z.
Farrell, O., & Brunton, J. (2020). A balancing act: A window into online student engagement experiences. International Journal of Educational Technology in Higher Education, 17(1), 1–19. https://doi.org/10.1186/s41239-020-00199-x.
Ferri, F., Grifoni, P., & Guzzo, T. (2020). Online learning and emergency remote teaching: Opportunities and challenges in emergency situations. Societies, 10(4), 1–18. https://doi.org/10.3390/soc10040086.
Fitriana, H., & Waswa, A. N. (2024). The influence of a realistic mathematics education approach on students’ mathematical problem solving ability. Interval: Indonesian Journal of Mathematical Education, 2(1), 29-35. https://doi.org/10.37251/ijome.v2i1.979.
Goretzko, D., Pham, T. T., & Buhner, M. (2019). Exploratory factor analysis: Current use, methodological developments and recommendations for good practice. Current Psychology, 40(7), 3510–3521. https://doi.org/10.1007/s12144-019-00300-2.
Habibi, M. W., Jiyane, L., & Ozsen, Z. (2024). Learning revolution: The positive impact of computer simulations on science achievement in madrasah ibtidaiyah. Journal of Educational Technology and Learning Creativity, 2(1), 13-19. https://doi.org/10.37251/jetlc.v2i1.976.
Harrington, D. (2009). Confirmatory factor analysis. Oxford, UK: Oxford University Press.
Hidayat, R., Liu, S., Mohd Saad, M. R., & Hermandra. (2023). The role of mastery goal on life satisfaction using perma as a mediator for college students. Emerging Science Journal, 7, 238–252. https://doi.org/10.28991/esj-2023-sied2-018.
Hidayat, R., Wan Idris, W. I., Qudratuddarsi, H., & Abdul Rahman, M. N. (2021). Validation of the mathematical modeling attitude scale for Malaysian mathematics teachers. Eurasia Journal of Mathematics, Science and Technology Education, 17(12), 1–16. https://doi.org/10.29333/ejmste/11375.
Hidayat, R., Zainuddin, Z., & Mazlan, N. H. (2024). The relationship between technological pedagogical content knowledge and belief among preservice mathematics teachers. Acta Psychologica, 249, 104432. https://doi.org/10.1016/j.actpsy.2024.104432.
Hoi, V. N., & Le Hang, H. (2021). The structure of student engagement in onlne learning: A bi-factor exploratory structural equation modelling approach. Journal of Computer Assisted Learning, 37(4), 1141–1153. https://doi.org/10.1111/jcal.12551.
Howard, M. C. (2015). A review of exploratory factor analysis decisions and overview of current practices: What we are doing and how can we improve? International Journal of Human-Computer Interaction, 32(1), 51–62. https://doi.org/10.1080/10447318.2015.1087664.
Inda-Caro, M., Maulana, R., Fernández-García, C.-M., Peña-Calvo, J.-V., Rodríguez-Menéndez, M. del, & Helms-Lorenz, M. (2018). Validating a model of effective teaching behavior and student engagement: Perspectives from Spanish students. Learning Environments Research, 22(2), 229–251. https://doi.org/10.1007/s10984-018-9275-z.
Irvine, J. (2020). Positively Influencing Student Engagement and Attitude in Mathematics through an Instructional Intervention Using Reform Mathematics Principles. Journal of Education and Learning, 9(2), 48-75. https://doi.org/10.5539/jel.v9n2p48.
Jabar, J. M., Hidayat, R., Samat, N. A., Rohizan, M. F. H., Rosdin, N. ‘A., Salim, N., & Norazhar, S. A. (2022). Augmented reality learning in mathematics education: A systematic literature review. Journal of Higher Education Theory and Practice, 22(15). https://doi.org/10.33423/jhetp.v22i15.5570.
Jackson, D. L., Gillaspy, J. A., & Purc-Stephenson, R. (2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14(1), 6–23. https://doi.org/10.1037/a0014694.
Joshi, D. R., Adhikari, K. P., Khanal, B., Khadka, J., & Belbase, S. (2022). Behavioral, cognitive, emotional and social engagement in mathematics learning during COVID-19 pandemic. PloS one, 17(11), e0278052. https://doi.org/10.1371/journal.pone.0278052.
Kahn, P., Everington, L., Kelm, K., Reid, I., & Watkins, F. (2016). Understanding student engagement in online learning environments: The role of reflexivity. Educational Technology Research and Development, 65(1), 203–218. https://doi.org/10.1007/s11423-016-9484-z.
Kannadass, P., Hidayat, R., Siregar, P. S., & Husain, A. P. (2023). Relationship between computational and critical thinking towards modelling competency among pre-service mathematics teachers. TEM Journal, 1370–1382. Portico. https://doi.org/10.18421/tem123-17.
Khalid, F. (2019). Students' identities and its relationships with their engagement in an online learning community. International Journal of Emerging Technologies in Learning (iJET), 14(05), 4–18. https://doi.org/10.3991/ijet.v14i05.8196.
Knekta, E., Runyon, C., & Eddy, S. (2019). One size doesn’t fit all: Using factor analysis to gather validity evidence when using surveys in your research. CBE—Life Sciences Education, 18(1), rm1. https://doi.org/10.1187/cbe.18-04-0064.
Larsen, R., & Warne, R. T. (2010). Estimating confidence intervals for eigenvalues in exploratory factor analysis. Behavior Research Methods, 42(3), 871–876. https://doi.org/10.3758/brm.42.3.871.
Lee, J., Song, H.-D., & Hong, A. (2019). Exploring factors, and indicators for measuring students’ sustainable engagement in e-learning. Sustainability, 11(4), 1–12. https://doi.org/10.3390/su11040985.
Li, S. (2022). Measuring cognitive engagement: An overview of measurement instruments and techniques. International Journal of Psychology and Educational Studies, 8(3), 63–76. https://doi.org/10.52380/ijpes.2021.8.3.239.
Mardiati, D. C., Alorgbey, B., & Zarogi, A. B. (2024). The relationship between educational level and the role of parents with learning achievement in mathematics. Interval: Indonesian Journal of Mathematical Education, 2(1), 22-28. https://doi.org/10.37251/ijome.v2i1.983.
Marshall, L., & Wolanskyj-Spinner, A. (2020). Covid-19: Challenges and opportunities for educators and generation Z learners. Mayo Clinic Proceedings, 95(6), 1135–1137. https://doi.org/10.1016/j.mayocp.2020.04.015.
Marta, P., Khairinal, K., & Murbojono, R. (2023). Innovative approaches: Unveiling integrated social sciences learning at SMP Negeri 7 Muaro Jambi. Tekno - Pedagogi : Jurnal Teknologi Pendidikan, 13(1), 41-49. https://doi.org/10.22437/teknopedagogi.v13i1.32541.
Martin, F., & Bolliger, D. U. (2018). Engagement matters: Student perceptions on the importance of engagement strategies in the Online Learning Environment. Online Learning, 22(1), 205–222. https://doi.org/10.24059/olj.v22i1.1092.
Mayer, R. E. (2018). Thirty years of research on online learning. Applied Cognitive Psychology, 33(2), 152–159. https://doi.org/10.1002/acp.3482.
McBrien, J. L., Cheng, R., & Jones, P. (2009). Virtual spaces: Employing a synchronous online classroom to facilitate student engagement in online learning. The International Review of Research in Open and Distributed Learning, 10(3), 1–16. https://doi.org/10.19173/irrodl.v10i3.605.
Mindrila, D., An, Y., Lawrence, M., & Elemen, J. (2017). Exploratory factor analysis: Applications in school improvement research. NovaScience Publisher.
Mizian, Z. (2023). Development of an electronic brick bonding module using the autocad program for students. Tekno-Pedagogi: Jurnal Teknologi Pendidikan, 13(2), 35-43. https://doi.org/10.22437/teknopedagogi.v13i2.32531.
Mosca, J., & Curtis, K. P. (2019). New approaches to learning for generation Z. Journal of Business Diversity, 19(3), 66–74. https://doi.org/10.33423/jbd.v19i3.2214.
Mohd Saad, M. R., Mamat, S., Hidayat, R., & Othman, A. J. (2023). Integrating technology-based instruction and mathematical modelling for STEAM-based language learning: a sociocultural and self-determination theory perspective. International Journal of Interactive Mobile Technologies (IJIM), 17(14), 55–80. https://doi.org/10.3991/ijim.v17i14.39477.
Moon, J., & Ke, F. (2020). In-game actions to promote game-based math learning engagement. Journal of Educational Computing Research, 58(4), 863-885. https://doi.org/10.1177/0735633119878611.
Mukhtar, K., Javed, K., Arooj, M., & Sethi, A. (2020). Advantages, limitations and recommendations for online learning during COVID-19 pandemic era. Pakistan Journal of Medical Sciences, 36(COVID19-S4). https://doi.org/10.12669/pjms.36.covid19-s4.2785.
Nahar, L. (2023). The effects of standardized tests on incorporating 21st century skills in science classrooms. Integrated Science Education Journal, 4(2), 36-42. https://doi.org/10.37251/isej.v4i2.324.
Nguyen, T. D., Cannata, M., & Miller, J. (2016). Understanding student behavioral engagement: Importance of student interaction with peers and teachers. The Journal of Educational Research, 111(2), 163–174. https://doi.org/10.1080/00220671.2016.1220359.
Nordholm, D., Arnqvist, A., & Nihlfors, E. (2020). Principals’ identity – developing and validating an analysis instrument in the Swedish context. International Journal of Leadership in Education, 1–16. https://doi.org/10.1080/13603124.2020.1804621.
Ouadoud, M., Rida, N., & Chafiq, T. (2021). Overview of e-learning platforms for teaching and learning. International Journal of Recent Contributions from Engineering, Science & IT (iJES), 9(1), 50–70. https://doi.org/10.3991/ijes.v9i1.21111.
Peng, W. (2017). Research on model of student engagement in online learning. Eurasia Journal of Mathematics, Science and Technology Education, 13(7), 3127–3142. https://doi.org/10.12973/eurasia.2017.00723a.
Perdana, F. A., Zakariah, S. H., & Alasmari, T. (2023). Development of learning media in the form of electronic books with dynamic electricity teaching materials. Journal of Educational Technology and Learning Creativity, 1(1), 1-6. https://doi.org/10.37251/jetlc.v1i1.619.
Qudratuddarsi, H., Hidayat, R., Shah, R. L., Nasir, N., Imami, M. K. W., & bin Nor, R. (2022). Rasch validation of instrument measuring Gen-Z science, technology, engineering, and mathematics (STEM) application in teaching during the pandemic. International Journal of Learning, Teaching and Educational Research, 21(6), 104–121. https://doi.org/10.26803/ijlter.21.6.7.
Qudratuddarsi, H., Sathasivam, R. V., & Hutkemri, H. (2019). Difficulties and correlation between phenomenon and reasoning tier of multiple-choice questions: A survey study. Indonesian Research Journal in Education| IRJE|, 249-264. https://doi.org/10.22437/irje.v3i2.6970.
Rahayu, W., Putra, M. D., Iriyadi, D., Rahmawati, Y., & Koul, R. B. (2020). A Rasch and factor analysis of an Indonesian version of the student perception of Opportunity Competence Development (SPOCD) questionnaire. Cogent Education, 7(1), 1–18. https://doi.org/10.1080/2331186X.2020.1721633.
Redmond, P., Heffernan, A., Abawi, L., Brown, A., & Henderson, R. (2018). An online engagement framework for higher education. Online Learning, 22(1), 183–204. https://doi.org/10.24059/olj.v22i1.1175.
Richardson, C., & Newby, T. (2006). The role of students' cognitive engagement in online learning. American Journal of Distance Education, 20(1), 23–37. https://doi.org/10.1207/s15389286ajde2001_3.
Roblek, V., Mesko, M., Dimovski, V., & Peterlin, J. (2019). Smart technologies as social innovation and complex social issues of the Z generation. Kybernetes, 48(1), 91–107. https://doi.org/10.1108/k-09-2017-0356.
Roche, A., Gervasoni, A., & Kalogeropoulos, P. (2023). Factors that promote interest and engagement in learning mathematics for low-achieving primary students across three learning settings. Mathematics Education Research Journal, 35(3), 525-556. https://doi.org/10.1007/s13394-021-00402-w.
Saputro, H. D., Rustaminezhad, M. A., Amosa, A. A., & Jamebozorg, Z. (2023). Development of e-learning media using adobe flash program in a contextual learning model to improve students’ learning outcomes in junior high school geographical research steps materials. Journal of Educational Technology and Learning Creativity, 1(1), 25-32. https://doi.org/10.37251/jetlc.v1i1.621.
Sarsar, F., & Kisla, T. (2016). Emotional presence in online learning scale: A scale development study. Turkish Online Journal of Distance Education, 50–61. https://doi.org/10.17718/tojde.87040.
Schreiber, J. B. (2021). Issues and recommendations for exploratory factor analysis and principal component analysis. Research in Social and Administrative Pharmacy, 17(5), 1004–1011. https://doi.org/10.1016/j.sapharm.2020.07.027.
Shek, D. T. L., & Yu, L. (2014). Confirmatory factor analysis using Amos: A demonstration. International Journal on Disability and Human Development, 13(2), 191–204. https://doi.org/10.1515/ijdhd-2014-0305.
Shrestha, N. (2021). Factor Analysis as a Tool for Survey Analysis. American Journal of Applied Mathematics and Statistics, 9(1), 4–11. https://doi.org/10.12691/ajams-9-1-2.
Siew, N. M., & Saidi, S. S. (2019). Reliability and validity analysis of statistical reasoning test survey instrument using the rasch measurement model. International Electronic Journal of Mathematics Education, 14(3). https://doi.org/10.29333/iejme/5755.
Singh, V., & Thurman, A. (2019). How many ways can we define online learning? A systematic literature review of definitions of online learning (1988-2018). American Journal of Distance Education, 33(4), 289–306. https://doi.org/10.1080/08923647.2019.1663082.
Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009). A motivational perspective on engagement and disaffection: Conceptualization and assessment of children's behavioral and emotional participation in academic activities in the classroom. Educational and psychological measurement, 69(3), 493-525. https://doi.org/10.1177/0013164408323233.
Stein, H., Gurevich, I., & Gorev, D. (2020). Integration of technology by novice mathematics teachers–what facilitates such integration and what makes it difficult? Education and Information Technologies, 25(1), 141-161. https://doi.org/10.1007/s10639-019-09950-y.
Story, D. A., & Tait, A. R. (2019). Survey research. Anesthesiology, 130(2), 192-202. https://doi.org/10.1097/ALN.0000000000002436.
Swami, V., & Barron, D. (2019). Translation and validation of body image instruments: Challenges, good practice guidelines, and reporting recommendations for test adaptation. Body Image, 31, 204–220. https://doi.org/10.1016/j.bodyim.2018.08.014.
Testa, I., et al. (2021). Development and validation of an instrument to measure students’ engagement and participation in science activities through factor analysis and Rasch analysis. International Journal of Science Education, 44(1), 18–47. https://doi.org/10.1080/09500693.2021.2010286.
Triyasmina, T., Rusdi, M., Asyhar, R., Dachia, H. A., & Rukondo, N. (2022). Chemistry learning revolution: Interactive multimedia e-learning with a problem based learning approach. Tekno-Pedagogi: Jurnal Teknologi Pendidikan, 12(2), 1-9. https://doi.org/10.22437/teknopedagogi.v12i2.32521.
Wara, E., Aloka, P. J., & Odongo, B. C. (2018). Relationship between emotional engagement and academic achievement among Kenyan secondary school students. Academic Journal of Interdisciplinary Studies, 7(1), 107–118. https://doi.org/10.2478/ajis-2018-0011.
Watkins, W. (2018). Exploratory factor analysis: A guide to best practice. Journal of Black Psychology, 44(3), 219–246. https://doi.org/10.1177/0095798418771807.
Watkins, W. (2021). A step-by-step guide to exploratory factor analysis with SPSS. Routledge.
Watson, J. C. (2017). Establishing evidence for internal structure using exploratory factor analysis. Measurement and Evaluation in Counseling and Development, 50(4), 232–238. https://doi.org/10.1080/07481756.2017.1336931.
Wei, H.-C., & Chou, C. (2020). Online learning performance and satisfaction: Do perceptions and readiness matter? Distance Education, 41(1), 48–69. https://doi.org/10.1080/01587919.2020.1724768.
Wu, X. V., Chan, Y. S., Tan, K. H., & Wang, W. (2018). A systematic review of online learning programs for nurse preceptors. Nurse Education Today, 60, 11–22. https://doi.org/10.1016/j.nedt.2017.09.010.
Xu, X. (2024). Student engagement and academic achievement in learning Calculus. (2024). International Journal of New Developments in Education, 6(2). https://doi.org/10.25236/ijnde.2024.060204.
Yohanie, D. D., Botchway, G. A., Nkhwalume, A. A., & Arrazaki, M. (2023). Thinking process of mathematics education students in problem solving proof. Interval: Indonesian Journal of Mathematical Education, 1(1), 24-29. https://doi.org/10.37251/ijome.v1i1.611.
Zakiyah, Z., Boonma , K., & Collado, R. (2024). Physics learning innovation: Song and animation-based media as a learning solution for mirrors and lenses for junior high school students. Journal of Educational Technology and Learning Creativity, 2(2), 54-62. https://doi.org/10.37251/jetlc.v2i2.1062.
Zepke, N., & Leach, L. (2010). Improving student engagement: Ten proposals for action. Active Learning in Higher Education, 11(3), 167–177. https://doi.org/10.1177/1469787410379680.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Riyan Hidayat, Muh Khairul Wajedi Imami, Sibo Liu, Hilman Qudratuddarsi, Mohd Rashid Mohd Saad
This work is licensed under a Creative Commons Attribution 4.0 International License.