The Developmental Mathematical Literacy Instruments for Junior High School Teachers: Construction and Validation Study

Niluh Sulistyani; Albertus Hariwangsa Panuluh; Kintan Limianingsih; Margaretha Madha Melissa

doi:10.22437/edumatica.v16i2.53077

Authors

Niluh Sulistyani Sanata Dharma University https://orcid.org/0000-0001-5506-7198 (unauthenticated)
Albertus Hariwangsa Panuluh Sanata Dharma University https://orcid.org/0000-0003-1393-5713 (unauthenticated)
Kintan Limiansih Sanata Dharma University https://orcid.org/0009-0004-6608-4152 (unauthenticated)
Margaretha Madha Melissa International Education Development Program, Hiroshima University https://orcid.org/0009-0003-4260-4005 (unauthenticated)

DOI:

https://doi.org/10.22437/edumatica.v16i2.53077

Keywords:

desain-based research (DBR), junior high school, mathematical literacy, tessmer, validation study

Abstract

Teachers are required to facilitate students' mathematical literacy, but teachers' literacy skills are rarely considered due to the lack of instruments for assessing teachers' mathematical literacy. This study aims to develop a mathematical literacy instrument for teachers, focusing on the processes of instrument construction and validation. This Design-Based Research (DBR) adopts the Tessmer formative evaluation development model, consisting of preliminary, self-evaluation, expert reviews, one-to-one, small-group, and field tests. This study involved two experts as instrument validators (logical validity) and a total of 60 junior high school teachers in Yogyakarta Province to assess readability at the one-to-one stage and in small groups as subjects of field tests (empirical validation). The logical validation results were analyzed using the Aiken V index; readability test results were analyzed descriptively; and the results of empirical validation were analyzed using SPSS. The instrument construction process follows the principles of educational assessment development within the PISA framework and produces 25 literacy items, each involving 10 stimuli with varying contexts. The average V Aikens of 0.95 indicates a valid instrument, while the average readability test score of 4.14 indicates the instrument is in the good readability category. Based on the field test results, each item is valid because it has a Sig value ≤ 0.05. Overall, the question instrument is reliable, with a Cronbach's alpha (α) of 0.860 > 0.7. The instrument can measure mathematical literacy skills among teachers and assess all levels of thinking and question difficulty.

References

Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1), 131–142. https://doi.org/https://doi.org/10.1177/0013164485451012

Aisyah Aisyah, D. J. (2022). The description of Indonesian students' mathematics literacy in the last decade. International Journal of Trends in Mathematics Education Research. https://doi.org/https://doi.org/10.33122/ijtmer.v5i1.114.

Alfaruqi, A. Z., & Nurwahidayah. (2025). Jurnal Pendidikan IPS. 15(1), 11–19. https://doi.org/https://doi.org/10.37630/jpi.v15i1.2559

Amin, B., Baharuddin, B., Nur, F., Nursalam, N., & Angriani, A. D. (2025). Pengembangan Tes Asesmen Kompetensi Mahasiswa Untuk Mengukur Kemampuan Literasi Numerasi. AKSIOMA: Jurnal Program Studi Pendidikan Matematika, 14(1), 54. https://doi.org/10.24127/ajpm.v14i1.9494

Apriandi;, D., Murtafiah;, W., Suprapto;, E., Masfingatin;, T., & Lusiana, R. (2024). The development of problem-based calculus E-modules to improve students’ numeracy literacy skills. AIP Conference Proceedings, 3148(1). https://doi.org/https://doi.org/10.1063/5.0243257

Benson, J., & Clark, F. (1982). A Guide for Instrument Development and Validation. The American Journal of Occupational Therapy, 36(12), 789–800. https://doi.org/https://doi.org/10.5014/ajot.36.12.789

Blum, W., & Leiss, D. (2020a). The Role of Contexts in Mathematical Modeling – Towards a Conceptual Framework. ZDM Mathematics Education, 52, 149–163. https://doi.org/10.1007/s11858-019-01100-1

Chen, X., Zhou, J., Wang, J., Wang, D., Liu, J., Shi, D., Yang, D., & Pan, Q. (2022). Visualizing Status, Hotspots, and Future Trends in Mathematical Literacy Research via Knowledge Graph. Sustainability (Switzerland), 14(21). https://doi.org/10.3390/su142113842

Chiu, T. K. F., Chen, Y., Yau, K. W., Chai, C. Sing, Meng, H., King, I., Wong, S., & Yam, Y. (2024). Developing and validating measures for AI literacy tests: From self-reported to objective measures. Computers and Education: Artificial Intelligence, 7(August), 100282. https://doi.org/10.1016/j.caeai.2024.100282

Citrawan, I. W., et al. (2024). Special Education Teachers ’ Ability in Literacy and Numeracy Assessments Based on Local Wisdom. 8(1), 145–157. https://doi.org/https://doi.org/10.22437/jiituj.v8i1.32608

DeVellis, R. F. (2003). Scale Development: Theory and Applications (Second Edition). SAGE Publications.

Golla, E., & Reyes, V. (2022b). Developing Mathematical Literacy Test Items for Junior High School Teachers Based on the PISA Framework. International Journal of Evaluation and Research in Education, 11(4), 2045–2058. https://doi.org/10.11591/ijere.v11i4.21983

Helstad, K., Solbrekke, T. D., & Wittek, A. L. (2017). Exploring teaching academic literacy in mathematics in teacher education. Education Inquiry, 8(4), 318–336. https://doi.org/10.1080/20004508.2017.1389225

Hilmi, I. &

Kismiantini. (2024). Teacher Competencies, The Shortage of School Resources, and Mathematics Achievement Based on PISA 2018 Indonesia. Jurnal Pendidikan Matematika, 18(2). https://doi.org/https://doi.org/10.22342/jpm.v18i2.pp245-258

Holenstein, M., Bruckmaier, G., & Grob, A. (2021). Transfer effects of mathematical literacy: an integrative longitudinal study. European Journal of Psychology of Education, 36(3), 799–825. https://doi.org/10.1007/s10212-020-00491-4

Hwang, J., & Ham, Y. (2021). Relationship between mathematical literacy and opportunity to learn with different types of mathematical tasks. Journal on Mathematics Education, 12(2), 199–222. https://doi.org/10.22342/JME.12.2.13625.199-222

Kandil, S., & Işıksal-Bostan, M. (2019). Effect of inquiry-based instruction enriched with origami activities on achievement and self-efficacy in geometry. International Journal of Mathematical Education in Science and Technology, 50(4), 557–576. https://doi.org/10.1080/0020739X.2018.1527407

Kappassova, S., Abylkassymova, A., Bulut, U., Zykrina, S., Zhumagulova, Z., & Balta, N. (2025). Mathematical literacy and its influencing factors: A decade of research findings (2015-2024). Eurasia Journal of Mathematics, Science and Technology Education, 21(7). https://doi.org/10.29333/ejmste/16615

Kioupi, V., & Voulvoulis, N. (2019). Education for sustainable development: A systemic framework for connecting the SDGs to educational outcomes. Sustainability (Switzerland), 11(21). https://doi.org/10.3390/su11216104

Manfreda Kolar, V.M, Hodnik, T. (2021). Mathematical Literacy from the Perspective of Solving Contextual Problems. European Journal of Educational Research, 10(1), 467–483. https://doi.org/10.12973/eu-jer.10.1.467

McKay, V. (2018). Literacy, lifelong learning, and sustainable development. Australian Journal of Adult Learning, 58(3), 390–425. https://ala.asn.au/wp-content/uploads/2021/01/AJAL-58-3-Full.pdf

McKenney, S., & Reeves, T. C. (2018). About educational design research. Conducting Educational Design Research, 5–33. https://doi.org/10.4324/9781315105642-3

Merino-Soto, C. (2023). Coeficientes V de Aiken: diferencias en los juicios de validez de contenido. MHSalud: Revista en Ciencias del Movimiento Humano y Salud, 20(1), 1–10. https://doi.org/10.15359/mhs.20-1.3

Miles, M. B.; Huberman, M. S. J. (2014). Qualitative Data Analysis (3rd ed.). SAGE Publication.

Mislevy, R. J., Haertel, G., & Almond, R. G. (2021). Evidence-Centered Design for Educational Assessment: A Research Update. Educational Measurement: Issues and Practice, 40(1), 16–31. https://doi.org/10.1111/emip.12424

Muhaimin, L. H., Sholikhakh, R. A., Yulianti, S., Ardani, H., A., & Sahara, S. (2024). Unlocking the secrets of students’ mathematical literacy to solve mathematical problems: A systematic literature review. Eurasia Journal of Mathematics, Science and Technology Education, 20(4). https://doi.org/10.29333/ejmste/14404

Nadila, S. A., & Lestiana, H. T. (2024). Kemampuan Literasi Matematika Siswa Berdasarkan Gaya Belajar. Edumatica: Jurnal Pendidikan Matematika, 14(03). https://doi.org/https://doi.org/10.22437/edumatica.v14i3.38303

Nieveen, N., & Folmer, E. (2013). Formative Evaluation in Educational Design Research. In Educational Design Research – Part A: An Introduction (pp. 152–169). Netherlands Institute for Curriculum Development (SLO).

Nitko, A. J., & Brookhart, S. M. (2022). Educational Assessment of Students (8th ed.). Pearson Education.

OECD. (2016). Program for International Student Assessment (PISA) Results From PISA 2015. https://www.oecd.org/pisa/PISA-2015-Indonesia.pdf

OECD. (2019). PISA 2018 Results. https://www.oecd.org/pisa/Combined_Executive_Summaries_PISA_2018.pdf

OECD. (2021). PISA 2021 Mathematics Framework. https://www.oecd.org/pisa/pisaproducts/pisa-2021-mathematics-framework.pdf

OECD. (2023). PISA 2022 Assessment and Analytical Framework. In Paris: Journal of OECH Publishing.

PISA. (2023). PISA 2022 Results Factsheets Indonesia. The Language of Science Education, 1, 1–9. https://oecdch.art/a40de1dbaf/C108.

Pulsiri, N., Vatananan-Thesenvitz, R., Tantipisitkul, K., Aung, T. H., Schaller, A. A., Schaller, A. M., Methananthakul, K., & Shannon, R. (2019). Achieving sustainable development goals for people with disabilities through digital technologies. PICMET 2019 - Portland International Conference on Management of Engineering and Technology: Technology Management in the World of Intelligent Systems, Proceedings, (November 2020). https://doi.org/10.23919/PICMET.2019.8893725

Shrotryia, V. K., & Dhanda, U. (2019). Content Validity of Assessment Instrument for Employee Engagement. https://doi.org/10.1177/2158244018821751

Sinclair, N., Bartolini Bussi, M. G., de Villiers, M., Jones, K., Kortenkamp, U., Leung, A., & Owens, K. (2016). Recent research on geometry education: an ICME-13 survey team report. ZDM, 48(5), 691–719. https://doi.org/10.1007/s11858-016-0796-6

Stacey, K. (2011). The PISA view of mathematical literacy in Indonesia. Journal on Mathematics Education, 2(2), 95–126. https://doi.org/10.22342/jme.2.2.746.95-126

Stacey, K. & Turner, R. (2015). Assessing Mathematical Literacy: The PISA Experience (R. Stacey, Kaye & Turner, Ed.). Springer Cham. https://doi.org/https://doi.org/10.1007/978-3-319-10121-7

Suciati, S. Munadi, Sugiman, & Wiwin Dwi Ratna Febriyanti. (2020). Design and Validation of Mathematical Literacy Instruments for Assessment for Learning in Indonesia. European Journal of Educational Research, 9(2), 865–875. https://doi.org/https://doi.org/10.12973/eu-jer.9.2.865

Sugiyono. (2019). Metode penelitian pendidikan: Pendekatan kuantitatif, kualitatif, dan R&D. Alfabeta.

Tessmer, M. (2005). Planning and Conducting Formative Evaluations: Improving the Quality of Education and Training. Routledge.

Tout, D., Gal, I., & Schlöglmann, W. (2023). Mathematical Reasoning and Literacy: Revisiting PISA’s Conceptualization for the Next Decade. Studies in Educational Evaluation, 78, 102210. https://doi.org/10.1016/j.stueduc.2023.102210

Tout, D., Coben, D., Geiger, V., Ginsburg, L., Hoogland, K., Maguire, T., Thomson, S., & Turner, Ross. (2017). Review of the PIAAC Numeracy Assessment Framework: Final Report.

Treagust, D. F., & Chandrasegaran, A. L. (2020). Developing Diagnostic Assessment Items to Evaluate Students’ Understanding of Science and Mathematics Concepts. International Journal of Science and Mathematics Education, 18, 689–708. https://doi.org/10.1007/s10763-019-09971-0

Umbara, U., & Suryadi, D. (2019). Re-interpretation of mathematical literacy based on the teacher’s perspective. International Journal of Instruction, 12(4), 789–806. https://doi.org/10.29333/iji.2019.12450a

UNESCO. (2024). Global report on teachers: addressing teacher shortages and transforming the profession. In the Global report on teachers: addressing teacher shortages and transforming the profession. https://doi.org/10.54675/figu8035

Wijaya, A., Retnawati, H., & Nusantara, T. (2024). Enhancing Teachers’ Competence in Developing Mathematical Literacy Assessment Aligned with the PISA Framework. Eurasia Journal of Mathematics, Science and Technology Education, 20(2), em2295. https://doi.org/10.29333/ejmste/13670

Wijaya, A., Yogyakarta, U. N., Dewayani, S., Litara, Y., Effendi, A., Yogyakarta, U. N., & Gunawan, H. (n.d.). Framework Asesmen Kompetensi Minimum ( AKM ).

Wong, V., Yun, S., Ulang, N., & Husain, S. H. (2023). Measuring the Internal Consistency and Reliability of the Hierarchy of Controls in Preventing Infectious Diseases on Construction Sites : The Kuder-Richardson ( KR-20 ) and Cronbach’s Alpha. 1(1), 392–405. https://doi.org/https://doi.org/10.37934/araset.33.1.392405

Yusmar, F., & Fadilah, R. E. (2023). Analisis Rendahnya Literasi Sains Peserta Didik Indonesia: Hasil Pisa Dan Faktor Penyebab. LENSA (Lentera Sains): Jurnal Pendidikan IPA, 13(1), 11–19. https://doi.org/10.24929/lensa.v13i1.283