Calibration of Students' Mathematical Literacy Instrumens Using IRT and Its Applications for Score

Authors

  • Hari Purnomo Susanto Sekolah Tinggi Keguruan dan Ilmu Pendidikan PRGI Pacitan
  • Heri Retnawati Universitas Negeri Yogyakarta

DOI:

https://doi.org/10.22437/edumatica.v13i01.23135

Keywords:

algebra content, CFA, IRT calibration, IRT scoring, mathematical literacy

Abstract

Mathematical literacy is one of the components that is of concern to the Minimum Competency Assessment (AKM). This policy is an effort by the Ministry of Education and Culture to overcome the low literacy skills of students, and the faktors that cause it. Multistage Adaptive Test (MSAT) is used as an assessment method in AKM. The MSAT was developed with the IRT concept. The purpose of this article is to apply Item Respons Theory (IRT) to calibrate mathematical literacy instrumens and utilize the calibration result item parameters to estimate scores for mathematical literacy skills. Many items in the instrumen used are 15 multiple choice items. Respondents were used as many as 66 grade 8 junior high school students. After going through the construct validation process using Confirmatory Factor Analysis (CFA), 2 items did not meet, because they had a low loading factor, and 13 items were used for the calibration process. The grain calibration and scoring processes were carried out using the R package Mirt program. The calibration results show that the instrument matches the Rasch model. All assumptions of unidimensional IRT are proven to be met, so there is no violation in estimating item parameters. The item parameter is the level of difficulty where there is one item that does not fit, namely item 3. The intersection of the information function and the standard error shows that the instrument will provide accurate information if it is used by students with abilities of -2.965 to 1.085. The resulting item parameters can be used to estimate the algebra content math literacy score.

Downloads

Download data is not yet available.

References

Brown, A. (2018). Item Response Theory Approaches to Test Scoring and Evaluating the Score Accuracy. In The Wiley Handbook of Psychometric Testing (pp. 607–638). Wiley. https://doi.org/10.1002/9781118489772.ch20

Brown, A., & Croudace, T. (2014). Scoring and Estimating Score Precision Using Multidimensional IRT Models. In S. P. Reise & D. A. Revicki (Eds.), Handbook of Item Response Theory Modeling (pp. 325–351). Routledge. https://doi.org/10.4324/9781315736013-26

Chalmers, R. P. (2012). mirt : A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6). https://doi.org/10.18637/jss.v048.i06

Dewanti, S. S., Hadi, S., & Nu’man, M. (2021). The Application of Item Response Theory in Analysis of Characteristics of Mathematical Literacy Test Items. İlköğretim Online, 20(1). https://doi.org/10.17051/ilkonline.2021.01.119

Edelen, M. O., & Reeve, B. B. (2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of Life Research, 16(S1), 5–18. https://doi.org/10.1007/s11136-007-9198-0

Edwards, M. C., Houts, C. R., & Cai, L. (2018). A diagnostic procedure to detect departures from local independence in item response theory models. Psychological Methods, 23(1), 138–149. https://doi.org/10.1037/met0000121

Finch, H. (2014). Measurement Invariance. In Encyclopedia of Quality of Life and Well-Being Research (pp. 3909–3912). Springer Netherlands. https://doi.org/10.1007/978-94-007-0753-5_1759

Hair, J. F. J., Black, W. C., Babin, B. J., & Anderson, R. E. (2018). Multivariate Data Analysis, Multivariate Data Analysis. In Multivariate Data Analysis.

Hambleton, R., Swaminathan, H., & Rogers, H. J. (1991). fundamental of item response theory. SAGE.

Hasanah, M., & Hakim, D. L. (2022). Kemampuan Literasi Matematis Pada Soal Matematika PISA Konten Quantity dan Konten Change and Relationship. JURING (Journal for Research in Mathematics Learning), 5(2), 157. https://doi.org/10.24014/juring.v5i2.13785

Hertiandito, L. T. (2016). Kemampuan literasi matematika siswa SMP pada pembelajaran Knisley dengan tinjauan gaya belajar. PRISMA, Prosiding Seminar Nasional Matematika, 2011.

Kemendikbud. (2019). Kajian Akademik dan Rekomendasi Reformasi Sistem Asesmen Nasional. Kementerian Pendidikan Dan Kebudayaan.

Kemendikbud. (2021). Asesmen Nasional: Lembar Tanya Jawab. Kementerian Pendidikan Dan Kebudayaan.

Larasaty, B. M., Mustiani, & Pratini, H. S. (2018). Peningkatan Kemampuan Literasi Matematika Siswa Kelas VIII SMP Bopkri 3 Yogyakarta Melalui Pendekatan PMRI Berbasis PISA Pada Materi Pokok SPLDV. Prosiding Seminar Nasional Etnomatnesia.

Magis, D., & Barrada, J. R. (2017). Computerized Adaptive Testing with R : Recent Updates of the Package catR. Journal of Statistical Software, 76(Code Snippet 1). https://doi.org/10.18637/jss.v076.c01

Magis, D., Yan, D., & von Davier, A. A. (2017). Computerized Adaptive and Multistage Testing with R. Springer International Publishing. https://doi.org/10.1007/978-3-319-69218-0

Mansur, N. (2018). Melatih Literasi Matematika Siswa dengan Soal PISA. Prisma, 1.

Masjaya, & Wardono. (2018). Pentingnya Kemampuan Literasi Matematika untuk Menumbuhkan Kemampuan Koneksi Matematika dalam Meningatkan SDM. PRISMA, Prosiding Seminar Nasional Matematika, 1.

Maydeu-Olivares, A. (2013). Goodness-of-Fit Assessment of Item Response Theory Models. Measurement: Interdisciplinary Research & Perspective, 11(3), 71–101. https://doi.org/10.1080/15366367.2013.831680

Maydeu-Olivares, A. (2014). Evaluating the Fit of IRT Models. In S. P. Reise & D. A. Revicki (Eds.), Handbook of Item Response Theory Modeling (pp. 129–145). Routledge. https://doi.org/10.4324/9781315736013-15

Nguyen, T. H., Han, H.-R., Kim, M. T., & Chan, K. S. (2014). An Introduction to Item Response Theory for Patient-Reported Outcome Measurement. The Patient - Patient-Centered Outcomes Research, 7(1), 23–35. https://doi.org/10.1007/s40271-013-0041-0

OECD. (2013). PISA 2012 Assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy.

Paek, I., & Cole, K. (2019). Using R for Item Response Theory Model Applications. Routledge. https://doi.org/10.4324/9781351008167

R Core Team. (2022). R: A Language and enviroment for statiscital computing. R Foundation for statistical Computing,. https: // www.R-project.org/.

Retnawati, H. (2014). Teori Respons Butir dan Penerapannya.

Retnawati, H. (2016). validitas dan reliabilitas dan karakteristik butir (1st ed.). Parama Publising.

Rotou, O., Patsula, L., Steffen, M., & Rizavi, S. (2007). COMPARISON OF MULTISTAGE TESTS WITH COMPUTERIZED ADAPTIVE AND PAPER-AND-PENCIL TESTS. ETS Research Report Series, 2007(1), i–27. https://doi.org/10.1002/j.2333-8504.2007.tb02046.x

Rupp, A. A., & Zumbo, B. D. (2006). Understanding Parameter Invariance in Unidimensional IRT Models. Educational and Psychological Measurement, 66(1), 63–84. https://doi.org/10.1177/0013164404273942

Susanto, H. P. (2022). package hpsCFA untuk Analisis Validitas Kontruk menggunakan CFA. https://github.com/SusantoHP/hpsCFA

Tabachnick, L. S., & Fidell, B. G. (2014). Using multivariate statistics: Pearson New International Edition.

Toland, M. D. (2014). Practical Guide to Conducting an Item Response Theory Analysis. The Journal of Early Adolescence, 34(1), 120–151. https://doi.org/10.1177/0272431613511332

Vale, P., Murray, S., & Brown, B. (2013). Mathematical literacy examination items and student errors: An analysis of English Second Language students’ responses. Per Linguam, 28(2). https://doi.org/10.5785/28-2-531

Wulandari, F., Hadi, S., & Haryanto, H. (2020). Computer-based Adaptive Test Development Using Fuzzy Item Response Theory to Estimate Student Ability. Computer Science and Information Technology, 8(3), 66–73. https://doi.org/10.13189/csit.2020.080302

Yuberta, K. R., Nari, N., & Gustia, E. (2020). KEMAMPUAN LITERASI MATEMATIS SISWA DENGAN MENERAPKAN MODEL PEMBELAJARAN CREATIVE PROBLEM SOLVING (CPS). Jurnal Saintika Unpam : Jurnal Sains Dan Matematika Unpam, 3(1), 68. https://doi.org/10.32493/jsmu.v3i1.6269

Downloads

Published

2023-04-30

How to Cite

Susanto, H. P., & Retnawati, H. (2023). Calibration of Students’ Mathematical Literacy Instrumens Using IRT and Its Applications for Score . Edumatica : Jurnal Pendidikan Matematika, 13(01), 23-36. https://doi.org/10.22437/edumatica.v13i01.23135

Issue

Vol. 13 No. 01 (2023): Edumatica: Jurnal Pendidikan Matematika

Section

Articles

License

Copyright (c) 2023 Hari Susanto, Heri Retnawati

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.