Development of a Competency-Based Evaluation Instrument for Assessing Employability Skills in Good Manufacturing Practices Training
DOI:
https://doi.org/10.22437/5584t042Keywords:
Competency-Based Evaluation, DDR, Employability Skills, GMP Training, Instrument DevelopmentAbstract
This study aims to develop a competency-based evaluation instrument for assessing employability skills in GMP training. The study was motivated by the limitation of conventional GMP evaluation instruments that mainly measure technical knowledge and procedural compliance, while providing limited evidence of participants’ ability to apply critical thinking, professionalism, communication, and teamwork in workplace-based situations. This study employed a DDR approach using the Needham model, which consisted of needs analysis, design and development, and evaluation. The instrument was developed by aligning GMP competency standards, employability skill indicators, cognitive levels, and item formats. The final prototype consisted of 20 items, including 10 multiple-choice items and 10 true-false items. Data were collected through expert validation and product testing with GMP training participants at the AGAVI Institute. The data were analyzed using the Gregory Index, KR-20 reliability test, item difficulty analysis, discrimination power analysis, and distractor effectiveness analysis. The results showed that the material validation obtained a Gregory index of 0.73, categorized as high validity, while the evaluation design validation obtained a Gregory index of 0.80, categorized as very high validity. The reliability coefficient was 0.63, indicating acceptable internal consistency for preliminary use. Item analysis showed that most items were easy, with limited discrimination power and distractor effectiveness. Therefore, the instrument is feasible as a validated prototype, although broader testing is needed to strengthen its psychometric stability.
References
Arifin, Z. (2017). Evaluasi pembelajaran: Prinsip, teknik, dan prosedur. Remaja Rosdakarya.
Arikunto, S. (2021). Dasar-dasar evaluasi pendidikan (3rd ed.). Bumi Aksara.
Asri, S. A., & Ishak, S. (2025). Analisis kualitas butir soal ulangan akhir semester genap matematika kelas VIII SMPN Sinjai. Pedagogy: Jurnal Pendidikan Matematika, 10(1), 120–133. https://doi.org/10.30605/pedagogy.v10i1.5672
Ajjawi, R., Tai, J., Dollinger, M., Dawson, P., Boud, D., & Bearman, M. (2023). From authentic assessment to authenticity in assessment: Broadening perspectives. Assessment & Evaluation in Higher Education, 49, 499–510. https://doi.org/10.1080/02602938.2023.2271193
Alt, D., Naamati-Schneider, L., & Weishut, D. (2023). Competency-based learning and formative assessment feedback as precursors of college students’ soft skills acquisition. Studies in Higher Education, 48, 1901–1917. https://doi.org/10.1080/03075079.2023.2217203
Augustin, F., Narad, R., Moran, B. M. B., & Bahr, K. O. (2023). The employability strengths American millennials contribute to the health administration workforce: A workplace readiness study at two California public universities. Journal of Teaching and Learning for Graduate Employability. https://doi.org/10.21153/jtlge2023vol14no1art1701
Bertaglia, E. B., Pereira, I., Rios, Á. C. L., Silva, E. M. P., & Dias, F. B. (2025). Self-assessment of competencies in project-based learning. Constructionism Conference Proceedings. https://doi.org/10.21240/constr/2025/14.x
Davies, A. C., Davies, A., Abdulhussein, H., Hooley, F., Eleftheriou, I., Hassan, L., Bromiley, P., Couch, P. A., Wasiuk, C., & Brass, A. (2022). Educating the healthcare workforce to support digital transformation. Studies in Health Technology and Informatics, 290, 934–936. https://doi.org/10.3233/SHTI220217
Davis, K. A., Grote, D. M., Mahmoudi, H., Perry, L. A., Ghaffarzadegan, N., Grohs, J., Hosseinichimeh, N., Knight, D., & Triantis, K. (2023). Comparing self-report assessments and scenario-based assessments of systems thinking competence. Journal of Science Education and Technology, 32, 793–813. https://doi.org/10.1007/s10956-023-10027-2
Enri, S., Yulistiani, R., & Fauziah, N. (2024). Penguatan good manufacturing practices (GMP) pada CV. Pawon Ibun dalam rangka mempersiapkan sertifikasi pangan. Journal of Science and Social Development. https://doi.org/10.55732/jossd.v7i1.1240
Evans, G., Evans, U., & Esin, J. O. (2026). Comparative analysis of psychometric profiling of AI-generated and teacher-constructed multiple-choice question items. Journal of Innovation in Educational Assessment. https://doi.org/10.66545/s7vd5d22
Fajri, A. S., & H. (2025). Critical and creative thinking in vocational education: Validation and reliability of a 21st-century skills instrument. JPI (Jurnal Pendidikan Indonesia). https://doi.org/10.23887/jpi-undiksha.v14i4.97122
González-Navarro, P., Córdoba-Iñesta, A. I., Casino-García, A. M., & Llinares-Insa, L. I. (2023). Evaluating employability in contexts of change: Validation of a scale. Frontiers in Psychology, 14, 1150008. https://doi.org/10.3389/fpsyg.2023.1150008
Gregory, R. J. (2015). Psychological testing: History, principles, and applications (7th ed.). Pearson.
Hidayanti, A. (2019). Hubungan antara social skills dengan employability skills pada mahasiswa Psikologi Universitas Muhammadiyah Prof. Dr. Hamka Jakarta (Disertasi, Universitas Mercu Buana). Mercu Buana University Institutional Repository.
Islami, S., Ambiyar, Sukardi, Zaus, M., & Zaus, A. A. (2024). Development of an online project-based learning assessment instrument for vocational education students. JKIP: Jurnal Kajian Ilmu Pendidikan. https://doi.org/10.55583/jkip.v4i2.804
Ismail, F. K. M., & Zubairi, A. M. B. (2022). Item analysis of a reading test in a Sri Lankan context using classical test theory. International Journal of Learning, Teaching and Educational Research. https://doi.org/10.26803/ijlter.21.3.3
Ismail, R. M. F. H. R., Rahim, M. B., & Sulaiman, J. (2023). Validity and reliability of research instrument in evaluation of work-based learning (WBL) elements. Online Journal for TVET Practitioners. https://doi.org/10.30880/ojtp.2023.08.02.011
Iyamu, I., Ramachandran, S., Chang, H.-J., Kushniruk, A., Ibáñez-Carrasco, F., Worthington, C., Davies, H., McKee, G., Brown, A., & Gilbert, M. (2025). Considerations for adapting digital competencies and training approaches to the public health workforce: An interpretive description of practitioners’ perspectives in Canada. BMC Public Health, 25. https://doi.org/10.1186/s12889-024-21089-1
Jaya, S., Zaharudin, R., Hashim, S. N. A., Ithnin, M. A., Zaid, S. M., Mapjabil, J., & Nordin, M. N. (2021). Employing design and development research (DDR) approach in designing next generation learning spaces (NGLS) in teachers’ pedagogy and technology tools. Review of International Geographical Education (RIGEO), 11(7), 1237–1246. https://doi.org/10.48047/rigeo.11.07.116
Jha, P. K., Ghorai, S., Jha, R. K., & Singh, S. P. (2026). Cognitive break up in operational practice of good manufacturing system: Knowledge-driven performance appraisal outlook. Benchmarking: An International Journal. https://doi.org/10.1108/BIJ-09-2024-0817
Kalyana, D., Hayatunnufus, G., Fauziah, F. H., Wijaya, H., Haryani, R., Sudarmaji, Sumarmia, Larashati, L., & Damayanti, F. (2025). Implementasi asesmen autentik berbasis kompetensi dalam pembelajaran abad ke-21. Jurnal Pengabdian Masyarakat dan Riset Pendidikan. https://doi.org/10.31004/jerkin.v4i2.3665
Kılcı, C. G. (2025). ChatGPT vs. DeepSeek: A comparative psychometric evaluation of AI tools in generating multiple-choice questions. International Journal of Assessment Tools in Education. https://doi.org/10.21449/ijate.1674995
Luesia, J., Benítez, I., Company-Córdoba, R., Gómez-Gómez, I., & Sánchez-Martín, M. (2023). Assessing the relevance of academic competencies in college admission tests from a higher-order thinking perspective: A systematic review. Thinking Skills and Creativity. https://doi.org/10.1016/j.tsc.2023.101251
Magdalena, I., Fauziah, S. N., Faziah, S. N., & Nupus, F. S. (2021). Analisis validitas, reliabilitas, tingkat kesulitan dan daya beda butir soal ujian akhir semester tema 7 kelas III SDN Karet 1 Sepatan. Bintang, 3(2), 198–214. https://doi.org/10.36088/bintang.v3i2.1291
McArthur, J. (2022). Rethinking authentic assessment: Work, well-being, and society. Higher Education, 85, 85–101. https://doi.org/10.1007/s10734-022-00822-y
Mirnawati, M., Sulfasyah, S., & Rahmawati, R. (2022). Validitas buku saku digital muatan mata pelajaran Bahasa Indonesia kelas lima sekolah dasar berbantuan aplikasi Android. Dwija Cendekia: Jurnal Riset Pedagogik, 6(2), 253–262. https://doi.org/10.20961/jdc.v6i2.62650
Mokhtar, S. A., Arifin, N. A. M., Rosman, M., Abdullah, N., Fadzil, F. H., & Isa, A. (2022). Graduates’ competencies and employability: A conceptual framework. International Academic Symposium of Social Science 2022. https://doi.org/10.3390/proceedings2022082054
National Association of Colleges and Employers. (2024). NACE competency assessment tool. https://cdn.uconnectlabs.com/wp-content/uploads/sites/282/2024/11/nace-competency-assessment-tool-combined-revised-aug-2024.pdf
Nickerson, C., & Davidson, P. (2024). Using scenario-based assessment in the development of students’ digital communication skills and professional competence. Business and Professional Communication Quarterly, 87(3), 324–336. https://doi.org/10.1177/23294906241240247
Ntumi, S., Agbenyo, S., & Bulala, T. (2023). Estimating the psychometric properties (item difficulty, discrimination and reliability indices) of test items using Kuder-Richardson approach (KR-20). Shanlax International Journal of Education. https://doi.org/10.34293/education.v11i3.6081
Nuis, W., Segers, M., & Beausaert, S. (2024). Measuring mentoring in employability-oriented higher education programs: Scale development and validation. Higher Education, 87(4), 899–921. https://doi.org/10.1007/s10734-023-01042-8
Nurhalima, A. S. (2023). Building competitive employees in the digital era with competency-based training. KOLOKIUM: Jurnal Pendidikan Luar Sekolah. https://doi.org/10.24036/kolokium.v11i3.767
Putri, A. M., & Nuryono, W. (2025). Validasi media bimbingan dan konseling modul self-control sebagai pencegahan penyalahgunaan narkoba. Jurnal BK UNESA, 15(2), 278–288. https://ejournal.unesa.ac.id/index.php/jurnal-bk-unesa/article/view/69186
Rahman, A. A., & Nasryah, C. E. (2019). Evaluasi pembelajaran. Uwais Inspirasi Indonesia.
Ramachandran, S., Chang, H.-J., Worthington, C., Kushniruk, A., Ibáñez-Carrasco, F., Davies, H. W., McKee, G., Brown, A., Gilbert, M., & Iyamu, I. (2023). Digital competencies and training approaches to enhance the capacity of practitioners to support the digital transformation of public health: Rapid review of current recommendations. JMIR Public Health and Surveillance, 10. https://doi.org/10.2196/52798
Rashidi, S. N., Majid, F. A., & Hashim, H. (2022). Building soft-employability skills (SES-KIT): Reliability, face validity and content validity testing. International Journal of Academic Research in Progressive Education and Development. https://doi.org/10.6007/IJARPED/v11-i3/14546
Rausch, A., Abele, S., Deutscher, V., Greiff, S., Kis, V., Messenger, S., Shackleton, J., Tramonte, L., Ward, M. E., & Winther, E. (2024). Designing an international large-scale assessment of professional competencies and employability skills: Emerging avenues and challenges of OECD’s PISA-VET. Vocations and Learning, 17, 393–432. https://doi.org/10.1007/s12186-024-09347-0
Retnawati, H. (2016). Analisis kuantitatif instrumen penelitian. Parama Publishing.
Rezigalla, A., Eleragi, A. M., Elhussein, A., Alfaifi, J., Alghamdi, M., Ameer, A. A. Y., Yahia, A., Mohammed, O. A., & Adam, M. I. (2024). Item analysis: The impact of distractor efficiency on the difficulty index and discrimination power of multiple-choice items. BMC Medical Education, 24. https://doi.org/10.1186/s12909-024-05433-y
Safitri, K. H., Sofiyah, E., Kusmana, S., & J. (2024). Authentic assessment based on higher order thinking skills in improving student literacy. British Journal of Teacher Education and Pedagogy. https://doi.org/10.32996/bjtep.2024.3.3.14
Setiawan, A., Muhyi, H. A., & Hakim, R. M. A. (2025). Pengaruh employability skills terhadap pengembangan adaptabilitas karier (Studi pada alumni Indonesian International Student Mobility Awards di Universitas Padjadjaran). JMPIS: Jurnal Manajemen Pendidikan dan Ilmu Sosial, 6(2), 1321–1332. https://doi.org/10.38035/jmpis.v6i2.4219
Shiihi, S., Okafor, U., Ekeocha, Z., Byrn, S., & Clase, K. (2021). Improving the outcome of GMP inspections by improving proficiency of inspectors through consistent GMP trainings. https://doi.org/10.5703/1288284317433
Singh, O. I. (2026). A fuzzy logic-based model for assessing employability skills in higher education: Preparing students for emerging job markets. International Journal of Science, Strategic Management and Technology. https://doi.org/10.55041/ijsmt.v2i3.260
Sugiyono. (2021). Metode penelitian kuantitatif, kualitatif, dan R&D (2nd ed.). Alfabeta.
Tan, C. P., Howes, D., Tan, R. K. W., & Dancza, K. M. (2021). Developing interactive oral assessments to foster graduate attributes in higher education. Assessment & Evaluation in Higher Education, 47, 1183–1199. https://doi.org/10.1080/02602938.2021.2020722
Thornhill-Miller, B., Camarda, A., Mercier, M., Burkhardt, J.-M., Morisseau, T., Bourgeois-Bougrine, S., Vinchon, F., Hayek, S. E., Augereau-Landais, M., Mourey, F., Feybesse, C., Sundquist, D., & Lubart, T. (2023). Creativity, critical thinking, communication, and collaboration: Assessment, certification, and promotion of 21st century skills for the future of work and education. Journal of Intelligence, 11. https://doi.org/10.3390/jintelligence11030054
Tischendorf, T., Hasseler, M., Schaal, T., Ruppert, S.-N., Marchwacka, M. A., Heitmann-Möller, A., & Schaffrin, S. (2024). Developing digital competencies of nursing professionals in continuing education and training: A scoping review. Frontiers in Medicine, 11. https://doi.org/10.3389/fmed.2024.1358398
Tong, M., & Gao, T. (2022). For sustainable career development: Framework and assessment of the employability of business English graduates. Frontiers in Psychology, 13, 847247. https://doi.org/10.3389/fpsyg.2022.847247
Tushar, H., & Sooraksa, N. (2023). Global employability skills in the 21st century workplace: A semi-systematic literature review. Heliyon, 9(11). https://doi.org/10.1016/j.heliyon.2023.e21023
Ulya, Z., Erwan, F., Zikri, I., Deliana, F., & Auza, M. (2025). Evaluation of good manufacturing practices (GMP) in Aceh patchouli industry. IOP Conference Series: Earth and Environmental Science, 1510. https://doi.org/10.1088/1755-1315/1510/1/012078
Urrofik, S. A. A., Arbarini, M., Suminar, T., & Cahyani, A. (2025). Competency-based training and self-efficacy on work readiness through employability skills in the Center for Vocational Training and Productivity. Jurnal Ilmiah Pendidikan dan Pembelajaran. https://doi.org/10.23887/jipp.v9i1.86823
Vijayalakshmi, P., & J, J. Q. P. (2025). Preparing for life: Investigating the relationship between career readiness and psychological well-being in students. International Scientific Journal of Engineering and Management. https://doi.org/10.55041/isjem.eseh022
Vlachopoulos, D., & Tsitiridou, O. P. (2026). Preparing university graduates for the labour market through employability skills development and university-industry collaboration: A systematic review. Education Sciences. https://doi.org/10.3390/educsci16030426
Waheed, Z. (2026). Implementation of competency-based assessment in technical and vocational education and training (TVET): A systematic review of models, quality assurance, and learner outcomes. Social Science and Human Research Bulletin. https://doi.org/10.55677/sshrb/2026-3050-0314
Winget, M., & Persky, A. M. (2022). A practical review of mastery learning. American Journal of Pharmaceutical Education, 86(10), 8906. https://doi.org/10.5688/AJPE8906
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Febi Fatia Malika, Rayendra Rayendra, Alwen Bentri, Khairul Alfahani
This work is licensed under a Creative Commons Attribution 4.0 International License.
