Exploring Computational Thinking Skills in Pre-Service Science Teachers: A Rasch Model Analysis

Riskawati Riskawati; Nurul Mutmainnah Herman; Dirgah Kaso Sanusi; Ihfa Indira Nurnaifah Idris; Abdurrahman Abdurrahman; Hendra B.; Sitti Rahma Yunus

doi:10.59329/gawi.v6i1.286

Authors

Riskawati Riskawati Universitas Negeri Makassar
Nurul Mutmainnah Herman Universitas Negeri Makassar
Dirgah Kaso Sanusi Universitas Negeri Makassar
Ihfa Indira Nurnaifah Idris Universitas Negeri Makassar
Abdurrahman Abdurrahman Universitas Negeri Makassar
Hendra B. IAI Yapnas Jeneponto
Sitti Rahma Yunus The University of Queensland

DOI:

https://doi.org/10.59329/gawi.v6i1.286

Keywords:

Computational thinking, Pre-service science teachers, Item analysis, Gender bias

Abstract

Computational Thinking (CT) has been increasingly recognized as a key competency in science education, especially in preparing future educators to foster 21st-century skills such as problem-solving and digital literacy. However, limited CT instruction in undergraduate programs has resulted in inadequate readiness among pre-service science teachers, particularly in contexts like Indonesia. This study addresses the gap in the evaluation of CT skills among pre-service teachers by using robust psychometric approaches. This article investigates the psychometric properties of a CT assessment tool, examines the CT ability levels of pre-service science teachers, and evaluates the presence of gender-based item bias. The study employed a quantitative, cross-sectional design utilizing Rasch model analysis to assess instrument reliability, item fit, and differential item functioning (DIF) across gender. A total of 419 pre-service science teachers from two universities in Indonesia were selected using stratified random sampling to ensure proportional representation across academic levels. Data collection and analysis: Data were collected using the 28-item Computational Thinking Test (CTt) and analyzed using Winsteps software for Rasch modeling, including item/person reliability, separation, fit statistics, Wright maps, and DIF analysis. Results indicated high item reliability and good model fit, though person reliability was relatively low, suggesting limited ability discrimination. Several items exhibited gender-based DIF, with some favoring males and others favoring females. The CTt shows strong potential for measuring CT, though refinement is needed to improve its ability to differentiate between skill levels and ensure gender fairness. CT is increasingly recognized as an essential competency in science education to support problem-solving, analytical reasoning, and 21st-century digital literacy skills. However, empirical evidence regarding the measurement quality of CT instruments among pre-service science teachers in Indonesia remains limited. This study aimed to evaluate the psychometric properties of the CTt, describe the distribution of CT skills among pre-service science teachers, and examine gender-based item bias using the Rasch model. A quantitative cross-sectional design was employed involving 419 pre-service science teachers from two Indonesian universities selected through stratified random sampling. Data were analyzed using Winsteps software to examine reliability, separation index, item fit, Wright map distribution, and DIF. The results showed that the CTt demonstrated high item reliability and good model fit, indicating that the instrument is appropriate for measuring CT skills. However, person reliability was relatively low, suggesting limited sensitivity in distinguishing individual ability levels. Wright map analysis indicated that most respondents were distributed at a moderate ability level. In addition, DIF analysis identified several items with potential gender bias, favoring both male and female respondents. Overall, the CTt shows strong potential as an instrument for measuring CT skills, although further refinement is needed to improve measurement sensitivity and ensure gender fairness. These findings contribute to the development of Rasch model–based CT assessment in pre-service science teacher education.

References

Albayrak, E., & Yilmaz Ozden, Ş. (2021). Improvement of pre-service teachers’ computational thinking skills through an educational technology course. Journal of Individual Differences in Education, 3(2), 97–112. https://doi.org/10.47156/jide.1027431

Aminger, W., Hough, S., Roberts, S. A., Meier, V., Spina, A. D., Pajela, H., McLean, M., & Bianchini, J. A. (2021). Preservice secondary science teachers’ implementation of an ngss practice: Using mathematics and computational thinking. Journal of Science Teacher Education, 32(2), 188–209. https://doi.org/10.1080/1046560X.2020.1805200

Bati, K. (2021). Integration of python into science teacher education, developing computational problem solving and using information and communication technologies competencies of pre-service science teachers. Informatics in Education. https://doi.org/10.15388/infedu.2022.12

Bond T. G., & Fox C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences. Routledge.

Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch Analysis in the Human Sciences. Springer Netherlands. https://doi.org/10.1007/978-94-007-6857-4

Boulden, D. C., Rachmatullah, A., Oliver, K. M., & Wiebe, E. (2021). Measuring in-service teacher self-efficacy for teaching computational thinking: Development and validation of the T-STEM CT. Education and Information Technologies, 26(4), 4663–4689. https://doi.org/10.1007/s10639-021-10487-2

Bradley, K. D., Peabody, M. R., Akers, K. S., & Knutson, N. M. (2015). Rating Scales in Survey Research: Using the Rasch model to illustrate the middle category measurement flaw. Survey Practice, 8(1), 1–12. https://doi.org/10.29115/SP-2015-0001

Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. AERA.

Butler, D., & Leahy, M. (2020). Using classroom practice as “an object to think with” to develop preservice teachers’ understandings of computational thinking. In T. Brinda, D. Passey, & T. Keane (Eds.), Empowering Teaching for Digital Equity and Agency (Vol. 595, pp. 56–65). Springer International Publishing. https://doi.org/10.1007/978-3-030-59847-1_6

Caballero, M. D., & Merner, L. (2018). Prevalence and nature of computational instruction in undergraduate physics programs across the United States. Physical Review Physics Education Research, 14(2), 020129. https://doi.org/10.1103/PhysRevPhysEducRes.14.020129

Chan, S.-W., Looi, C.-K., & Sumintono, B. (2021). Assessing computational thinking abilities among Singapore secondary students: A Rasch model measurement analysis. Journal of Computers in Education, 8(2), 213–236. https://doi.org/10.1007/s40692-020-00177-2

Connolly, C., Hijón Neira, R., & Garcia-Iruela, M. (2021). Developing and assessing computational thinking in secondary education using a tpack guided scratch visual execution environment. International Journal of Computer Science Education in Schools, 4(4), 3–23. https://doi.org/10.21585/ijcses.v4i4.98

Dong, W., Li, Y., Sun, L., & Liu, Y. (2023). Developing pre-service teachers’ computational thinking: A systematic literature review. International Journal of Technology and Design Education. https://doi.org/10.1007/s10798-023-09811-3

Elbahan, H., Elbahan, M. H., & Balbağ, M. Z. (2023). Determining the level of computational thinking skills of science teacher candidates. Osmangazi Journal of Educational Research, 10 (Special Issue), 254–272. https://doi.org/10.59409/ojer.1369711

Fülöp, M. T., Udvaros, J., Gubán, Á., & Sándor, Á. (2022). Development of computational thinking using microcontrollers integrated into oop (object-oriented programming). Sustainability, 14(12), 7218. https://doi.org/10.3390/su14127218

Ghani, A., Griffiths, D., Salha, S., Affouneh, S., Khalili, F., Khlaif, Z. N., & Burgos, D. (2022). Developing teaching practice in computational thinking in palestine. Frontiers in Psychology, 13, 870090. https://doi.org/10.3389/fpsyg.2022.870090

González, M. R. (2015). Computational thinking test: Design Guidelines and Content Validation. In EDULEARN15 proceedings (pp. 2436-2444). IATED. https://doi.org/10.13140/RG.2.1.4203.4329

Ismail, I., Riandi, R., Kaniawati, I., Sopandi, W., Supriyadi, S., Suhendar, S., & Hidayat, F. A. (2024). Gender roles in understanding and implementing green energy technology in indonesian schools: Rasch analysis. Qubahan Academic Journal, 4(3), 298–314. https://doi.org/10.48161/qaj.v4n3a752

Ismarmiaty, I., Agustin, K., Madani, M., Sriwinarti, N. K., Zainuddin, Z., & Supatmiwati, D. (2022). Penguatan kemampuan computational thinking pada pemberdayaan guru dan siswa sekolah dasar di pulau lombok. Transformasi: Jurnal Pengabdian Masyarakat, 18(2), 253–267. https://doi.org/10.20414/transformasi.v18i2.5034

Kaleli̇Oğlu, F., Gülbahar, Y., & Kukul, V. (2016). A framework for computational thinking based on a systematic research review. Baltic J. Modern Computing, 4(3), 583–596.

Khine, M. S. (2020). Objective measurement in psychometric analysis. In M. S. Khine (Ed.), Rasch Measurement (pp. 3–7). Springer Singapore. https://doi.org/10.1007/978-981-15-1800-3_1

Kong, S., & Lai, M. (2023). Effects of a teacher development program on teachers’ knowledge and collaborative engagement, and students’ achievement in computational thinking concepts. British Journal of Educational Technology, 54(2), 489–512. https://doi.org/10.1111/bjet.13256

Kumala, F. N., Yasa, A. D., Jait, A. B. H., Wibawa, A. P., & Hidayah, L. (2023). Patterns of computational thinking skills for elementary prospectives teacher in science learning: Gender analysis studies. International Journal of Elementary Education, 7(4), 646–656. https://doi.org/10.23887/ijee.v7i4.68611

Lee, I., Grover, S., Martin, F., Pillai, S., & Malyn-Smith, J. (2020). Computational thinking from a disciplinary perspective: Integrating computational thinking in k-12 science, technology, engineering, and mathematics education. Journal of Science Education and Technology, 29(1), 1–8. https://doi.org/10.1007/s10956-019-09803-w

Liu, T., Gonzalez-Maldonado, D., Harlow, D. B., Edwards, E. E., & Franklin, D. (2023). Qupcakery: A puzzle game that introduces quantum gates to young learners. SIGCSE - Proc. ACM Tech. Symp. Comput. Sci. Educ., 1, 1143–1149. Scopus. https://doi.org/10.1145/3545945.3569837

Lyon, J. A., & J. Magana, A. (2020). Computational thinking in higher education: A review of the literature. Computer Applications in Engineering Education, 28(5), 1174–1189. https://doi.org/10.1002/cae.22295

M Esteve-Mon, F., Adell-Segura, J., Ángeles Llopis Nebot, M., Valdeolivas Novella, G., & Pacheco Aparicio, J. (2019). The development of computational thinking in student teachers through an intervention with educational robotics. Journal of Information Technology Education: Innovations in Practice, 18, 139–152. https://doi.org/10.28945/4442

Marifah, S. N. (2022). Systematic literatur review: Integrasi computational thinking dalam kurikulum sekolah dasar di indonesia. COLLASE (Creative of Learning Students Elementary Education), 5(5), 928-938.

Newman, C. (2014). Time to address gender discrimination and inequality in the health workforce. Human resources for health, 12(1), 25.

Odden, T. O. B., & Burk, J. (2020). Computational essays in the physics classroom. The Physics Teacher, 58(4), 252–255. https://doi.org/10.1119/1.5145471

Peel, A., Sadler, T. D., & Friedrichsen, P. (2022). Algorithmic explanations: An unplugged instructional approach to integrate science and computational thinking. Journal of Science Education and Technology, 31(4), 428–441. https://doi.org/10.1007/s10956-022-09965-0

Peters-Burton, E., Rich, P. J., Kitsantas, A., Laclede, L., & Stehle, S. M. (2022). High school science teacher use of planning tools to integrate computational thinking. Journal of Science Teacher Education, 33(6), 598–620. https://doi.org/10.1080/1046560X.2021.1970088

Prastowo, A., & Fitriyaningsih, F. (2020). Learning material changes as the impact of the 2013 curriculum policy for the primary school/madrasah ibtidaiyah. Edukasia : Jurnal Penelitian Pendidikan Islam, 15(2), 251. https://doi.org/10.21043/edukasia.v15i2.7947

Subramaniam, S., Maat, S. M. M., & Mahmud, M. S. M. (2023). Designing problem-solving module based on computational thinking in mathematics education: Nominal group technique approach. International Journal of Academic Research in Progressive Education and Development, 12(2), Pages 1381-1396. https://doi.org/10.6007/IJARPED/v12-i2/17262

Sumintono, B., & Widhiarso, W. (2014). Aplikasi model rasch untuk penelitian ilmu-ilmu sosial (edisi revisi). Trim Komunikata Publishing House

Syafril, S., Rahayu, T., & Ganefri, G. (2022). Prospective science teachers’ self-confidence in computational thinking skills. Jurnal Pendidikan IPA Indonesia, 11(1), 119–128. https://doi.org/10.15294/jpii.v11i1.33125

Tongal, A., Yıldırım, F. S., Özkara, Y., Say, S., & Erdoğan, Ş. (2024). Examining teachers’ computational thinking skills, collaborative learning, and creativity within the framework of sustainable education. Sustainability, 16(22), 9839. https://doi.org/10.3390/su16229839

Tsakeni, M. (2021). Preservice teachers’ use of computational thinking to facilitate inquiry-based practical work in multiple-deprived classrooms. Eurasia Journal of Mathematics, Science and Technology Education, 17(1), em1933. https://doi.org/10.29333/ejmste/9574

Verawati, N. N. S. P., Rijal, K., & Grendis, N. W. B. (2023). Examining stem students’ computational thinking skills through interactive practicum utilizing technology. International Journal of Essential Competencies in Education, 2(1), 54–65. https://doi.org/10.36312/ijece.v2i1.1360

Wardani, I. R. W., Putri Zuani, M. I., & Kholis, N. (2023). Teori belajar perkembangan kognitiv lev vygotsky dan implikasinya dalam pembelajaran. DIMAR: Jurnal Pendidikan Islam, 4(2), 332–346. https://doi.org/10.58577/dimar.v4i2.92

Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215

Wing, J. M. (2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 366(1881), 3717–3725. https://doi.org/10.1098/rsta.2008.0118

Zhu, M., & Wang, C. (2023). Core competencies of K-12 computer science education from the perspectives of college faculties and K-12 teachers. International Journal of Computer Science Education in Schools, 6(2). https://doi.org/10.21585/ijcses.v6i2.161

Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical bayes approach to mantel‐haenszel dif analysis. Journal of Educational Measurement, 36(1), 1–28. https://doi.org/10.1111/j.1745-3984.1999.tb00543.x