Measuring Scientific Reasoning in Indonesian Senior High School Students: A Rasch Model Analysis
DOI:
https://doi.org/10.59329/gawi.v6i2.323Keywords:
Lawson test, psychometric analysis, Rasch model, Scientific reasoning, Wright MapAbstract
This study aimed to evaluate the psychometric quality of a scientific reasoning instrument using the Rasch measurement model. A descriptive survey method with a non-experimental cross-sectional design was employed. The participants comprised 65 eleventh-grade senior high school students from a single school in Central Sulawesi, Indonesia, selected through purposive sampling. The instrument was adapted from Lawson's Classroom Test of Scientific Reasoning (LCTSR) and consisted of 12 multiple-choice items covering four dimensions of scientific reasoning: proportional reasoning, control of variables, combinatorial reasoning, and probabilistic reasoning. Data were analyzed using Rasch model analysis to examine reliability, item fit, person ability, and unidimensionality. The results indicated functional but imperfect measurement quality: person reliability was moderate to weak (0.63), item reliability was strong (0.92), and Cronbach's alpha (0.66) reflected acceptable internal consistency. One item (Q6) showed clear misfit (Outfit MNSQ = 2.39), and two items (Q4 and I0011) showed weaker-than-ideal discrimination (PTMEA CORR = 0.31), indicating that not all items met the fit criteria established in this study. Students' ability estimates ranged from −2.85 to +2.93 logits, reflecting heterogeneous levels of scientific reasoning ability, and the Wright Map showed a reasonably good, though imperfect, alignment between student ability and item difficulty. Item Q6, representing the control-of-variables dimension, was identified as the most difficult and most misfitting item, while item Q7, representing combinatorial reasoning, was the easiest, suggesting that students experienced greater difficulty in causal reasoning and variable control than in combinatorial reasoning tasks. Differential Item Functioning (DIF) analysis by gender indicated no statistically significant bias for most items, although item Q7 showed a significant DIF contrast on one of two tests used; this result should be interpreted cautiously, given the small and unbalanced gender subgroups. These findings suggest that scientific reasoning develops unevenly across dimensions and may be influenced by task complexity, while also highlighting specific items requiring further refinement before the instrument is used more broadly.
References
Abate, T., Michael, K., & Angell, C. (2020). Assessment of scientific reasoning: Development and validation of scientific reasoning assessment tool. Eurasia Journal of Mathematics, Science and Technology Education, 16(12), em1927.
Aminudin, A. H., Adimayuda, R., Kaniawati, I., Suhendi, E., Samsudin, A., & Coştu, B. (2019). Rasch analysis of Multitier Open-ended Light-Wave Instrument (MOLWI): Developing and assessing second-years sundanese-scholars alternative conceptions. Journal for the Education of Gifted Young Scientists, 7(3), 557–579. https://doi.org/10.17478/jegys.574524
Amiruddin, M. Z. Bin, Samsudin, A., Suhandi, A., Kaniawati, I., COŞTU, B., Aminuddin, A. H., & Kuniawan, F. (2023). Validity and Reliability of the Global Warming Instrument: A Pilot Study Using Rasch Model Analysis. Jurnal Pendidikan MIPA, 24(4), 912–922.
Bachtiar, R. W., Meulenbroeks, R. F. G., & van Joolingen, W. R. (2022). Mechanistic reasoning in science education: A literature review. Eurasia Journal of Mathematics, Science and Technology Education, 18(11), em2178.
Bao, L., Koenig, K., Xiao, Y., Fritchman, J., Zhou, S., & Chen, C. (2022). Theoretical model and quantitative assessment of scientific thinking and reasoning. Physical Review Physics Education Research, 18(1), 10115.
Boone, W. J., Staver, J. R., Yale, M. S., Boone, W. J., Staver, J. R., & Yale, M. S. (2014a). Some Wright map nuances: How to set the probability of success at 65%(or whichever percentage you wish to choose). Rasch Analysis in the Human Sciences, 255–271.
Boone, W. J., Staver, J. R., Yale, M. S., Boone, W. J., Staver, J. R., & Yale, M. S. (2014b). Wright maps: First steps. Rasch Analysis in the Human Sciences, 111–136.
Brown, N. J. S., Furtak, E. M., Timms, M., Nagashima, S. O., & Wilson, M. (2010). The evidence-based reasoning framework: Assessing scientific reasoning. Educational Assessment, 15(3–4), 123–141.
Chen, C.-T., & She, H.-C. (2015). The effectiveness of scientific inquiry with/without integration of scientific reasoning. International Journal of Science and Mathematics Education, 13(1), 1–20.
Choowong, K., & Worapun, W. (2021). The Development of Scientific Reasoning Ability on Concept of Light and Image of Grade 9 Students by Using Inquiry-Based Learning 5E with Prediction Observation and Explanation Strategy. Journal of Education and Learning, 10(5), 152–159.
Depaoli, S., Tiemensma, J., & Felt, J. M. (2018). Assessment of health surveys: fitting a multidimensional graded response model. Psychology, Health & Medicine, 23(sup1), 1299–1317.
Díaz, C., Dorner, B., Hussmann, H., & Strijbos, J.-W. (2023). Conceptual review on scientific reasoning and scientific thinking. Current Psychology, 42(6), 4313–4325.
Facione, P. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction (The Delphi Report).
Giri, V., & Paily, M. U. (2020). Effect of scientific argumentation on the development of critical thinking. Science & Education, 29(3).
Gopnik, A. (2012). Scientific thinking in young children: Theoretical advances, empirical research, and policy implications. Science, 337(6102), 1623–1627.
Han, J. (2013). Scientific reasoning: Research, development, and assessment. The Ohio State University.
Kambeyo, L., & Csapó, B. (2018). Scientific reasoning skills: A theoretical backgrounds to science education. Reform Forum: Journal for Educational Research in Namibia, 26(1), 27–36.
Khan, K. A., & Rana, R. A. (2021). Effect of Higher Order Thinking Instructional Model on Scientific Reasoning of Grade VIII Students. Bulletin of Education and Research, 43(1), 77–92.
Koslowski, B. (1996). Theory and evidence: The development of scientific reasoning. Mit Press.
Kuhn, D. (2007). Reasoning about multiple variables: Control of variables is not the only challenge. Science Education, 91(5), 710–726.
Linacre, J. M. (1999). Understanding Rasch measurement: estimation methods for Rasch measures. Journal of Outcome Measurement, 3, 382–405.
Linacre, J. M. (2011). A User’s Guide to Winsteps: Rasch-Model Computer Programs, winsteps. com.
Medová, J., Bulková, K. O., & Čeretková, S. (2020). Relations between generalization, reasoning and combinatorial thinking in solving mathematical open-ended problems within mathematical contest. Mathematics, 8(12), 2257.
Novia, N., Syamsu, S., & Riandi, R. (2018). Student’s achievement in Lawson’s classroom scientific reasoning (LCTSR): the effect of gender and age on scientific reasoning ability. International Conference on Mathematics and Science Education of Universitas Pendidikan Indonesia, 3, 542–547.
Nurhayati, N., Suhandi, A., Muslim, M., Kaniawati, I., Wahyudi, W., Misbah, M., & Darman, D. R. (2025). Assessing the Knowledge and Skills of Prospective Physics Teachers in Designing 4C Skills-Oriented Learning: Rasch Analysis. Qubahan Academic Journal, 5(1), 718–741.
Omarchevska, Y., Lachner, A., Richter, J., & Scheiter, K. (2022). It takes two to tango: How scientific reasoning and self-regulation processes impact argumentation quality. Journal of the Learning Sciences, 31(2), 237–277.
Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2), 20111.
Samsudin, A., Wulandari, N., Suhandi, A., Yusup, M., Supriyatman, S., Aminudin, A. H., Sumarni, W., & Kapıcı, H. Ö. (2025). Augmented Reality-based Rebuttal Texts (ARaRaT) on momentum-impulse: Rasch analysis on students’ conceptual change. Qubahan Academic Journal, 5(1), 368–387.
Sapia, P., Napoli, F., & Bozzo, G. (2022). The Lawson’s test for scientific reasoning as a predictor for university formative success: a prospective study. Education Sciences, 12(11), 814.
Schauble, L., Glaser, R., Raghavan, K., & Reiner, M. (1991). Causal models and experimentation strategies in scientific reasoning. The Journal of the Learning Sciences, 1(2), 201–238.
Sumintono, B. (2018). Rasch model measurements as tools in assesment for learning. 1st International Conference on Education Innovation (ICEI 2017), 38–42.
Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan rasch pada assessment pendidikan. Trim komunikata.
Van Vo, D., & Csapó, B. (2021). Development of scientific reasoning test measuring control of variables strategy in physics for high school students: Evidence of validity and latent predictors of item difficulty. International Journal of Science Education, 43(13), 2185–2205.
Xiao, Y., Han, J., Koenig, K., Xiong, J., & Bao, L. (2018). Multilevel Rasch modeling of two-tier multiple choice test: A case study using Lawson’s classroom test of scientific reasoning. Physical Review Physics Education Research, 14(2), 20104.
Zhou, S., Han, J., Koenig, K., Raplinger, A., Pi, Y., Li, D., Xiao, H., Fu, Z., & Bao, L. (2016). Assessment of scientific reasoning: The effects of task context, data, and design on student reasoning in control of variables. Thinking Skills and Creativity, 19, 175–187.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Risnatati Risnatati, Ridwan Efendi, Winny Liliawati, Achmad Samsudin, Duden Saepuzaman
This work is licensed under a Creative Commons Attribution 4.0 International License.
