Aligning the CULI Test with CEFR Illustrative Scales

Sasithorn Limgomolvilas

doi:10.66947/pasaa.v73ispc.2105

Authors

Sasithorn Limgomolvilas Chulalongkorn University Language Institute

DOI:

https://doi.org/10.66947/pasaa.v73ispc.2105

Keywords:

CEFR, standard setting, illustrative scales, large-scale assessment

Abstract

The Chulalongkorn University Language Institute (CULI) Test was first developed in 2006, which was before Thailand required language tests to be aligned with the Common European Framework of Reference (CEFR). Aligning the CULI Test, a local test, to the CEFR, however, can offer stakeholders a more reliable indicator of test-takers’ proficiency level. This study sought to align each item on the CULI Test with the CEFR communicative language competence scales through quantitative and qualitative methods utilizing eight expert judgments and Rasch measurements. The results showed that most of the items were between the B1 and B2 level; other items were at the A2 and C1 level. Qualitative analysis revealed that the items included 8 out of 42 specific activities and strategies highlighted in the CEFR illustrative scales. Reading for information and argument was the most common question type, accounting for 24 of 100 items, followed by overall written comprehension with 20 items, while the least common type was overall oral comprehension with five items. The results indicate that the CULI test is mainly suitable for differentiating test takers at two CEFR levels: B1 and B2.

References

Adams, R. J., Wu, M. L., Cloney, D., Berezner, A., & Wilson, M. R. (2020). ACER ConQuest: Generalized item response modelling software (Version 5.29) [Computer software]. Australian Council for Educational Research. https://www.acer.org/au/conquest

Anantapol, W., Keeratikorntanayod, W., & Chobphon, P. (2018). Developing English proficiency standards for English language teachers in Thailand. Manutsayasat Wichakan, 25(2), 1–35.

Athiworakun, C., & Wudthayagorn, J. (2018). Mapping Srinakharinwirot University-Standardized English Test (SWU-SET) onto the Common European Framework of Reference (CEFR). Suranaree Journal of Social Science, 12(2), 69–84. https://doi.org/10.55766/CTUU4836

Azman, H., Othman, Z., Shamsuddin, C. M., Wahi, W., Aziz, M. S. A., Mohamad, W. N. W., Othman, S., & Amin, M. H. M. (2021). Relating a sustained monologue speaking production rest to CEFR: Towards alignment. Pertanika Journal of Social Sciences & Humanities, 29, 385–400. https://doi.org/10.47836/pjssh.29.S3.20

Baharum, N., Ismail, L., Nordin, N., & Razali, A. (2021). Aligning a university English language proficiency measurement tool with the CEFR: A case in Malaysia. Pertanika Journal of Social Sciences and Humanities. https://doi.org/10.47836/pjssh.29.s3.09

Bramley, T., & Wilson, F. (2016). Maintaining test standards by expert judgement of item difficulty. Research Matters: A Cambridge Assessment Publication, 21, 48–54.

Chen, L.-C., Chang, K.-H., Yang, S.-C., & Chen, S.-C. (2023). A corpus-based word classification method for detecting difficulty level of English proficiency tests. Applied Sciences, 13(3), Article 1699. https://doi.org/10.3390/app13031699

Cizek, G. J. (Ed.). (2012). Setting performance standards: Foundations, methods, and innovations (2nd ed.). Routledge. https://doi.org/10.4324/9780203848203

Cizek, G. J., & Bunch, M. B. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. Sage Publications.

Council of Europe (2001). Common European Framework of Reference for languages: Learning, teaching, assessment. https://rm.coe.int/1680459f97

Council of Europe (2020). Common European Framework of Reference for languages: Learning, teaching, assessment – Companion volume. https://www.coe.int/lang-cefr

Eckes, T. (2015). Introduction to many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments (2nd ed.). Peter Lang.

Education Testing Service (2020). TOEFL® research insight series, Volume 6: TOEFL program history. https://www.ets.org/pdfs/toefl/toefl-ibt-insight-s1v6.pdf

Flowerdew, J. (2002). Ethnographically inspired approaches to the study of academic discourse. In J. Flowerdew (Ed.), Academic discourse (pp. 235–252). Pearson Education.

Green, A. (2017). Linking tests of English for academic purposes to the CEFR: The score user’s perspective. Language Assessment Quarterly, 15(1), 59–74. https://doi.org/10.1080/15434303.2017.1350685

Hidri, S. (2021). Linking the international English language competency assessment suite of examinations to the common European framework of reference. Language Testing in Asia, 11(1), 1–24. https://doi.org/10.1186/s40468-021-00123-8

Humphry, S., Heldsinger, S., & Andrich, D. (2014). Requiring a consistent unit of scale between the responses of students and judges in standard setting. Applied Measurement in Education, 27(1), 1–18.

Impara, J. C., & Plake, B. S. (1997). Standard setting: An alternative approach. Journal of Educational Measurement, 34(4), 353–366. http://www.jstor.org/stable/1435114

Jabeen, F., Ruqia, & Bahar, L. (2025). Examine the effects of high-stake resting on student’s mental health at university level: A survey research. Kashf Journal of Multidisciplinary Research, 2(5), 1–19. https://doi.org/10.71146/kjmr430

Lee, T., Milanovic, M., & Pike, N. (2022). Equating Rasch values and expert judgement through externally-referenced anchoring. International Journal of TESOL Studies, 4(1), 187–202. https://doi.org/10.46451/ijts.2022.01.12

Martyniuk, W. (Ed.). (2010). Aligning tests with the CEFR: Reflections on using the Council of Europe's draft manual (Vol. 33). Cambridge University Press.

Min, S., & Bishop, K. (2024). A shortened test is feasible: Evaluating a large-scale multistage adaptive English language assessment. Language Testing, 41, 627–648. https://doi.org/10.1177/02655322231225426

Papageorgiou, S., Tannenbaum, R. J., Bridgeman, B., & Cho, Y. (2015). The association between TOEFL iBT® test scores and the Common European Framework of Reference (CEFR) levels. Research Memorandum No. RM-15-06. Educational Testing Service.

Sae-Ong, U. (2025). University students’ self-efficacy and assessment literacy: Insights from the English Exit Exam policy in Thailand. Language Testing in Asia, 15, Article 55. https://doi.org/10.1186/s40468-025-00379-4

Shak, P., & Read, J. (2021). Aligning the language criteria of a group oral test to the CEFR: The case of a formal meeting assessment in an English for occupational purposes classroom. Pertanika Journal of Social Sciences & Humanities, 29, 133–156. https://doi.org/10.47836/pjssh.29.s3.08

Siripol, P., Rhee, S., Thirakunkovit, S., & Liang-Itsara, A. (2025). Evaluating the consistency of automated CEFR analyzers: A study of English language text classification. International Journal of Evaluation and Research in Education, 14(4), 3283–3294. https://doi.org/10.11591/ijere.v14i4.33528

Tangsakul, S., & Poonpon, K. (2024). Aligning academic reading tests to the Common European Framework of Reference for languages (CEFR). rEFLections, 31(2), 614–638. https://doi.org/10.61508/refl.v31i2.275057

Wudthayagorn, J. (2018). Mapping the CU-TEP to the Common European Framework of Reference (CEFR). LEARN Journal: Language Education and Acquisition Research Network, 11(2), 163–180. https://so04.tci-thaijo.org/index.php/LEARN/article/view/161641/116576