Developing a standardized English proficiency test based on the CEFR and language test development manual
DOI:
https://doi.org/10.62819/jel.2025.1007Keywords:
CEFR, language test development manual, standardized English proficiency testAbstract
This study was conducted to construct a Council of Europe Common European Framework of Reference-aligned Standardized English Proficiency Test to be reliable and valid. It utilized the instrument development research design participated in by 217 students from one university in Thailand during the second semester of academic year 2024. The actual test material was made based on the Council of Europe’s (2011) Manual for Language Test Development and Examining, with its four major stages: planning, designing, try-out, and informing stakeholders. The listening and reading tests were adopted from the existing references and examination utilized by the university. The findings revealed that the English Proficiency Test was valid, with ds=0.35 for the Listening Test and ds=0.35 for the Reading Test, both at a moderate level. The test reliability result was r=1.35>0.05. Therefore, the test can be considered good and valid since the committee carefully designed and considered the processes of development aligned with the Council of Europe’s (2011) Manual for Language Test Development and Examining. An English proficiency test was developed based on the Council of Europe’s (2011) Manual for Language Test Development and Examining. Findings showed that the test is valid and reliable, making it a useful tool for measuring English proficiency.
References
Abd Gani, N. I., Rathakrishnan, M., & Krishnasamy, H. N. (2020). A pilot test for establishing validity and reliability of qualitative interview in the blended learning English proficiency course. Journal of Critical Reviews, 7(5), 140–143. https://doi.org/10.31838/jcr.07.05.23
Cheewasukthaworn, K. (2020). Developing a standardized English proficiency test in alignment with the CEFR. Journal of Language Teaching and Learning in Thailand, 63, 66–92. https://doi.org/10.58837/CHULA.PASAA.63.1.3
Chen, Y. (2024). Rethinking contextualized English assessment: A case of reading and listening in China’s university entrance exams. Language Assessment Quarterly, 21(1), 1–21. https://doi.org/10.1080/15434303.2023.2285164
Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment – Structured overview of all CEFR scales. https://rm.coe.int/168045b15e
EF Education First. (2022). EF English proficiency index: A ranking of 111 countries and regions by English skills. https://www.ef.com/assetscdn/WIBIwq6RdJvcD9bc8RMd/cefcom-epi-site/reports/2022/ef-epi-2022-english.pdf
Creswell, J. W., & Guetterman, T. C. (2019). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (6th ed.). Pearson.
DeVellis, R. F. (2016). Scale development: Theory and applications (4th ed.). Sage Publications.
Elleman, A. M., & Oslund, E. L. (2019). Reading comprehension research: Implications for practice and policy. Policy Insights from the Behavioral and Brain Sciences, 6(1), 3–11. https://doi.org/10.1177/2372732218816339
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2019). How to design and evaluate research in education (10th ed.). McGraw-Hill Education.
Hendricks, R., & Smith, A. (2024). Designing curriculum-aligned English proficiency tests: Lessons from the ALLTest project in Malaysia. Asian EFL Journal, 26(2), 45–67. https://doi.org/10.55545/aej.v26i2.1053
Irwing, P., & Hughes, D. J. (2018). Test development. In P. Irwing, T. Booth, & D. J. Hughes (Eds.), The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (pp. 1–47). Wiley. https://doi.org/10.1002/9781118489772.ch1
Jayanti, D., Husna, N., & Hidayat, D. N. (2019). The validity and reliability analysis of English national final examination for junior high school. Voices of English Language Education Society, 3(2), 127–135. https://doi.org/10.29408/veles.v3i2.1551.g929
Jing, X. (2019). The reliability and validity of language proficiency assessments for English language learners. Frontier of Higher Education, 1(1), 36–42. https://doi.org/10.36012/fhe.v1i1.893
Khanal, P. (2020). Key considerations in test construction, scoring and analysis: A guide to pre-service and in-service teachers. International Journal of Research Studies in Education, 9(5), 15–24. https://doi.org/10.5861/ijrse.2020.5027
Orozco, R. A. Z., & Shin, S. Y. (2019). Developing and validating an English proficiency test. MEXTESOL Journal, 43(3), 1-11.
Oxford University Press. (2006). Preparation course for the TOEIC test (New ed.). Oxford University Press.
Papageorgiou, S., Wu, S., So, E., & Wu, J. (2022). Aligning a global English language test to a local scale: Validity, methodology, and implementation. Language Testing, 39(3), 404–426. https://doi.org/10.1177/02655322221076377
Park, J. Y., Harding, L., & Shin, D.-S. (2022). Developing a locally situated academic listening test using unscripted videos: A needs-based approach. Assessing Writing, 52, 100600. https://doi.org/10.1016/j.asw.2022.100600
Purnawan, A., Nurharjanto, A. A., & Ilmi, A. N. (2023). Problems faced by English teacher candidates in developing test kits for assessing students’ learning. Script Journal: Journal of Linguistics and English Teaching, 8(2), 215–225. https://doi.org/10.24903/sj.v8i2.1441
Rosaroso, R. C. (2015). Using reliability measures in test validation. European Scientific Journal, 11(18).
Setiabudi, A., Mulyadi, M., & Puspita, H. (2019). An analysis of validity and reliability of a teacher-made test. Journal of English Education and Teaching, 3(4), 522–532. https://doi.org/10.33369/jeet.3.4.522-532
Sims, J. M. (2015). A valid and reliable English proficiency exam: A model from a university language program in Taiwan. English as a Global Language Education (EaGLE) Journal, 2(1), 91–93. https://doi.org/10.6294/EaGLE.2015.0102.04
Sridhanyarat, K., Pathong, S., Suranakkharin, T., & Ammaralikit, A. (2021). The development of STEP, the CEFR-based English proficiency test. English Language Teaching, 14(7), 95–106. https://doi.org/10.5539/elt.v14n7p95
Sugianto, A. (2017). Validity and reliability of English summative test for senior high school. Indonesian EFL Journal: Journal of ELT, Linguistics, and Literature, 3(2), 22–38.
Tannenbaum, R. J., & Wylie, E. C. (2008). Linking English language test scores onto the common European framework of reference: An application of standard setting methodology. ETS Research Report Series, 2008(1), i–75. https://doi.org/10.1002/j.2333-8504.2008.tb02120.x
Lougheed, L. (2018). TOEIC practice exams (4th ed.). Barron’s Educational Series.
van Splunder, F., Verguts, C., De Moor, T., & De Paepe, S. (2022). The interuniversity test of academic English (ITACE) assessing lecturers’ English proficiency in Flanders. Journal of English-Medium Instruction, 1(2), 255–274. https://doi.org/10.1075/jemi.21007.van
Wudthayagorn, J. (2022). An exploration of the English exit examination policy in Thai public universities. Language Assessment Quarterly, 19(2), 107–123. https://doi.org/10.1080/15434303.2021.1937174
Yu, L. T., Chen, M. C., Chiu, C. W., Hsu, C. C., & Yuan, Y. P. (2022). Examining English ability-grouping practices by aligning CEFR levels with university-level General English courses in Taiwan. Sustainability, 14(8), 4629. https://doi.org/10.3390/su14084629
Yu, M. H., Reynolds, B. L., & Ding, C. (2021). Listening and speaking for real-world communication: What teachers do and what students learn from classroom assessments. SAGE Open, 11(2). https://doi.org/10.1177/21582440211009163
Downloads
Published
How to Cite
Issue
Section
