A Comparative Case Study of the Effectiveness of an Automated Writing Evaluation System in Improving Grammatical Accuracy Across Myanmar EFL Students of Varying English Proficiency Levels

Nan Ingyin Phyu; Tanyaporn Arya

doi:10.66947/pasaa.v72i1.2196

Authors

Nan Ingyin Phyu -
Tanyaporn Arya

DOI:

https://doi.org/10.66947/pasaa.v72i1.2196

Keywords:

Automated Writing Evaluation (AWE), Grammatical Accuracy, CEFR Proficiency Levels, Myanmar EFL Learners, English Writing

Abstract

This exploratory case study examines how Grammarly combined with
reinforcement strategies impacts grammatical accuracy among three
Myanmar EFL students at CEFR levels A2, B1, and B2. Over six weeks,
participants completed writing tasks using Grammarly's free version
alongside weekly one-on-one meetings and self-reflection with Grammar
in Use resources. Progress was assessed through pre- and post-tests and
weekly error tracking. Findings indicate positive effects across all levels,
with differentiated patterns: the A2 learner showed dramatic
improvement relying heavily on automated feedback for remedial
support; the B1 learner demonstrated steady progress requiring balanced
feedback and reflection for consolidation; the B2 learner achieved
refinement using AWE selectively while depending primarily on
reinforcement for deeper understanding. Weekly tracking revealed nonlinear
improvement trajectories, with the greatest gains at lower
proficiency levels. While findings cannot be generalized statistically due
to the small sample, they provide analytical insights into how learners at
different developmental stages process automated feedback. The study
offers practical guidance for educators integrating AWE into CEFRaligned
instruction in resource-constrained EFL contexts.

References

Abdi Tabari, M., Zhuang, J., & Farahanynia, M. (2025). Task repetition and L2 written

performance: A meta-analysis. Journal of Second Language Writing. https://doi.org/10.1016/j.jslw.2025.101255

Abu Guba, M. N. A., Awad, A., & Abu Quba, A. (2024). Grammarly in Teaching Writing to EFL learners at low levels: How useful is it? World Journal of English Language, 14(3), 1. https://doi.org/10.5430/wjel.v14n3p1

Ahangari, S., & Babapour, M. (2015). The Effect of Self-Correction and Peer- Correction on EFL Learners’ Writing. Modern Journal of Language Teaching Methods, 5(2), 465–473.

Al-Alami, S. E. (2024). EFL learners’ attitudes towards utilizing ChatGPT for acquiring writing skills in higher education: A case study of computing students. Journal of Language Teaching and Research, 15(4), 1029–1038. https://doi.org/10.17507/jltr.1504.01

Bruner, J. S. (1966). Toward a theory of instruction. Harvard University Press.

Cambridge University Press. (2019). A2 key for Schools trainer 1.

Cambridge University Press. (2019). A2 key for Schools trainer 2.

Cambridge University Press. (2019). B1 preliminary for Schools trainer 1.

Cambridge University Press. (2020). B1 preliminary for Schools trainer 2.

Cambridge University Press. (2019). B2 first for Schools trainer 1.

Cambridge University Press. (2020). B2 first for Schools trainer 2.

Cambridge English. (2023). The Cambridge English Scale explained.

https://www.cambridgeenglish.org/images/210434-converting-practice-test-scores-tocambridge-english-scale-scores.pdf

Celce-Murcia, M., & Larsen-Freeman, D. (1999). The grammar book: An ESL/EFL teacher's course. Heinle & Heinle.

Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press.

DeKeyser, R. M. (2007). Skill acquisition theory. In B. VanPatten & J. Williams (Eds.), Theories in second language acquisition (pp. 97-113). Routledge.

DeKeyser, R. M. (2018). Task repetition for language learning: A perspective from skill acquisition theory. In M. Bygate (Ed.), Learning language through task repetition.

Dembsey, J. M. (2017). Closing the Grammarly Gaps: A Study of Claims and Feedback from an Online Grammar Program. Deleted Journal, 36(1). https://doi.org/10.7771/2832-9414.1815

Dizon, G., & Gayed, J. M. (2021). Examining the impact of Grammarly on the quality of mobile L2 writing. JALT CALL Journal, 17(2), 74–92.

Dodigovic, M., & Tovmasyan, A. (2021). Automated writing evaluation: The accuracy of Grammarly’s feedback on form. International Journal of TESOL Studies, 3(2), 71–88.

Ellis, R. (2006). Current issues in the teaching of grammar: An SLA perspective. TESOL Quarterly, 40(1), 83-107.

Ellis, R. (2009). Corrective Feedback and teacher development. L2 Journal: An Open Access Refereed Journal for World Language Educators, 1(1).

Fan, N., & Ma, Y. (2022). The Effects of Automated Writing Evaluation (AWE) feedback on Students’ English writing quality: A Systematic literature review. Language Teaching Research Quarterly, 28, 53–73. https://doi.org/10.32038/ltrq.2022.28.03

Ferris, D. (2002). Treatment of error in second language student writing. University of Michigan Press.

Ferris, D. (2010). Second language writing research and written corrective feedback in SLA. Studies in Second Language Acquisition, 32(2), 181-201.

Fitria, T.N., 2021. Articial intelligence (AI) in education: using AI tools for teaching and learning process. Proceeding Seminar Nasional & Call for Paper STIE AAS 4 (1), 134–147. https://prosiding.stie-aas.ac.id/index.php/prosenas/article/view/106/

Ghufron, M. A., & Rosyida, F. (2018). The role of Grammarly in assessing English as a Foreign Language (EFL) writing. Lingua Cultura, 12(4), 395–403.

Grammarly. (2020). The Grammarly Editor: New look, more suggestions, better writing. https://www.grammarly.com/blog/product/better-writing-with-grammarly/

Grammarly. (2024a). AI-powered paraphrasing tool. https://www.grammarly.com/paraphrasing-tool

Grammarly. (2024b). Releases. https://www.grammarly.com/releases

Hall, N. (2009). Punctuation and grammar. In R. Beard, D. Myhill, J. Riley, & M. Nystrand

(Eds.), The SAGE handbook of writing development (pp. 271-283). SAGE Publications.

Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.

Inpanich, P., Chaya, P., Wongsittikan, N., & D. Evans, M. (2023). A Comparison of Grammatical Errors Made by Thai EFL Undergraduate Students in Different Types of Essays. Manutsat Paritat: Journal of Humanities, 45(2), 165–191.

Jarvis, S., & Pavlenko, A. (2008). Crosslinguistic influence in language and cognition. Routledge.

Koltovskaia, S. (2020). Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing, 44, 100450. https://doi.org/10.1016/j.asw.2020.100450

Miranty, D., Widiati, U., Cahyono, B. Y., & Tengku Sharif, T. I. S. (2023). Automated writing evaluation tools for Indonesian undergraduate English as a foreign language students’ writing. International Journal of Evaluation and Research in Education

(IJERE, 12(3), 1705–1715. https://doi.org/10.11591/ijere.v12i3.24958

Murphy, R. (2015). Essential grammar in use (4th ed.). Cambridge University Press.

Murphy, R. (2019). English grammar in use (5th ed.). Cambridge University Press.

Ng, V., Ling, Y. L., & Kho, M. G. W. (2020). Error analysis of the written English essays of diploma students in Kuching, Sarawak. Paper presented at the National Research Innovation Conference (NRICon 2020), Kuching, Sarawak, Malaysia.

Nguyen, T. H. (2020). Common grammatical errors in English writing: A case study with second-year students of information technology at HAUI. Can Tho University Journal of Science, 12(1), 37–44. https://doi.org/10.22144/ctu.jen.2020.005

Nova, M. (2018). Utilizing Grammarly in evaluating academic writing: A narrative research on EFL students’ experience. Premise: Journal of English Education and Applied Linguistics, 7(1), 80–96.

Nurmayanti, N., & Suryadi. (2023). The effectiveness of using Quillbot in improving writing for students of the English education study program. Jurnal Teknologi Pendidikan: Jurnal Penelitian dan Pengembangan Pembelajaran, 8(1), 32–40. https://ejournal.

undikma.ac.id/index.php/jtp/index

Petchprasert, A. (2021). Utilizing an automated tool analysis to evaluate EFL students’ writing performances. Asian-Pacific Journal of Second and Foreign Language Education, 6(1), 1–16. https://doi.org/10.1186/s40862-020-00107-w

Piaget, J. (1952). The origins of intelligence in children. W. W. Norton. Qassemzadeh, A., & Soleimani, H. (2016). The impact of feedback provision by Grammarly software and teachers on learning passive structures by Iranian EFL learners. Theory and Practice in Language Studies, 6(9), 1884–1894. https://doi.org/10.17507/tpls.0609.23

Ridha, N. (2012). The effect of EFL learners' mother tongue on their writings in English: An error analysis study. Journal of the College of Arts, University of Basrah, 60, 22–45.

Sawalmeh, M.H.M. (2013). Error analysis of written English essays: The case of students of the Preparatory Year Program in Saudi Arabia. English for Specific Purposes World, 14(40), 1-17

Savage, K. L., Bitterlin, G., & Price, D. (2010). Grammar matters: Teaching Grammar in Adult ESL Programs. Cambridge University Press.

Singh, C. K. S., Singh, A. K. J., Razak, N. Q. A., & Ravinthar, T. (2017). Grammar errors made by ESL tertiary students in writing. English Language Teaching, 10(5), 16–27.

Solfiyatuzzahro, S., Santihastuti, A., & Erfan, E. (2019). Grammatical Accuracy using Teacher’s Written Corrective Feedback. Eralingua Jurnal Pendidikan Bahasa Asing Dan Sastra, 3(2). https://doi.org/10.26858/eralingua.v3i2.9051

Stake, R. E. (1995). The art of case study research. Sage Publications.

Thi, N. K., Vo, D. V., & Nikolov, M. (2023). Investigating syntactic complexity and

language-related error patterns in EFL students' writing: Corpus-based and epistemic network analyses. Language Learning in Higher Education, 13(1), 127–151. https://doi.org/10.1515/cercles-2023-2009

Thornbury, S. (1999). How to teach grammar. Pearson Education.

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.

Wenden, A. (1998). Metacognitive knowledge and language learning. Applied Linguistics, 19(4), 515-537.

Yin, R. K. (2018). Case study research and applications: Design and methods (6th ed.). Sage Publications.

Zimmerman, B.J. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41(2), 64-70. https://doi.org/10.1207/s15430421tip4102_2