Beyond Linguistic Accuracy: A Culturally Embedded Framework for Assessing Cross-Cultural Speaking Ability through Performance and Reflective Inquiry

Chotiwan Yaemkhayai; Ruedeerath  Chusanachoti; Theerawat  Theerapojjanee

doi:10.66947/pasaa.v74i1.2819

Authors

Chotiwan Yaemkhayai Panyapiwat institute of management
Ruedeerath Chusanachoti Faculty of Education, Chulalongkorn University, Thailand
Theerawat Theerapojjanee Faculty of Arts, Chulalongkorn University, Thailand

DOI:

https://doi.org/10.66947/pasaa.v74i1.2819

Keywords:

Cross-Cultural Speaking Ability, language assessment, Video-Stimulated Recall Interviews

Abstract

This study presents a preliminary assessment framework for Cross-Cultural Speaking Ability in foreign language education by integrating performance-based role-play assessment and Video-Stimulated Recall Interviews (VSRI). Drawing on theories of cross-cultural communication, communicative competence, and applied linguistics, the framework conceptualizes Cross-Cultural Speaking Ability as consisting of three related dimensions: (1) linguistic accuracy, (2) strategic management of cross-cultural relationships, and (3) culturally appropriate use of verbal and non-verbal communication. A mixed-methods design was employed to support the development and preliminary examination of the framework. The participants comprised 18 undergraduates enrolled on
intermediate-level foreign language communication courses. Research instruments included role-play speaking tasks, cross-cultural
communication scenarios, an analytic assessment rubric, and VSRI interview protocols. The framework and instruments were developed through documentary analysis and expert review conducted by five fieldrelated specialists. Participants completed role-play speaking tasks based on cross-cultural communication scenarios, followed by VSRI interviews to explore learners’ reflective interpretations and cross-cultural reasoning underlying their observable communicative behaviors. The performances
and reflective data were evaluated using an analytic rubric by two trained assessors. Quantitative findings from rubric scoring and inter-rater agreement were integrated with qualitative thematic findings from VSRI data. The findings indicate that the framework may be useful for examining linguistic and cross-cultural speaking performance aspects. The instruments demonstrated acceptable content validity (IOC = 0.60–1.00), and the assessment results showed high inter-rater reliability (r =0.978). Although the findings provide initial support for the framework, further research involving larger, more diverse learner groups is needed to examine its applicability across contexts.

Author Biographies

Ruedeerath Chusanachoti, Faculty of Education, Chulalongkorn University, Thailand

Ruedeerath Chusanachoti: An assistant professor at Faculty of Education, Chulalongkorn University, Thailand. She received her Ph.D. in Curriculum, Instruction, and Educational Policy from Michigan State University, USA. Her areas of expertise include language curriculum and instruction development, as well as second/foreign language teaching and learning.

Theerawat Theerapojjanee, Faculty of Arts, Chulalongkorn University, Thailand

Theerawat Theerapojjanee: An assistant professor at Faculty of Arts, Chulalongkorn University, Thailand. He received his Ph.D. in Linguistics and Applied Linguistics from Peking University. His areas of expertise include Chinese Syntax and Semantics, Teaching Chinese as a Foreign Language, and Thai as a Foreign Language.

References

Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice. Oxford University Press.

Brown, H. D., & Abeywickrama, P. (2019). Language assessment: Principles and classroom practices (3rd ed.). Pearson Education.

Byram, M. (1997). Teaching and assessing intercultural communicative competence. Multilingual Matters.

Cambridge English. (2014). Cambridge English: First speaking (from 2015): Sample test with examiner's comments. University Cambridge. https://www.cambridgeenglish.org/images/173977-cambridge-english-first-speaking-test-examiner-comments.pdf

Carbaugh, D. (2016). The handbook of communication in cross-cultural perspective. Routledge.

Celce-Murcia, M. (2007). Rethinking the role of communicative competence in language teaching. In E. A. Soler & M. P. Safont Jordà (Eds.), Intercultural language use and language learning (pp. 41–57). Springer. https://doi.org/10.1007/978-1-4020-5639-0_3

Fongsataporn, K., Chusanachoti, R., & Chirasombutti, V. (2024). Effects of instructional model based on coordinated management of meaning and reader’s theatre on Japanese oral communication ability. PASAA, 68(1), 375–407. DOI: 10.58837/CHULA.PASAA.68.1.11

Fulcher, G. (2015). Re-examining Language Testing: A Philosophical and Social Inquiry (1st ed.). Routledge. https://doi.org/10.4324/9781315695518

Goh, C. M., & Burns, A. (2012). Teaching speaking: A holistic approach. Cambridge University Press.

Harwati, L. N., & Sathian, M. R. (2024). Mitigating pedagogical challenges through culturebased approach: Javanese language learning in rural Yogyakarta, Indonesia. Indonesian Journal of Applied Linguistics, 14(2), 262–272. https://doi.org/10.17509/ijal.v14i2.

Hurn, B., & Tomalin, B. (2013). Cross-cultural communication: Theory and practice. Palgrave Macmillan.

Kecskes, I. (2014). Intercultural pragmatics. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199892655.001.0001

Littlewood, W. (2014). Communication-oriented language teaching: Where are we now? Where do we go from here? Language Teaching, 47(3), 349-362. https://doi.org/10.1017/S0261444812000134

Lyle, J. (2003). Stimulated recall: A report on its use in naturalistic research. British Educational Research Journal, 29(6), 861–878. https://doi.org/10.1080/0141192032000137349

Matyakhan, T., Chusanachoti, R., & Santos, J. A. L. (2024). “I think I speak without hesitation”: Learning-Oriented Assessment to Enhance English Oral Communication of Thai Pre-Service Teachers. REFLections, 31(3), 966–989. https://doi.org/10.61508/refl.v31i3.275912

McNamara, T., & Roever, C. (2006). Language testing: The social dimension. Blackwell.

Mushtaq, M., Aziz, A., Ramzan, M., & Jahan, K. (2021). Identifying the rules for the evaluation of English language speaking skill: A library-based review. Ilkogretim Online -Elementary Education Online, 20(5), 1089–1099.

Nguyen, N. T., McFadden, A., Tangen, D. J., & Beutel, D. A. (2013, December 1–5). Videostimulated recall interviews in qualitative research [Conference paper]. Australian Association for Research in Education (AARE) Annual Conference, Adelaide, Australia.

Savignon, S. J. (2002). Communicative language teaching: Linguistic theory and classroom practice. In S. J. Savignon (Ed.), Interpreting communicative language teaching: Contexts and concerns in teacher education (pp. 1–27). Yale University Press.

Ting-Toomey, S. (2005). The matrix of face: An updated face-negotiation theory. In W. B. Gudykunst (Ed.), Theorizing about intercultural communication (pp. 71–92). Sage Publications.

Young, R. F. (2011). Interactional competence in language learning, teaching, and testing. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning (Vol. 2, pp. 426–443). Routledge.