|
|
《传承性与创新性：基于证据的六级、雅思、托福考试效度对比研究》引介
|
Abstract:
本文系国家社科基金重点项目结题成果《传承性与创新性：基于证据的六级、雅思、托福考试效度对比研究》(14AYY010)的引介，由引言和结语构成，是项目研究的整体设计、具体实施与完成的全局性概览。该研究以“社会–认知效度验证框架”为理论基础，使用了效度研究一些传统的研究方法，如问卷调查、有声思维等，也尝试了一些创新性的研究方法，如数据挖掘、眼动追踪等。该研究成果不仅丰富了六级、雅思、托福三项考试的效度证据，为教育、人事部门及广大利益相关者提供入学、就业、人才流动等决策依据，而且为其他语言测试效度对比研究提供了思路与方法上的借鉴。
This paper is an introduction to the accomplishment of the key research project of the National Philosophy and Social Sciences Foundation of China Continuity and Innovation: An Evidence-Based Comparative Validation Study of CET-6, ILETS and TOEFL iBT (Fund No. 14AYY010). It consists of the introduction and the conclusion parts, which provides an overall glimpse of the whole research design and its implementation processes. This study, utilizing the evidence-based “Socio-cognitive” Framework for test validation as its theoretical basis, employs some traditional validation study methodologies, such as questionnaire survey and verbal protocol, and explores some innovative technologies such as data-mining and eye-tracking. The research findings not only enrich the validation evidence of the three tests, CET-6, IELTS and TOEFL iBT, for decision-making references for study, employment and migration for education, human resources and other relevant stakeholders, but also offer references for comparative validation studies of other language assessments in methodologies and ways of thinking.
| [1] | APA, AERA, and NCME (2014) Standards for Educational and Psychological Testing. Revised Version, American Educational Research Association, Washington DC. |
| [2] | Kelly, T.L. (1927) Interpretation of Educational Measurements. New World Book Company, New York. |
| [3] | Kunnan, A.J. (1998) An Introduction to Structural Equation Modelling for Language Assessment Research. Language Testing, 15, 295-332. https://doi.org/10.1177/026553229801500302 |
| [4] | Lado, R. (1961) Language Testing. McGraw-Hill, New York. |
| [5] | APA, AERA, and NCME (1966) Standards for Educational and Psychological Tests and Manuals. Ameri-can Psychological Association, Washington DC. |
| [6] | Messick, S. (1989) Validity. In: Linn, R.L., Ed., Educational Measurement, 3rd Edition, Macmillan, New York, 13-103. |
| [7] | Alderson, J.C. and Banerjee, J. (2001) Language Testing and Assessment. Language Teaching, 35, 79-113.
https://doi.org/10.1017/S0261444800014464 |
| [8] | Davies, A., Hamp-Lyons, L. and Kemp, C. (2003) Whose Norms? International Proficiency Tests in English. World Englishes, 22, 571-584. https://doi.org/10.1111/j.1467-971X.2003.00324.x |
| [9] | Bachman, L.F. (1990) Fundamental Considerations in Language Testing. Oxford University Press, Oxford. |
| [10] | Bachman, L.F., Davidson, F., Ryan, K. and Choi, I. (1995) An Investigation into the Comparability of Two Tests of English as a Foreign Language: The Cambridge-TOEFL Comparability Study. Cambridge University Press, Cambridge. |
| [11] | Bachman, L. and Palmer, A. (1996) Language Testing in Practice. Oxford University, Oxford. |
| [12] | 韩宝成, 罗凯洲. 语言测试效度及其验证模式的嬗变[J]. 外语教学与研究, 2013, 45(2): 411-425. |
| [13] | Kane, M.T. (1992) An Argument-Based Approach to Validity. Phycological Bulletin, 112, 537-535.
https://doi.org/10.1037/0033-2909.112.3.527 |
| [14] | Chapelle, C.A., Enright, M.K. and Jamieson, J.M. (2008) Building a Validity Argument for the Test of English as a Foreign Language. Routledge, New York. |
| [15] | Bachman, L. (2005) Building and Supporting a Case for Test Use. Language Assessment Quarterly, 2, 1-34.
https://doi.org/10.1207/s15434311laq0201_1 |
| [16] | Bachman, L. and Palmer, A. (2010) Language Assessment in Practice. Oxford University Press, Oxford. |
| [17] | Weir, C. (2005) Language Testing and Validation. Prentice Hall, London. https://doi.org/10.1057/9780230514577 |
| [18] | Cheung, K.Y.F. and Emery, J. (2017) Applying the So-cio-Cognitive Framework to the Bio-Medical Admissions Test (BMAT). Cambridge University Press, Cam-bridge. |
| [19] | Papp, S. and Rixon, S. (2018) Examining Young Learners: Research and Practice in Assessing the English of School-Age Learners. Cambridge University Press, Cambridge. |
| [20] | Shaw, S.D. and Weir, C.J. (2007) Examining Writing: Research and Practice in Assessing Second Language Writing. Cambridge University Press, Cambridge. |
| [21] | 李清华. 语言测试之效度理论发展五十年[J]. 现代外语, 2006, 29(1): 214-217. |
| [22] | Jin, Y. and Yang, H. (2006). The English Proficiency of College and University Students in China: As Reflected in the CET. Language, Culture & Curriculum, 19, 21-36. https://doi.org/10.1080/07908310608668752 |
| [23] | 杨惠中, Weir, C. 大学英语四、六级考试效度研究[M]. 上海: 上海外语教育出版社, 1998. |
| [24] | He, L.Z. and Dai, Y. (2006) A Corpus-Based Investigation into the Validity of the CET-SET Group Discussion. Language Testing, 23, 370-401. https://doi.org/10.1191/0265532206lt333oa |
| [25] | 贾国栋. 大学英语口语测试的预期反拨效应——以全国大学英语四、六级口语测试为例[J]. 外语测试与教学, 2016(4): 1-9. |
| [26] | 金艳, 吴江. 以“内省法”检验CET阅读理解测试的效度[J]. 外语界, 1998(2): 47-52. |
| [27] | 金艳. 计算机化语言测试的效度研究——浅析计算机能力与测试构念的关系[J]. 外语电化教学, 2012(1): 11-15. |
| [28] | 王跃武, 朱正才, 杨惠中. 作文网上评分信度的多面Rasch测量分析[J]. 外语界, 2006, 27(1): 69-76. |
| [29] | 朱正才. 大学英语四、六级考试分数等值研究——一个基于铆题和两参数IRT模型的解决方案[J]. 心理学报, 2005, 37(2): 280-284. |
| [30] | 辜向东. 正面的还是负面的——大学英语四六级考试反拨效应实证研究[M]. 重庆: 重庆大学出版社, 2007. |
| [31] | 辜向东. 大学英语四六级考试反拨效应历时研究(上、下卷)[M]. 成都: 四川大学出版社, 2013. |
| [32] | 辜向东, 张正川, 刘晓华. 改革后的CET对学生课外英语学习过程的反拨效应实证研究——基于学生的学习日志[J]. 解放军外国语学院学报, 2014, 37(5): 44-164. |
| [33] | Davies, A. (2008) Assessing Academic English: Testing English Proficiency 1950-1989—The IELTS Solution. Cambridge University Press, Cambridge. |
| [34] | Taylor, L. and Weir, C. (2012) IELTS Collected Papers 2: Research in Reading and Listening Assessment. Cambridge University Press, Cambridge. |
| [35] | Annie, B. (2003) An Examination of the Rating Process in the Revised IELTS Speaking Test. IELTS Research Report, Vol. 6, 11-30. |
| [36] | Yates, L., Zielinski, B. and Pryor, E. (2011) The Assessment of Pronunciation and the New IELTS Pro-nunciation Scale. IELTS Research Report, Vol. 12, 1-44. |
| [37] | Merrifield, G. (2012) An Impact Study into the Use of IELTS by Professional Associations in the United Kingdom, Canada, Australia and New Zealand. IELTS Research Report, Vol. 13, 1-53. |
| [38] | Read, J. and Hayes, B. (2003) The Impact of IELTS on Preparation for Academic Study in New Zealand. IELTS Research Report, Vol. 4, 153-191. |
| [39] | Biber, D. and Gray, B. (2013) Discourse Characteristics of Writing and Speaking Task Types on the TOEFL iBT Test: A Lexico-Grammatical Analysis. TOEFL iBT Research Report Series, 2013, i-128. |
| [40] | Stricker, L.J. and Attali, Y. (2010) Test Takers’ Attitudes about the TOEFL iBT. TOEFL iBT Research Report Series, 2010, i-16. |
| [41] | Powers, D.E., Roever, C., Huff, K.L. and Trapani, C.S. (2003) Validating Language? Courseware Scores against Faculty Ratings and Student Self-Assessments. TOEFL iBT Research Report Series, 2003, i-25. |
| [42] | Sawaki, Y., Lawrence, J., Stricker, H.O. and Andreas, H.O. (2009) Factor Structure of the TOEFL Internet-Based Test. Language Testing, 26, 5-30. https://doi.org/10.1177/0265532208097335 |
| [43] | Wolfe, E.W. and Manalo, J.R. (2005) An Investigation of the Impact of Composition Medium on the Quality of TOEFL Writing Scores. TOEFL iBT Research Report Series, 2005, i-58. |
| [44] | Rahimi, F., Bagheri, M.S., Sadighi, F. and Yarmoh, A. (2014) Using an Argument-Based Approach to En-sure Fairness of High-Stakes Tests’ Score-Based Consequence. Procedia—Social and Behavioral Sciences, 98, 1461-1468.
https://doi.org/10.1016/j.sbspro.2014.03.566 |
| [45] | Weigle, S.C. (2011) Validation of Automated Scores of TOEFL iBT Tasks against Non-Test Indicators of Writing Ability. TOEFL iBT Research Report Series, 2011, i-63. |
| [46] | Xi, X., Higgins, D., Zechner, K. and Williamson, D. (2012) A Comparison of Two Scoring Methods for an Automated Speech Scoring System. Language Testing, 29, 371-394. https://doi.org/10.1177/0265532211425673 |
| [47] | ETS (2011) Reliability and Comparability of TOEFL iBT Scores. TOEFL iBT Research Report Series, 2011, i-12. |
| [48] | Zhang, Y. (2008) Repeater Analyses for the TOEFL iBT Test. ETS Research Memorandum, i-12. |
| [49] | Jamieson, J. and Poonpon, K. (2013) Developing Analytic Rating Guides for TOEFL iBT Integrated Speaking Tasks. TOEFL iBT Research Report Series, 2011, i-93. |
| [50] | Richard, J., Tannenbaum, R.J. and Wylie, E.C. (2008) Linking English-Language Test Scores onto the Common European Framework of Reference: An Application of Standard-Setting Methodology. TOEFL iBT Research Report Series, 2011, i-93. |
| [51] | Taylor, L. (2004) Issues of Test Comparability. Research Notes 15, 2-12. |
| [52] | 仇茵晴, 张艳莉. 新老大学英语四级和雅思听力试题的对比研究——改革后新四级成效初探[J]. 外语测试与教学, 2011(3): 29-38. |
| [53] | 金艳, 张晓艺. 技能综合对语言测试构念效度的影响——培生英语考试与大学英语六级网考的对比研究[J]. 外语电化教学, 2013(6): 3-10. |
| [54] | 李鑫, 修旭东. 雅思和我国高考英语阅读测试题型的对比[J]. 解放军外国语学院学报, 2009, 32(5): 60-71. |
| [55] | 王丽. 三种大规模标准化英语考试听力测试部分之比较——一项基于语篇、任务、说话人相关因素的研究[J]. 外语电化教学, 2007(2): 67-72. |
| [56] | Council of Europe (2001) Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge University Press, Cambridge. |
| [57] | 教育部考试中心. 中国英语能力等级量表[Z]. 北京: 高等教育出版社, 2019. |
