簡易檢索 / 詳目顯示

研究生: 陳德海
Chen, Te-hai
論文名稱: 整體式與分項式翻譯測驗評分之等化: 多面向羅許分析模式
Score Equivalence between Holistic and Analytic Rating Rubrics in Chinese-English Translation Tests: A Many-Facet Rasch Analysis
指導教授: 曾文鐽
Tseng, Wen-Ta
學位類別: 碩士
Master
系所名稱: 英語學系
Department of English
論文出版年: 2016
畢業學年度: 104
語文別: 英文
論文頁數: 58
中文關鍵詞: 翻譯測驗多面向羅許分析模式評分規準
英文關鍵詞: translation, assessment, MFRM, rating rubric
DOI URL: https://doi.org/10.6345/NTNU202204013
論文種類: 學術論文
相關次數: 點閱:140下載:16
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 英文在台灣的教學環境中是主要的核心科目之一,而且在普通高級中學英文課程綱要中,培養學生中英翻譯的能力也是其主要的目標。在重要的升學入學考試如大學學科能力測驗與指定科目考試中,中譯英常常是測驗題型的其中一種。翻譯評量的給分本質上是相當複雜的,採用何種評分規準以及評分者的合適與否皆對於判斷受試者的表現產生重大的影響。因此,本研究旨在檢視現行翻譯測驗題型中分項式給分的效度,並為翻譯測驗的效用提供實證。 本研究亦提出整體式評分規準為現行分項式給分之替代方案,並檢視兩者能否平等地評量考生於翻譯測驗中的表現以及評分者的專業程度與評分經驗是否在某些程度上影響他們評比受試者們的作答。本研究採用多面向羅許模式 (Many-Facets Rasch Measurement Model)來分析實際資料,期盼能透過實例來檢視翻譯測驗的分數與評分者專業程度及評分經驗之間的關係。除此之外,整體式與分項式評分規準應用於翻譯測驗的優點與缺點將會被仔細的檢視與探討。

    English has been one of the core subjects in the EFL context of Taiwan, and cultivating students’ translation skills is a major objective in the national senior high school curriculum. In high-stakes examination like General Scholastic Ability Test and Advanced Subjects Test, translation has always been part of the two tests. The nature of grading performance assessment, such as translation, is quite convoluted. The adaptation of rating scales and recruitment of adequate raters become crucial in judging test takers’ performance. Hence, this research aims to inspect the validity of the current analytic rating rubrics of translation, and proffer empirical evidence for the utility of translation items. The author also proposes the holistic rating rubrics as an alternative to check the extent to which the translation performance can be equally assessed and to examine the extent to which rater’s expertise and rating experience correlates with their verdict on test taker’s performance. In the research, the Many-Faceted Rasch measurement model (MFRM) will be taken as the approach to analyzing the empirical data. The expectation result of this research is to provide empirical evidence that the translation scores may have significant relationships with rater’s expertise and rating experience. Also, the pros and cons of holistic and analytic rating rubrics in relation to translation test will be critically examined and discussed.

    TABLE OF CONTENTS 摘要 i ABSTRACT ii ACKNOWLEDGEMENT iii TABLE OF CONTENTS iv LIST OF TABLES v LIST OF FIGURES vi CHAPTER ONE INTRODUCTION 1 Background and Purpose of the Study 1 Significance of the study and Research Aims 2 Research Questions: 2 Organization of the Thesis 3 CHAPTER TWO LITERATURE REVIEW 4 From Recognition Assessment to Performance Assessment 4 Translation 5 The Merits of Translation 6 Criticisms towards Translation 7 Translation in Language Teaching 9 Holistic and Analytic Scoring 10 Novice and Expert raters 12 Many-Faceted Rasch Measurement Model (MFRM) 15 CHAPTER THREE RESEARCH METHODOLOGY 22 Participants 22 Instrument 23 Scoring and Coding 25 Procedure 26 Data Analysis 26 Expected Result and Suggestion 27 CHAPTER FOUR RESULTS AND DISCUSSIONS 28 Overview of the study 28 Major Findings 28 CHAPTER FIVE CONCLUSION 45 Summary of Major Findings 45 Limitations of the Study 46 Directions for Future Research 47 REFERENCES 48

    Alexander, P. A., & Judy, J. E. (1988). The interaction of domain-specific and strategic knowledge in academic performance. Review of Educational research, 58(4), 375-404.
    Ali Shamim (2011). Improving Access to Second Language Vocabulary and Comprehension Using Bilingual Dictionaries.
    Allford, D. (1999). Translation in the communicative classroom. In N. Pachler (Ed.), Teaching modern languages at advanced level (pp. 230–250). London:Routledge.
    Atkinson, D. (1987). The mother tongue in the classroom: A neglected resource?. ELT journal, 41(4), 241-247.
    Baynham, M. (1983). Mother tongue materials and second language literacy. ELT Journal, 37(4), 312-318.
    Bachman, L. F. (2004). Statistical analyses for language assessment. Ernst Klett Sprachen.
    Bell, R.T. (1991): Translation and Translating, London: Longman.
    Benner, P. (1982). From Novice to Expert. American Journal of Nursing, 402-407.
    Bereiter, C., &Scardamalia, M. (1993). Surpassing ourselves. An inquiry into the nature and implications of expertise. Chicago: Open Court.
    Berliner, D. (1994). Teacher expertise. Teaching and learning in the secondary school, 107-113.
    Bond, T.G. & Fox C.M. (2007). Applying the Rasch Model: Fundamental measurement in the human sciences. (2nd .)Lawrence Erlbaum.
    Bond, T. G., & Fox, C. M. (2013). Applying the Rasch model: Fundamental measurement in the human sciences.Psychology Press.
    Brentari, E., &Golia, S. (2007). Unidimensionality in the Rasch model: how to detect
    and interpret. Statistica, 67(3), 253-261.
    Brooks-Lewis, K.A. (2009). Adult learners’ perceptions of the incorporation of their L1 in foreign language teaching and learning. Applied Linguistics, 30, 216–235.
    Buck, G. (1992). Translation as a language testing procedure: does it work? Language Testing, 9,123–148.
    Butzkamm, W., & Caldwell, J.A.W. (2009). The bilingual reform: A paradigm shift in foreign language teaching. Tübingen: NarrStudienbucher.
    Carreres, A. (2006, December). Strange bedfellows: Translation and languageteaching. The teaching of translation into L2 in modern languages degrees: Uses and limitations.In Sixth Symposium on Translation, Terminology and Interpretation in Cuba and Canada.Canadian Translators, Terminologists and Interpreters Council. Retrieved from: http://www.cttic.org/ACTI/2006/papers/Carreres.pdf
    Carreres, A., & Noriega-Sanchez, M. (2011). Translation in language teaching: Insights from professionaltranslator training. The Language Learning Journal, 39, 281–297.
    Chellappan, K. (1991). The role of translation in learning English as a second language. International Journal of Translation, 3, 61-72.
    Chi, M. T., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of
    physics problems by experts and novices. Cognitive science, 5(2), 121-152.
    Chi, M.T.H., Glaser, R., & Rees, E. (1982). Expertise in problem solving.In R. Sternberg (Ed.), Advances inthe psychology of human intelligence, 1, 7-75. Hillsdale, NJ: Erlbaum.
    Chi, M. T. H. (2006). Two approaches to the study of experts’ characteristics. In K. A. Ericsson,N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expertperformance (pp. 21–30). Cambridge: Cambridge University Press.
    Cook, G. (2009). Use of translation in language teaching. In M. Baker, & G. Saldanha (Eds.), Routledge encyclopedia of translation studies (2nd ed., pp. 112–115). New York/London:Routledge.
    Cook, G. (2010). Translation in language teaching: An argument for reassessment. Oxford: Oxford University Press.
    Corder, S. P., & Corder, S. P. (1981). Error analysis and interlanguage (Vol. 112).Oxford: Oxford University Press.
    Cumming, A. (1990). Expertise in evaluating second language compositions. Language Testing, 7(1), 31-51.
    Douglas, S. P., & Craig, C. S. (2007). Collaborative and iterative translation: An alternativeapproach to back translation. Journal of International Marketing, 15, 30–43.
    Dreyfus, H.L., & Dreyfus, S.E. (1980). A five-stage model of the mental activities involved in directed skill acquisition. Unpublished reportsupported by the Air Force Office of Scientific Research (AFSC), USAF, University of California at Berkeley.
    Ellis, R. (1985). Understanding second language acquisition. Oxford: Oxford University Press.
    Engelhard Jr, G. (1992). The measurement of writing ability with a many-faceted Rasch model. Applied Measurement in Education, 5(3), 171-191.
    Farquhar, S., & Fitzsimons, P. (2011). Lost in translation: The power of language. Educational Philosophy and Theory, 43(6), 652-662.
    Farrell, T. S. C. (2013). Reflecting on ESL teacher expertise: A case study.
    System,41(4), 1070-1082.
    Feltovich, P. J., Prietula, M. J., & Ericsson, K. A. (2006). Studies of Expertise from
    Psychological Perspectives.
    Glaser, R., Chi, M. T., & Farr, M. J. (Eds.). (1988). The nature of expertise. Lawrence Erlbaum Associates.
    Gonzalez-Davies, M. (2004). Multiple voices in the translation classroom: Activities, tasks andprojects. Amsterdam: John Benjamins.
    Hobus, P. P., Schmidt, H. G., Boshuizen, H. P., & Patel, V. L. (1987). Contextual factors in the activation offirst diagnosis hypotheses: Expert-novice differences. Medical Education, 21, 471–476.
    Howatt, A.P.R. 1984. (1stedn.) A history of English language teaching. Oxford: Oxford University Press.
    Hughes, A. (2007). Testing for language teachers. Cambridge: Cambridge University Press.
    Kelly, N., & Bruen, J. (2015). Translation as a pedagogical tool in the foreign language classroom: A qualitative study of attitudes and behaviours. Language Teaching Research, 19(2), 150-168.
    Kim, E. Y. (2011). Using translation exercises in the communicative EFL writing classroom. ELT journal, 65(2), 154-160.
    Klapper, J. (2006). Understanding and developing goodpractice teaching languages in higher education. London:CILT.
    Kobayashi, H., & Rinnert, C. (1992). Effects of First Language on Second Language Writing: Translation versus Direct Composition*. Language Learning, 42(2), 183-209.
    Kuiken, F., & Vedder, I. (2014). Raters’ decisions, rating procedures and rating scales. Language testing, 31(3), 279-284.
    Kuiken, F., & Vedder, I. (2014). Rating written performance: What do raters do and why?. Language Testing, 31(3), 329-348.
    Lado, R. (1964). Language testing: The construction and use of foreign language tests; a teacher's book. New York: McGraw-Hill.
    Laufer, B., &Girsai, N. (2008). Form-focused instruction in second language vocabulary learning:A case for contrastive analysis and translation. Applied Linguistics, 29, 694–716.
    Lems, K., Miller, L., & Soro, T. (2010). Teaching reading to English language learners: Insightsfrom linguistics. New York: Guilford Publications.
    Liamputtong, P. (2010). Performing qualitative cross-cultural research. Cambridge:
    Cambridge University Press.
    Lim, G. S. (2011). The development and maintenance of rating quality in performance writing assessment: A longitudinal study of new and experienced raters. Language Testing, 0265532211406422.
    Linacre, J. M., & Wright, B. D. (2000).Winsteps. URL: http://www. winsteps. com/index. htm [accessed 2013-06-27][WebCite Cache].
    Linacre, J. M. (2002). What do infit and outfit, mean-square and standardized mean. Rasch Measurement Transactions, 16(2), 878.
    MacNamara, T. F. (1996). Measuring second language performance. London: Longman.
    Malmkjær, K. (Ed.) (1998). Translation and language teaching. Manchester: St. Jerome.
    Martínez, R., Orellana, M., Pacheco, M. (2008). Found in translation: Connecting translating experiences to academicwriting. Language Arts, 85(6), 421–431
    McNamara, T. F. (1996). Measuring second language performance. Addison Wesley
    Longman.
    McQuillan, J., & Tse, L. (1995). Child language brokering in linguistic minority communities: Effects on culturalinteraction, cognition, and literacy. Language and Education, 9(3), 195–215.
    Mehrens, W.A. (1992). Using performance assessment for accountability purposes. Educational Measurement: Issues and Practice, 11(1), 3-9, 20.
    Matthews-Breský, R.J.H. 1972. “Translation as a testing device.” English Language Teaching Journal 27 (I): 58-65
    Myford, C. M., & Wolfe, E. W. (2004). Detecting and measuring rater effects using many-facet Rasch measurement: Part 2. Journal of Applied Measurement, 5(2), 189-227.
    Naznean, A. (2014). THE IMPORTANCE OF THE STUDENTS'NATIVE LANGUAGE IN TEACHING TRANSLATION. Studia Universitatis Petru Maior. Philologia, (17), 76.
    Newmark, P. (1991). About translation (Vol. 74). Multilingual matters.
    Nord, C. (1997). Translation as a purposeful activity. Functionalist approaches explained. Manchester: St. Jerome Publishing.
    Nord, C. (2005). Text analysis in translation: Theory, methodology, and didactic application of a model for translation-oriented text analysis (No. 94). Rodopi.
    Oxford, R.L. (1990). Language Learning Strategies: What every Teacher Should know. NewYork: Newbury House.
    PACTE (2000): “Acquiring Translation Competence: Hypotheses and Methodological Problems ofa Research Project,” in A. Beeby, D. Ensinger, M. Presas(eds.).Investigating Translation. Amsterdam/Philadelphia: John Benjamins, 99-106.
    Prince, P. (1996). Second language vocabulary learning: The role of context versus translations as a function of proficiency. The modern language journal,80(4),478-493.
    Ross, K. G., Shafer, J. L., & Klein, G. (2006). Professional judgments and ‘‘naturalistic decision making’’. InK. A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge handbook ofexpertise and expert performance (pp. 403–420). Cambridge: Cambridge University Press.
    Schaefer, E. (2008). Rater bias patterns in an EFL writing assessment. Language Testing, 25(4), 465-493.
    Schäffner, C. (1998). Qualification for professional translators. Translation in language teaching versus teaching translation. In K. Malmkjær (Ed.), Translation and language teaching: Language teaching and translation. (pp. 117–133). Manchester: St. Jerome Publishing.
    Schjoldager, A. (2004). Are L2 learners more prone to err when they translate? In K. Malmkjær,& A.E. McEnery (Eds.), Translation in undergraduate degree programmes(pp. 127–14).Amsterdam : John Benjamins.
    Scribner, S. (1985). Thinking in action: Some characteristics of practical thought. Practical intelligence: Nature and origins of competence in the everyday world, 13, 60.
    Shohamy, E., Gordon, C. M., & Kraemer, R. (1992). The effect of raters' background and training on the reliability of direct writing tests. The Modern Language Journal, 76(1), 27-33.
    Short, D.J. (1993). Assessing integrated language and content instruction. TESOL Quarterly, 27(4), 627-656.
    Smith, R., Schumacker, R. and Bush, J.1995: Using item mean squares to evaluate fit to the Rasch model. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
    Swanson, H. L., O’Connor, J. E., & Cooney, J. B. (1990). An information processing analysis of expert and novice teachers’ problem solving. American Educational Research Journal, 27(3), 533-556.
    Tavakoli, M., Ghadiri, M., & Zabihi, R. (2014). Direct versus Translated Writing: The Effect of Translation on Learners' Second Language Writing Ability. GEMA Online Journal Of Language Studies, 14(2), 61-74.
    Titford, C. (1985). Translation: A post-communicative activity. C. Titford and A. E. Hieke (eds.) Translation in foreign language teaching and testing. Tübingen: Narr. 73-86.
    Tsui, A. (2003). Understanding expertise in teaching. Ernst Klett Sprachen.
    Voss, J. F., Tyler, S. W., &Yengo, L. A. (1983). Individual differences in the solving of social science problems. In R. Dillon & R. Schmeck (Eds.), Individual differences in cognition (pp. 205–232). New York: Academic Press.
    Weigle, S. C. (1998). Using FACETS to model rater training effects. Language Testing, 15(2), 263-287.
    Weigle, S. C. (2002). Assessing writing. Cambridge language assessment series. Cambridge: CUP.
    Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge university press.
    White, E. M. (1985). Teaching and Assessing Writing: Recent Advances in Understanding, Evaluating, and Improving Student Performance. The Jossey-Bass Higher Education Series.Jossey-Bass Publishers, 433 California St., Suite 1000, San Francisco, CA 94104-2091.
    Widdowson, H. G. (2003). Defining issues in English language teaching. Oxford: Oxford University Press.
    Witte, A., Harden, T., & Ramos de Oliveira Harden, A.,(Eds.). (2009). Translation in second language learning and teaching. Oxford, UK: Peter Lang.
    Wolfe, E. W., Kao, C. W., & Ranney, M. (1998). Cognitive differences in proficient and nonproficient essay scorers. Written Communication, 15(4), 465-492.

    下載圖示
    QR CODE