研究生: |
曹家豪 |
---|---|
論文名稱: |
以自然語言技術自動評估學生答案之研究 Automatic Assessment of Students’ Answers by Natural Language Processing Techniques |
指導教授: | 侯文娟 |
學位類別: |
碩士 Master |
系所名稱: |
資訊工程學系 Department of Computer Science and Information Engineering |
論文出版年: | 2010 |
畢業學年度: | 98 |
語文別: | 中文 |
論文頁數: | 53 |
中文關鍵詞: | 自由文本 、自然語言處理 、支撐向量機 |
英文關鍵詞: | free-text assessment, natural language processing, support vector machine |
論文種類: | 學術論文 |
相關次數: | 點閱:182 下載:20 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
為了增進老師與學生之間的互動以提高學生學習的意願,最主要的事是讓老師能迅速的了解學生的學習狀況,一個智慧型電腦系統應該要具備有自動評量學生分數的功能,當老師提出了問題後,一開始我們先建立評量的文件集,有了這些文件集後,我們依照下列的步驟去擷取出有意義的資訊:(1)為了得到句法的相關資訊,我們一開始對文件做詞性標記,(2)因為標點符號以及十進位數字會對我們造成干擾的資訊,我們也對此進行去除的動作,(3)為了聚集更好的資訊,我們也對句子進行正規化以及還原成字根的步驟,(4)擷取另外的資訊。在這篇論文中我們將評量的問題改成以分類的角度來進行實驗,即將學生分數分成兩個類別,其中一個類別是得分為6分到10分,另外一個類別就是0到5分。我們得到平均精確度為65.2%,並且從初步的二分類擴展到多分類,precision由原本的65.2%提高到70.8%藉由SVM進行的實驗得到了一個令人振奮的結果,在未來希望能有更進一步的成果。
For improving the interaction between students and teachers, it is fundamental for teachers to understand students' learning levels. An intelligent computer system should be able to automatically evaluate students' answers when the teacher asks some questions. We first built the assessment corpus. With the corpus, we applied the following procedures to extract the relevant information: (1) apply the part-of-speech tagging such that the syntactic information is extracted, (2) remove the punctuation and decimal numbers because they play the noise roles, and (3) for grouping the information, apply the stemming and normalization procedure to sentences, (4) extract other features. In this study, we treated the assessment problem as the classifying problem, i.e., classifying students’ scores as two classes such as above/below 6 out of 10. We got an average of 65.2% precision rate, and extended from binary to multiple categories, precision from the original 65.2% to 70.8%. The experiments with SVM show exhilarating results and some improving efforts will be further made in the future.
1. Alfonseca, E., Carro, R.M., Freire, M., Ortigosa, A., Perez, D., Rodríguez, P. Educational Adaptive Hypermedia meets Computer Assisted Assessment. In: Proceedings of the Workshop on Authoring of Adaptive and Adaptable Educational Hypermedia, at AH 2004, Eindhoven, pp. 4--12 (2004)
2. Chowdhury, G.G.: Natural language processing. Annual Review of Information Science and Technology. pp. 51-90. (2003)
3.Dessus, P., Lemaire, B., Vernier, A.: Free Text Assessment in a Virtual Campus. In: Proceedings of the 3rd International Conference on Human System Learning, pp. 61--75. (2000)
4.Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T., Harshman R.: Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, vol. 41, no. 6, pp. 391--407 (1990)
5. Foltz, T., Kintsch, W., Landauer, T.: The Measurement of Textual Coherence with Latent Semantic Analysis. Discourse Processes vol. 25, no. 2-3, Special Issue: Quantitative Approaches to Semantic Knowledge Representations (1998)
6. Friedman, C., Hripcsak G., DuMouchel, W., Johnson, S.B., Clayton, P.D.: Natural language processing in an operational clinical information system. Nat.Lang.Eng.
pp. 83-108. (1995)
7. Gheorghiu, R. and VanLehn, K. XTutor: An Intelligent Tutor System for Science and Math Based on Excel. Lecture Notes in Computer Science Springer Berlin / Heidelberg Volume 5091/2008 pp.749-751(2009)
8. Hsu, C.W., Chang, C.C., Lin, C.J.: A Practical Guide to Support Vector Classification. http://www.csie.ntu.edu.tw/~cjlin/libsvm/index.html (2003)
9. Jackson, P. and Moulinier, I.: Natural Language Processing for Online Applications: Text Retrieval, Extraction and Categorization. John Benjamins Publishing Company, (2002)
10. Kanejiya, D., Kumary, A., Prasad, S.: Automatic Evaluation of Students’ Answers using Syntactically Enhanced LSA. In: Proceedings of the HLT-NAACL Workshop on Building Educational Applications Using Natural Language Processing, pp. 53--60 (2003)
11. Kraaij, W., Pohlmann, R.: Porter's Stemming Algorithm for Dutch. In L.G.M. Noordman and W.A.M. de Vroomen, editors, Informatiewetenschap 1994: Wetenschappelijke bijdragen aan de derde STINFON Conferentie, pages 167-180, (1994)
12. Litman, D. and Silliman, S. ITSPOKE: An intelligent tutoring spoken Dialogue system. In Companion Proc. Of the Human Language Technology Conf. of the North American Chap. of the Assoc. for Computational Linguistics(HLT/NAACL) (2004)
13. Marcus, M.P., Santorini, B., Marcinkiewicz, M.A.: Building a Large Annotated Corpus of English: The Penn Treebank. In: Computational Linguistics,vol.19,no2, pp. 313--330 (1993)
14. Naismith, L., Blanchard, E.G., Ranellucci, J., Lajoie, S.P.: EAGLE: An Intelligent Tutoring System to Support Experiential Learning Through Video Games.Proceeding of the 2009 conference on Artificial Intelligence in Education pp. 719-721 (2009)
15. Pérez, D., Alfonseca, E. and Rodríguez, P.: Application of the BLEU Method for Evaluating Free-text Answers in an E-learning Environment. In: Proceedings of Language Resources and Evaluation Conference (LREC-2004), Portugal (2004)
16. Perez-Marin, D., Pascual-Nieto, I., Alfonseca, E., Anguiano, E., Rodriguez, P.: A Study on the Impact of the Use of an Automatic and Adaptive Free-text ssessment
System during a University Course. In: Proceedings of the International Conference on Web-based Learning (ICWL), pp. 186--195, Prentice Hall (2007)
17. Support vector machine, http://svmlight.joachims.org/
18. Sebastiani, F.: “Machine learning in automated text categorization” ACM Comput Surveys (CSUR) 34, pp.1 – 47, (2002)
19. Vapnik, V.: The Nature of Statistical Learning Theory. Springer-Verlag (1995)
20. Wiemer-Hastings, P., Zipitria, I.: Rules for Syntax, Vectors for Semantics. In: Proceedings of the 23rd Annual Conference of the Cognitive Science Society, pp.
1140--1145 (2001)
21. Wang, Y., and Wang, X.J.: “ A New Approach to feature selection in Text Classification”, Proceedings of 4th International Conference on Machine Learning and Cybernetics, IEEE- 2005, Vol.6, pp. 3814-3819, (2005)
22. Wang, Z.Q., Xiao, S., Zhang D.X., Li, X.: “An Optimal Svm-Based Text Classification Algorithm” Fifth International Conference on Machine Learning and Cybernetics, Dalian, (2006)