隨著網際網路的蓬勃發展，越來越多的訊息在社群網站、線上購物網站、論壇等各種網路平台間傳遞，而這一些訊息可能都表達了人們的看法或是評價。但是只依靠人力來觀察如此龐大的資訊量是非常沒有效率的，因此如何讓電腦得以代替人類完成這一項工作量龐大的任務是必要的。
自然語言處理(Natural Language Processing,NLP)是一種讓電腦可以理解人類語言的技術，而情感分析(Sentiment Analysis)則是NLP其中的一項常見應用。它能夠了解字句間所表達的情緒，比如分析網路上對於某些產品、名人、事件等事物的評論立場為何，像是有好感還是持有負面態度。
本實驗使用含有長文的IMDB資料集進行情感分析，該資料集將評論分為正面和負面，並且建立深度學習模型讓它藉由評論內容判斷評論表達的情緒是正面或負面，除了基本的LSTM和BERT模型以外，本實驗還有嘗詴讓BERT合併BERT或LSTM模型，希望藉由增加模型獲得的特徵來提高準確度，並且對各種模型的實驗結果進行比較。

With the vigorous development of the Internet, more and more information is transmitted among various network platforms such as social networking sites, online shopping sites, forums, etc., and these messages may express people's opinions or evaluations. However, it is very inefficient to observe such a huge amount of information only by people. Therefore, it is necessary to allow computers to replace people to complete this huge task.
Natural Language Processing (NLP) is a technology that enables computers to understand human language, and sentiment analysis is a common application in the NLP domain. Sentiment analysis aims to understand the emotions expressed between words, such as analyzing the stance of comments on certain products, celebrities, events, etc. on the Internet, and then identifying positive or negative polarity.
This experiment uses the IMDB dataset containing long texts for sentiment analysis, which divides comments into positive and negative polarity. This thesis proposes a deep learning model to judge whether the sentiment expressed by the comment is positive or negative based on the content of the comment. In addition to the LSTM and BERT model, this experiment also tried to combine the BERT or LSTM model with BERT, hoping to improve the accuracy by increasing the features obtained by the model, and to compare the experimental results of different models.

[1] Khan, M. T., Durrani, M., Ali, A., Inayat, I., Khalid, S., & Khan, K. H. (2016). Sentiment analysis and the complex natural language. Complex Adaptive Systems Modeling, 4(1), 1-19.

[2]Dai, W., Xue, G. R., Yang, Q., & Yu, Y. (2007, July). Transferring naive bayes classifiers for text classification. In AAAI (Vol. 7, pp. 540-545).

[3]Shin, K., Abraham, A., & Han, S. Y. (2006, February). Improving kNN text categorization by removing outliers from training set. In International conference on intelligent text processing and computational linguistics (pp. 563-566). Springer, Berlin, Heidelberg.

[4]Joachims, T. (1998, April). Text categorization with support vector machines: Learning with many relevant features. In European conference on machine learning (pp. 137-142). Springer, Berlin, Heidelberg.

[5] Chen, Y. (2015). Convolutional neural network for sentence classification (Master's thesis, University of Waterloo).

[6] Zhang, Y., Liu, Q., & Song, L. (2018). Sentence-state lstm for text representation. arXiv preprint arXiv:1805.02474.

[7] Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

[8] Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.

[9] Maas, A., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011, June). Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies (pp. 142-150).

[10] Pennington, J., Socher, R., & Manning, C. D. (2014, October). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532-1543).

[11] Varsamopoulos, S., Bertels, K., & Almudever, C. G. (2018). Designing neural network based decoders for surface codes. arXiv preprint arXiv:1811.12456.

[12] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.

[13] Sun, C., Qiu, X., Xu, Y., & Huang, X. (2019). How to Fine-Tune BERT for Text Classification?. arXiv e-prints, arXiv-1905.

[14]Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R. R., & Le, Q. V. (2019). Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32.

[15] Howard, J., & Ruder, S. (2018). Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146.

[16] Johnson, R., & Zhang, T. (2016, June). Supervised and semi-supervised text categorization using LSTM for region embeddings. In International Conference on Machine Learning (pp. 526-534). PMLR.

[17] McCann, B., Bradbury, J., Xiong, C., & Socher, R. (2017). Learned in translation: Contextualized word vectors. Advances in neural information processing systems, 30.

[18] Seo, M., Min, S., Farhadi, A., & Hajishirzi, H. (2017). Neural speed reading via skim-rnn. arXiv preprint arXiv:1711.02085.

[19] Camacho-Collados, J., & Pilehvar, M. T. (2017). On the role of text preprocessing in neural network architectures: An evaluation study on text categorization and sentiment analysis. arXiv preprint arXiv:1707.01780.