本論文提出基於感測器的連續手勢辨識系統，使用者右手握著智慧手機做出動作，透過收集搭載在手機上的感測器－三軸陀螺儀與三軸加速度計產生的訊號，總計六維度的資料來構成手勢。面對手勢這種時間序列資料，加上為解決連續手勢中找出切割點(Spotting)的問題，本論文中提出基於Feedforward Neural Networks建立出的深度學習模型，整合現行架構中已被證實能夠更加有效利用Convolutional Neural Networks的結構－ResNet、GoogLeNet與Inception-ResNet，將這些概念與PairNet做結合。
實驗中使用透過手機蒐集的11種手勢，在測試時一次會輸入含有1~4個手勢的資料，進到事先訓練好的類神經網路模型之中，再經由後處理得到辨識結果，而這樣的演算法則能處理傳統方法上無法有效解決的Spotting問題。另外，根據提出的模型ResPairNet在連續手勢上的辨識率，比LSTM高出7%以上的結果也可推得－Feedforward Neural Networks在時間序列資料的處理上，比Recurrent Neural Networks更加強大、有效，將這些原本應用於影像領域的結構，套用到處理時間資料的問題上，能夠更進一步提升Feedforward Neural Networks得學習能力。

[1] S. Mitra and T. Acharya, "Gesture Recognition: A Survey" in IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 37, no. 3, pp. 311-324, May 2007.

[2] Rung-Huei Liang and Ming Ouhyoung, "A real-time continuous gesture recognition system for sign language," Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, 1998, pp. 558-567.

[3] T. Starner, J. Weaver and A. Pentland, "Real-time American sign language recognition using desk and wearable computer based video," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1371-1375, Dec. 1998

[4] A. Agarwal and M. K. Thakur, "Sign language recognition using Microsoft Kinect," 2013 Sixth International Conference on Contemporary Computing (IC3), Noida, 2013, pp. 181-185.

[5] E. Ohn-Bar and M. M. Trivedi, "Hand Gesture Recognition in Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and Evaluations," in IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 6, pp. 2368-2377, Dec. 2014.

[6] Hyun Kang, Chang Woo Lee, and Keechul Jung. 2004. Recognition-based gesture spotting in video games. Pattern Recognition Letters. 25, 15 (November 2004), 1701-1714

[7] G. D. Clark and J. Lindqvist, "Engineering Gesture-Based Authentication Systems," in IEEE Pervasive Computing, vol. 14, no. 1, pp. 18-25, Jan.-Mar. 2015.

[8] Oza, P., & Patel, V.M. (2019). Active Authentication using an Autoencoder regularized CNN-based One-Class Classifier. CoRR, abs/1903.01031.

[9] A. Pozo, J. Fierrez, M. Martinez-Diaz, J. Galbally, and A. Morales, “Exploring a statistical method for touchscreen swipe biometrics,” in Security Technology (ICCST), 2017 International Carnahan Conference on. IEEE, 2017, pp. 1–4.

[10] P. Perera and V. M. Patel, “Extreme value analysis for mobile active user authentication,” in 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). IEEE, 2017, pp. 346–353.

[11] H. Zhang, V. M. Patel, M. Fathy, and R. Chellappa, “Touch gesture based active user authentication using dictionaries,” in 2015 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2015, pp. 207–214.

[12] P. Perera and V. M. Patel, “Towards multiple user active authentication in mobile devices,” in Automatic Face & Gesture Recognition (FG 2017), 2017 12th IEEE International Conference on. IEEE, 2017, pp. 354–361.

[13] Rautaray, Siddharth S., and Anupam Agrawal. "Vision based hand gesture recognition for human computer interaction: a survey." Artificial Intelligence Review 43.1 (2015): 1-54.

[14] Murthy, G. R. S., and R. S. Jadon. "A review of vision based hand gestures recognition." International Journal of Information Technology and Knowledge Management 2.2 (2009): 405-410.

[15] M. Elmezain, A. Al-Hamadi, J. Appenrodt and B. Michaelis, "A Hidden Markov Model-based continuous gesture recognition system for hand motion trajectory," 2008 19th International Conference on Pattern Recognition, Tampa, FL, 2008, pp. 1-4.

[16] T. Starner and A. Pentland, "Real-time American Sign Language recognition from video using hidden Markov models," Proceedings of International Symposium on Computer Vision - ISCV, Coral Gables, FL, USA, 1995, pp. 265-270.

[17] Malima, Ozgur and Cetin, "A Fast Algorithm for Vision-Based Hand Gesture Recognition for Robot Control," 2006 IEEE 14th Signal Processing and Communications Applications, Antalya, 2006, pp. 1-4.

[18] H. P. Gupta, H. S. Chudgar, S. Mukherjee, T. Dutta and K. Sharma, "A Continuous Hand Gestures Recognition Technique for Human-Machine Interaction Using Accelerometer and Gyroscope Sensors," in IEEE Sensors Journal, vol. 16, no. 16, pp. 6425-6432, Aug.15, 2016.

[19] X. Zhang, X. Chen, Y. Li, V. Lantz, K. Wang and J. Yang, "A Framework for Hand Gesture Recognition Based on Accelerometer and EMG Sensors," in IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, vol. 41, no. 6, pp. 1064-1076, Nov. 2011.

[20] T. Tai, Y. Jhang, Z. Liao, K. Teng and W. Hwang, "Sensor-Based Continuous Hand Gesture Recognition by Long Short-Term Memory," in IEEE Sensors Letters, vol. 2, no. 3, pp. 1-4, Sept. 2018, Art no. 6000704.

[21] L. Yun and Z. Peng, "An Automatic Hand Gesture Recognition System Based on Viola-Jones Method and SVMs," 2009 Second International Workshop on Computer Science and Engineering, Qingdao, 2009, pp. 72-76.

[22] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based Learning Applied to Document Recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.

[23] J. Deng, W. Dong, R. Socher, L. Li, Kai Li and Li Fei-Fei, "ImageNet: A large-scale hierarchical image database," 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, 2009, pp. 248-255.

[24] A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet Classification with Deep Convolutional Neural Networks. In NIPS, 2012.

[25] Svozil, Daniel & Kvasnicka, Vladimir & Pospíchal, Jiří. (1997). Introduction to multi-layer feed-forward neural networks. Chemometrics and Intelligent Laboratory Systems. 39. 43-62.

[26] J. Ilonen, J.K. Kamarainen, J. Lampinen, Differential evolution training algorithm for feed-forward neural networks, Neural Processing Letters 17 (2003) 93–105.

[27] Hochreiter, Sepp. "The vanishing gradient problem during learning recurrent neural nets and problem solutions." International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 6.02 (1998): 107-116.

[28] Pascanu, Razvan, Tomas Mikolov, and Yoshua Bengio. "On the difficulty of training recurrent neural networks." In Proc. 30th International Conference on Machine Learning 1310–1318 (2013).

[29] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.

[30] Gers, F. A., Schmidhuber, J., & Cummins, F. Learning to forget: Continual prediction with LSTM. Neural Computation, 12(10):2451–2471, 2000.

[31] M. Schuster and K. K. Paliwal, "Bidirectional recurrent neural networks," in IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, Nov. 1997.

[32] M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In D. J. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, editors, ECCV, volume 8689 of Lecture Notes in Computer Science, pages 818–833. Springer, 2014.

[33] 張筠婕，《基於PairNet的連續手勢辨識》，國立臺灣師範大學資訊工程研究所

[34] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016, pp. 770-778.

[35] K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In ECCV, 2016

[36] C. Szegedy et al., "Going deeper with convolutions," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, 2015, pp. 1-9.

[37] C. Szegedy, S. Ioffe, and V. Vanhoucke. Inception-v4, inception-resnet and the impact of residual connections on learning. In ICLR Workshop, 2016.