本研究提出視覺化互動式音樂指揮系統供使用者培養音樂節奏感。利用快速平均位移追蹤演算法即時追蹤指揮動作，並藉由分析指揮軌跡可以決定指揮速度。使用者練習指揮時，本系統可以提供聽覺與視覺的節奏提示，來幫助使用者穩定節奏感。甚至音樂速度可隨著指揮速度改變，作為進階的節奏訓練。
實驗結果顯示本研究的指揮追蹤與指揮速度偵測是很可靠的。音樂指揮實驗顯示使用者藉由指揮系統練習節奏，對提升節奏感是很有效果的。

This paper presents a vision-based interactive music conducting practice system. A fast mean-shift object tracking algorithm is implemented to track the baton of the conductor in real time. By analyzing the tracked baton path, the conducting tempo is determined. When practicing, the system provides both visual and audio cues to help user to improve their steadiness. Furthermore, practice conduction of MP3 music with realtime feedback is possible.
Experimental results show that the system is robust in terms of baton tracking and tempo detection. Furthermore, music conduction experiments show that the practicing system is effective for steading user's conducting tempo and enhancing conducting skills with only limited number of practices.

[1] 張統星，“音樂科教材教法”，全音樂譜出版社，1999

[2] T.H. Andersen, "Mixxx: towards novel DJ interfaces," Proceedings of the 2003 conference on New interfaces for musical expression, National University of Singapore Singapore, Singapore, pp. 30- 35, 2003

[3] R. Boulanger, "The 1997 mathews Radio-Baton and improvisation modes," Proceedings of the International Computer Music Conference, pp. 395- 398, 1997

[4] J. Borchers, E. Lee, W. Samminger, and M. Mühlhäuser, "Personal orchestra: a real-time audio/video system for interactive conducting," Multimedia Systems, vol. 9, pp. 458- 465, 2004

[5] R.V. Babu, P. Pérez, and P. Bouthemy, "Robust tracking with motion estimation and local Kernel-based color modeling," Image and Vision Computing, vol. 25, pp. 1205- 1216, 2007

[6] C.P. Chuang, "A Vision-based Real-time Conductor Gesture Tracking System," M.S. Degree Thesis, National Taiwan Normal University, 2007

[7] A. Camurri, S. Hashimoto, M. Ricchetti, A. Ricci, K. Suzuki, R. Trocca, and G.. Volpe, "EyesWeb: Toward Gesture and Affect Recognition in Interactive Dance and Music Systems," Computer Music Journal, vol. 24, pp. 57- 69. 2000

[8] D. Comaniciu, V. Ramesh, and P. Meer, "Kernel-Based Object Tracking", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25 pp. 564- 577, 2003

[9] M.T. Driscoll, "A Machine Vision System for Capture and Interpretation of an Orchestra Conductor’s Gestures," M.S Degree, Worcester Polytechnic Institute, 1999

[10] A. Elgammal, R. Duraiswami, D. Harwood, and L.S. Davis, "Background and foreground modeling using nonparametric kernel density estimation for visual surveillance," Proceedings of the IEEE, vol. 90, pp. 1151- 1163, 2002

[11] P.C. Hernandez, J. Czyz, F. Marques, T. Umeda, X. Marichal, and B. Macq, "Bayesian Approach for Morphology-Based 2-D Human Motion Capture," IEEE Transactions on multimedia, vol. 9, pp. 754- 765, 2007

[12] T. Ilmonen and T. Takala, "Conductor following with artificial neural networks," Proceedings of the International Computer Music Conference, pp. 367- 370, 1999

[13] P. Kakumanu, S. Makrogiannis, and N. Bourbakis, "A survey of skin-color modeling and detection methods," Pattern Recognition, vol. 40, pp. 1106- 1122, 2007

[14] P. Kolesnik, "Conducting Gesture Recognition, Analysis and Performance System," M.S. Degree, McGill University, 2004

[15] E. Lee, "You're the conductor: A realistic interactive conducting system for children," Proceedings of the NIME 2004 Conference on New Interfaces for Musical Expression, pp. 68- 73, 2004

[16] M. Lee, G. Garnett, and D. Wessel, "An Adaptive Conductor Follower," Proceedings of the International Computer Music Conference, pp. 454- 454, 1992

[17] B.D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," International Joint Conference on Artificial Intelligence, 1981

[18] T. Marrin and J. Paradiso, "The Digital Baton: a Versatile Performance Instrument," Proceedings of the International Computer Music Conference, pp. 313- 316, 1997

[19] T. Marrin and R. Picard, "The conductor's jacket: A device for recording expressive musical gestures," Proceedings of the International Computer Music Conference, 1998

[20] H. Morita, S. Hashimoto, and S. Ohteru, "A computer music system that follows a human conductor," Computer, vol. 24, pp. 44- 53, 1991

[21] D. Murphy, T.H. Andersen, and K. Jensen, "Conducting Audio Files via Computer Vision," Lecture Notes in Computer Science, pp. 529-540, 2004

[22] W.K. Pratt, Digital Image Processing: PIKS Inside, 3rd Edition, wiley-Interscience, 2001

[23] D.A. Ross, J. Lim, R.S. Lin, and M.H. Yang, "Incremental Learning for Robust Visual Tracking," International Journal of Computer Vision, vol. 77, pp.125- 141, 2008

[24] U. Satoshi and M. Yasunori, "A conducting recognition system on the model of musician's process," Journal of the Acoustical Society of Japan, 1998

[25] J. Segen, J. Gluckman, and S. Kumar, "Visual Interface for Conducting Virtual Orchestra," Interactional Conference on Pattern Recognition, pp. 276- 279, 2000

[26] T. Sim, D. Ng, and R. Janakiraman, "VIM: Vision for Interactive Music," IEEE Workshop on Applications of Computer Vision, pp. 32- 32, 2007

[27] C. Shen, M.J. Brooks, and A. Henge, "Fast Global Kernel Density Mode Seeking: Application to Localization and Tracking," IEEE Transactions on Image Processing, vol. 16, pp. 1457- 1469, 2007

[28] H. Veeraraghavan, P. Schrater, and N. Papanikolopoulos, "Robust target detection and tracking through integration of motion, color, and geometry," Computer Vision and Image Understanding, vol. 103, pp. 121- 138, 2006

[29] W. Wang and S. Maybank, "A survey on visual surveillance of object motion and behaviors," IEEE Transactions on Systems, Man and Cybernetics, Part C, vol. 34, pp. 334-352, 2004

[30] C.R. Wren, A. Azarbayejani, T. Darrell, and A.P. Pentland, "Pfinder: Real-Time Tracking of the Human Body," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19. pp. 780-785, 1997

[31] H. Wang and D. Suter, "A consensus-based method for tracking: Modeling background scenario and foreground appearance," Pattern Recognition, vol. 40, pp. 1091-1105, 2007

[32] A. Yilmaz, O. Javed, and M. Shah, "Object tracking: A survey," ACM Computing Surveys, vol. 38, pp. 1- 45, 2006

[33] Bass Library, http://www.un4seen.com/