教師資料查詢 | 類別: 期刊論文 | 教師: 武士戎 WU, SHIH-JUNG (瀏覽個人網頁)

標題:3D finger tracking and recognition image processing for real-time music playing with depth sensors
學年106
學期2
出版(發表)日期2018/04/01
作品名稱3D finger tracking and recognition image processing for real-time music playing with depth sensors
作品名稱(其他語言)
著者Wu, Shih-Jung, et al.
單位
出版者
著錄名稱、卷期、頁數Multimedia Tools and Applications 77(8), p.9233-9248
摘要In this research, we propose a state-of-the-art 3D finger gesture tracking and recognition method. We use the depth sensors for both hands in real time music playing. In line with the development of 3D depth cameras, we implemented a set of 3D gesture-based instruments, such as Virtual Cello and Virtual Piano, which need precise finger tracking in 3D space. For hands tracking, model-based tracking for left hand and appearance-based tracking for right hand techniques are proposed. To detect finger gestures, our approaches consist number of systematic steps as reducing noise in depth map and geometrical processing for Virtual Cello. For Virtual Piano, we introduce the Neural Network (NN) method to detect special hand gestures. It has Multilayer Perceptron (MLP) structure with back propagation training. Literature has few examples using touch screen as medium, with fixed-coordinates, and 2D–gestures to control MIDI input. The end users should no longer carry anything on their hands. We use Senz3D and Leap Motion due to a few technical benefits. Senz3D and Leap Motion use a closer distance to hands, thus detailed finger gestures can be precisely identified. In the past years, we announced a set of virtual musical instruments and the MINE Virtual Band. Our research work is tested on lab environment and professional theatrical stage. More information and demonstrations of the proposed method can be accessed at: http://video.minelab.tw/DETS/VMIB/.
關鍵字3D finger gesture;3D finger tracking;3D finger recognition, Gesture recognition, Neural network;Senz3D;Leap motion Virtual musical instruments
語言英文
ISSN1380-7501; 1573-7721
期刊性質國外
收錄於SCI;EI;
產學合作
通訊作者
審稿制度
國別美國
公開徵稿
出版型式,電子版,紙本
相關連結
Google+ 推薦功能,讓全世界都能看到您的推薦!