Turkish Sign language recognition using spatio-temporal features on kinect RGB video sequences and depth maps Kinect RGB görüntülerde ve derinlik haritalarinda uzam-zamansal özellikleri kullanarak türk işaret dili tanima

Memiş A., VARLI S.

2013 21st Signal Processing and Communications Applications Conference, SIU 2013, Haspolat, Türkiye, 24 - 26 Nisan 2013 identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/siu.2013.6531360
  • Basıldığı Şehir: Haspolat
  • Basıldığı Ülke: Türkiye
  • Anahtar Kelimeler: Depth maps, Dynamic signs, Kinect sensor, Sign language recognition, Spatio-temporal features, Turkish sign language
  • İstanbul Üniversitesi Adresli: Hayır

Özet

This paper presents a Turkish Sign Language recognition system that uses spatio-temporal features on Kinect sensor RGB video sequences and depth maps. Proposed system uses cumulative motion images which based on motion differences and represent the temporal characteristics of dynamic signs in motion sequences. Cumulative motion images represent the whole motions of signers. 2-D Discrete Cosine Transform (DCT) is applied to cumulative sign images in order to obtain spatial features of signs and transformed images that represent the energy density of signs are obtained. Two transform images are obtained by applying referred methods to both of RGB video sequences and depth maps seperately. Feature vectors of dynamic signs are produced by combining a certain amount of DCT coefficients that contain higher energy via zig-zag scanning on transform images. K-Nearist Neighbor classifier with Manhattan distance used for recognition process. System performance is evaluated on a sign database that contains 1002 signs belongs to 111 words in three different categories of Turkish Sign Language (TID). Proposed sign language recognition system has a recognition rate about %90. © 2013 IEEE.