Deteksi Gestur Tangan Berbasis Pengolahan Citra

Abdullah Sani, Suci Rahmadinni


Hand sign language is a medium of communication for people with disabilities (deaf and speech impaired). However, in social practice, persons with disabilities may have to communicate with non-disable persons who do not understand sign language. These problems can be overcome with the help of translators or normal people learning sign language through existing media such as videos. Unfortunately, this method will probably cost a lot of money and time. In respons to this issue, the present study designed a sistem to detect hand gestures based on image processing. The method used is the You Only Look Once (YOLO) algorithm. The YOLO algorithm can detect and classify objects at once without being influenced by the light intensity and background of the object. This algorithm is a deep learning method that is more accurate than other deep learning methods. From this research, the system can detect and classify hand gestures with different backgrounds, light intensity, and distances with an accuracy rate above 90%.


deep learning; sign language; tracking; YOLO-v3

Full Text:



I. Steinberg, T. M. London and C. D. Dotan, "Hand Gesture Recognition in Images dan and Video," CCIT Report, vol. 763, pp. 1-20, 2010.

W. Gazali and H. Soeparno, "Pendeteksian Gerak Tangan Manusia Sebagai Input Pada Komputer," Mat Sat, vol. 11, no. 2, pp. 129-137, 2011.

M. D. Rosyadi, F. Hafidh and M. Y. Kurniawan, "Pengenalan Real Time Abjad Bahasa Isyarat Indonesia Menggunakan Segmentasi YCbCr," vol. 2, no. 2, pp. 1-5, 2017.

F. Asriani and H. Susilawati, "Pengenalan Isyarat Tangan Statis Pada Sistem Isyarat Bahasa Indonesia Berbasis Jaringan Syaraf Tiruan Perambatan Balik," Makara Journal of Technology, vol. 14, no. 2, pp. 150-154, 2010.

S. E. Octaviani, "Tracking dan Pengenalan Pola Gerak Tangan Berbasis Penginderaan Visual," Tugas Akhir, Politeknik Negeri Batam, Batam, 2019.

J. Redmon, S. Divvala, R. Girshick and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 779-788, doi: 10.1109/CVPR.2016.91.

L. Chen, F. Wang, H. Deng and K. Ji, "A Survey on Hand Gesture Recognition," Conference: Proceedings of the 2013 International Conference on Computer Sciences and Applications, vol. DOI:10.1109/CSA.2013.79, 2013.

Kanchan Dabre and D. Surekha, "Machine Learning Model for Sign Language Interpretation using Webcam Images," International Conference on Circuits, Systems, Communication and Information Technology Applications (CSCITA), pp. 317-321, 2014.

L. Pigou, S. Dieleman and P.-J. Kindermas, "Sign Language Recognition Using Convolutional Neural Networks," pp. 572-578, 2015.

A. Y. R, "YOLO (you only look once)”, Universitas Gadjah Mada Menara Ilmu Machine Learning, 5 Agustus 2018. [Online]. Available: [Accessed 20 Apr 2021].

D. Karunakaran, "Deep learning series 1: Intro to deep learning”, Medium, 23 April 2018. [Online]. Available: deep-learning-abb1780ee20. [Accessed 20 Apr 2021].

J. W. G. Putra, Pengenalan Konsep Pembelajaran Mesin dan Deep Learning, self-published work, 2020.

S. Sena, "Pengenalan Deep Learning Part 7 : Convolutional Neural Network (CNN)", Medium, 13 Nov 2017. [Online]. Available: part-7-convolutional-neural-network-cnn-b003b477dc94. [Accessed 20 Apr 2021].

Fawwaz, M. A. A. Fawwaz, K. N. Ramadhani and F. Sthevanie, "Klasifikasi Ras pada Kucing menggunakan Algoritma Convolutional Neural Network(CNN)," e-Proceeding of Engineering, vol. 8, no. 1, pp. 715-730, 2021.

Hanin, M. A. Hanin, R. Patmasari and R. Y. N. Fu'adah, "Sistem Klasifikasi Penyakit Kulit Menggunakan Convolutional Neural Network (CNN)," e-Proceeding of Engineering, vol. 8, no. 1, pp. 273-281, 2021.

S. E. Rudiawan, R. Analia, D. S. P and H. Soebakti, "The Deep Learning Development for Real-Time Ball and Goal Detection of Barelang-FC," International Electronics Symposium on Engineering Technology and Applications (IES-ETA), September 2017.

Q. Aini, N. Lutfiani, H. Kusumah and M. S. Zahran, "Deteksi dan Pengenalan Objek Dengan Model Machine Learning: Model YOLO," CESS (Journal of Computer Engineering System and Science), vol. 6, no. 2, pp. 192-199, 2021.

M. Mantripragada, "Digging deep into YOLO V3 - A hands-on guide Part 1", Towards Data Science, 16 August 2020. [Online]. Available: YOLO-v3-a-hands-on-guide-part-1-78681f2c7e29. [Accessed January 2021].

B. P. G. Pamungkas, B. Nugroho and F. Anggraeny, "Deteksi dan Menghitung Manusia Menggunakan YOLO_CNN," Jurnal Informatika dan Sistem Informasi (JIFoSI), vol. 02, no. 1, pp. 67-76, 2021.

F. Indaryanto, A. Nugroho and A. F. Suni, "Aplikasi Penghitung Jarak dan Jumlah Orang Berbasis YOLO Sebagai Protokol Kesehatan Covid-19," Edu Kompatika, vol. 8, no. 1, pp. 31-38, 2021



  • There are currently no refbacks.

View My Stats


Creative Commons License

Jurnal Rekayasa Elektrika (JRE) is published under license of Creative Commons Attribution-ShareAlike 4.0 International License.