Hand Symbol Classification for Human-Computer Interaction Using the Fifth Version of YOLO Object Detection
DOI:
https://doi.org/10.21512/commit.v17i1.8520Keywords:
Hand Symbol Classification, Human- Computer Interaction, YOLO Fifth Version, Object DetectionAbstract
Human-Computer Interaction (HCI) nowadays mostly uses physical contact, such as people using the mouse to choose something in an application. However, there are certain problems that people face in using conventional HCI. The research tries to overcome some problems when people use conventional HCI using the computer vision method. The research focuses on creating and evaluating the object detection model for classifying hand symbols. The research applies the fifth version of YOLO with the architecture of YOLOv5m to classify hand symbols in real time. The methods are divided into three steps. Those steps are dataset creation consisting of 100 images in each class, training phase, and performance evaluation of the model. The hand gesture classes made in the research are ‘ok’, ‘cancel’, ‘previous’, ‘next’, and ‘confirm’, the dataset is made by the researchers custom. After the training phase, the validation results show 93% for accuracy, 99% for precision, 100% for recall, and 99% for F1 score. Meanwhile, in real-time detection, the performance of the model for classifying hand symbols is 80% for accuracy, 95% for precision, 84% for recall, and 89% for F1 score. Although there are differences, it still acceptable for the research and can be improved in future research.
Plum Analytics
References
A. Mujahid, M. J. Awan, A. Yasin, M. A. Mohammed, R. Damaˇseviˇcius, R. Maskeli¯unas, and K. H. Abdulkareem, “Real-time hand gesture recognition based on deep learning YOLOv3 model,” Applied Sciences, vol. 11, no. 9, pp. 1–15, 2021.
S. Ahmed and S. H. Cho, “Hand gesture recognition using an IR-UWB radar with an inception module-based classifier,” Sensors, vol. 20, no. 2, pp. 1–18, 2020.
S. Parikh, S. Banka, I. Lautrey, and D. Yedurkar, “Human-computer interaction using dynamic hand gesture recognition to conveniently control the system,” International Journal of Engineering Applied Sciences and Technology, vol. 5, no. 9, pp. 206–210, 2021.
K. Nguyen-Trong, H. N. Vu, N. N. Trung, and C. Pham, “Gesture recognition using wearable sensors with bi-long short-term memory convolutional neural networks,” IEEE Sensors Journal, vol. 21, no. 13, pp. 15 065–15 079, 2021.
M. A. M. A. Shariah, R. N. Ainon, R. Zainuddin, and O. O. Khalifa, “Human computer interaction using isolated-words speech recognition technology,” in 2007 International Conference on Intelligent and Advanced Systems. Kuala Lumpur, Malaysia: IEEE, Nov. 25–28, 2007, pp. 1173–1178.
O. K¨op¨ukl¨u, A. Gunduz, N. Kose, and G. Rigoll, “Real-time hand gesture detection and classification using convolutional neural networks,” in 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019). Lille, France: IEEE, May 14–18, 2019, pp. 1–8.
S. Daniels, N. Suciati, and C. Fathichah, “Indonesian sign language recognition using YOLO method,” in IOP Conference Series: Materials Science and Engineering, vol. 1077. IOP Publishing, 2021, pp. 1–9.
Z. Zheng, “Gesture recognition real-time control system based on YOLOV4,” Journal of Physics: Conference Series, vol. 2196, pp. 1–9, 2022.
D. Hu, J. Zhu, J. Liu, J. Wang, and X. Zhang, “Gesture recognition based on modified Yolov5s,” IET Image Processing, vol. 16, no. 8, pp. 2124–2132, 2022.
E. Padmalatha, S. Sailekya, R. R. Reddy, C. A. Krishna, and K. Divyarsha, “Sign language recognition,” International Journal of Recent Technology and Engineering (IJRTE), vol. 8, no. 3, pp. 2128–2137, 2019.
M. Rivera-Acosta, J. M. Ruiz-Varela, S. Ortega-Cisneros, J. Rivera, R. Parra-Michel, and P. Mejia-Alvarez, “Spelling correction real-time american sign language alphabet translation system based on YOLO network and LSTM,” Electronics, vol. 10, no. 9, pp. 1–18, 2021.
A. Erol, G. Bebis, M. Nicolescu, R. D. Boyle, and X. Twombly, “Vision-based hand pose estimation: A review,” Computer Vision and Image Understanding, vol. 108, no. 1-2, pp. 52–73, 2007.
M. A. M. M. Asri, Z. Ahmad, I. A. Mohtar, and S. Ibrahim, “A real time Malaysian sign language detection algorithm based on YOLOv3,” International Journal of Recent Technology and Engineering, vol. 8, no. 2, pp. 651–656, 2019.
P. Soviany and R. T. Ionescu, “Optimizing the trade-off between single-stage and two-stage deep object detectors using image difficulty prediction,” in 2018 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC). Timisoara, Romania: IEEE, Sept. 20–23, 2018, pp. 209–214.
L. Aziz, M. S. B. H. Salam, U. U. Sheikh, and S. Ayub, “Exploring deep learning-based architecture, strategies, applications and current trends in generic object detection: A comprehensive review,” IEEE Access, vol. 8, pp. 170 461–170 495, 2020.
S. Chaudhari, N. Malkan, A. Momin, and M. Bonde, “YOLO real time object detection,” International Journal of Computer Trends and Technology, vol. 68, no. 6, pp. 70–76, 2020.
J. Redmon and A. Farhadi, “YOLO9000: Better, faster, stronger,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 7263–7271.
——, “YOLOv3: An incremental improvement,” 2018. [Online]. Available: https://arxiv.org/abs/1804.02767
A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, “YOLOv4: Optimal speed and accuracy of object detection,” 2020. [Online]. Available: https://arxiv.org/abs/2004.10934
U. Nepal and H. Eslamiat, “Comparing YOLOv3, YOLOv4 and YOLOv5 for autonomous landing spot detection in faulty UAVs,” Sensors, vol. 22, no. 2, pp. 1–15, 2022.
H. Liu, F. Sun, J. Gu, and L. Deng, “SF-YOLOv5: A lightweight small object detection algorithm based on improved feature fusion mode,” Sensors, vol. 22, no. 15, pp. 1–14, 2022.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Sugiarto Wibowo, Indar Sugiarto
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
a. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License - Share Alike that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
b. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
c. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
USER RIGHTS
All articles published Open Access will be immediately and permanently free for everyone to read and download. We are continuously working with our author communities to select the best choice of license options, currently being defined for this journal as follows: Creative Commons Attribution-Share Alike (CC BY-SA)
