Deep Transfer Learning for Sign Language Image Classification: A Bisindo Dataset Study
DOI:
https://doi.org/10.21512/emacsjournal.v5i3.10621Keywords:
BISINDO, Sign Language, Resnet50, effisienNetB1, MobileNetV4Abstract
This study aims to identify and categorize the BISINDO sign language dataset, primarily consisting of image data. Deep learning techniques are used, with three pre-trained models: ResNet50 for training, MobileNetV4 for validation, and InceptionV3 for testing. The primary objective is to evaluate and compare the performance of each model based on the loss function derived during training. The training success rate provides a rough idea of the ResNet50 model's understanding of the BISINDO dataset, while MobileNetV4 measures validation loss to understand the model's generalization abilities. The InceptionV3-evaluated test loss serves as the ultimate litmus test for the model's performance, evaluating its ability to classify unobserved sign language images. The results of these exhaustive experiments will determine the most effective model and achieve the highest performance in sign language recognition using the BISINDO dataset.
Plum Analytics
References
Alam, I. N., Kartowisastro, I. H., & Wicaksono, P. (2022). Transfer Learning Technique with EfficientNet for Facial Expression Recognition System. Revue d’Intelligence Artificielle, 36(4), 543–552. https://doi.org/10.18280/ria.360405
Arisandi, L., & Satya, B. (2022). Sistem Klarifikasi Bahasa Isyarat Indonesia (Bisindo) Dengan Menggunakan Algoritma Convolutional Neural Network. Jurnal Sistem Cerdas, 5(3), 135–146. https://doi.org/10.37396/jsc.v5i3.262
Bantupalli, K., & Xie, Y. (2019). American Sign Language Recognition using Deep Learning and Computer Vision. Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018, 4896–4899. https://doi.org/10.1109/BigData.2018.8622141
Bestari, H. (2018). Mengenal Bahasa Isyarat. Website. https://www.ypedulikasihabk.org/2018/11/09/mengenal-bahasa-isyarat/
Developer Google. (2022). Classification: Accuracy. Website. https://developers.google.com/machine-learning/crash-course/classification/accuracy
Fadlilah, U., Mahamad, A. K., & Handaga, B. (2021). The Development of Android for Indonesian Sign Language Using Tensorflow Lite and CNN: An Initial Study. Journal of Physics: Conference Series, 1858(1). https://doi.org/10.1088/1742-6596/1858/1/012085
Fauzi, M. Z., Sarno, R., & Hidayati, S. C. (2023). Recognition of Real-Time BISINDO Sign Language-to-Speech using Machine Learning Methods. International Conference on Computer Science, Information Technology and Engineering (ICCoSITE). https://doi.org/10.1109/ICCoSITE57641.2023.10127743
Handhika, T., Zen, R. I. M., Murni, Lestari, D. P., & Sari, I. (2018). Gesture recognition for Indonesian Sign Language (BISINDO). Journal of Physics: Conference Series, 1028, 012173. https://doi.org/10.1088/1742-6596/1028/1/012173
Hasan, M. M., Srizon, A. Y., Sayeed, A., & Hasan, M. A. M. (2020). Classification of American Sign Language by Applying a Transfer Learned Deep Convolutional Neural Network. ICCIT 2020 - 23rd International Conference on Computer and Information Technology, Proceedings, 19–21. https://doi.org/10.1109/ICCIT51783.2020.9392703
Indra, D., Purnawansyah, Madenda, S., & Wibowo, E. P. (2019). Indonesian Sign Language Recognition Based on Shape of Hand Gesture. Procedia Computer Science, 161, 74–81. https://doi.org/10.1016/j.procs.2019.11.101
Khaleel, M., Ahmed, A. A., & Alsharif, A. (2023). Artificial Intelligence in Engineering. Brilliance: Research of Artificial Intelligence, 3(1), 32–42. https://doi.org/10.47709/brilliance.v3i1.2170
Li, G., Tang, H., Sun, Y., Kong, J., Jiang, G., Jiang, D., Tao, B., Xu, S., & Liu, H. (2019). Hand Gesture Recognition Based on Convolution Neural Network. Cluster Computing, 22, 2719–2729. https://doi.org/10.1007/s10586-017-1435-x
Mursita, R. A. (2015). Respon Tunarungu Terhadap Penggunaan Sistem Bahasa Isyarat Indonesa (Sibi) Dan Bahasa Isyarat Indonesia (Bisindo) Dalam Komunikasi. Inklusi, 2(2), 221. https://doi.org/10.14421/ijds.2202
Noer, A. (2021). Bahasa Isyarat Indonesia (BISINDO) Alphabets. Kaggle. https://www.kaggle.com/datasets/achmadnoer/alfabet-bisindo/data
Pusbisindo. (2023). Mengapa Belajar BISINDO? Website. https://www.pusbisindo.org/#mengapa
Susanty, M., Fadillah, R. Z., & Irawan, A. (2021). Model Penerjemah Bahasa Isyarat Indonesia (BISINDO) Menggunakan Pendekatan Transfer Learning. Petir, 15(1), 1–9. https://doi.org/10.33322/petir.v15i1.1289
Toengi, R. (2018). Application of Transfer Learning to Sign Language Recognition Using an Inflated 3D Deep Convolutional Neural Network.
Triwijoyo, B. K., Karnaen, L. Y. R., & Adil, A. (2023). An Approach for Sign Language Recognition with Deep Learning Algorithm. 9(1), 1–10. https://doi.org/10.1007/978-981-99-1435-7_1
Wadhawan, A., & Kumar, P. (2020). Deep Learning-Based Sign Language Recognition System for Static Signs. Neural Computing and Applications, 32(12), 7957–7968. https://doi.org/10.1007/s00521-019-04691-y
Yin, H., Gu, Y. H., Park, C. J., Park, J. H., & Yoo, S. J. (2020). Transfer Learning-Based Search Model for Hot Pepper Diseases and Pests. Agriculture (Switzerland), 10(10), 1–16. https://doi.org/10.3390/agriculture10100439
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Engineering, MAthematics and Computer Science Journal (EMACS)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
a. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License - Share Alike that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
b. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
c. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
USER RIGHTS
All articles published Open Access will be immediately and permanently free for everyone to read and download. We are continuously working with our author communities to select the best choice of license options, currently being defined for this journal as follows: Creative Commons Attribution-Share Alike (CC BY-SA)
