Extensive Deep Learning Models Evaluation For Indonesian Sign Language Recognition

Abstract

Sign language is a vital communication method for individuals with hearing loss or deafness, with variations reflecting unique cultural contexts. Real-time recognition of sign language can bridge communication gaps, yet­­ developing tools for Indonesian Sign Language (BISINDO) is challenging due to limited datasets. This research addresses these challenges by enhancing BISINDO detection and real-time rec­­ognition, focusing on flexible dataset collection and adaptation to varying lighting conditions. Three convolutional neural networks—InceptionV3, MobileNetV2, and ResNet50—are evaluated with optimizers SGD, Adagrad, and Adam to determine the best architecture-optimizer combination. Models were trained on a common dataset and analyzed for optimal performance. Real-time recognition uses MobileNetV2 SSD, integrating data augmentation to improve performance under diverse lighting. The system was deployed on a mobile device for practical use. Results showed the real-time model attained a mean Average Precision (mAP) of 90.34%. This study demonstrates significant advancements in BISINDO recognition and real-time application