Machine Learning Used to Close the Communication Gap through a Linguistic Tool for Deaf People

Abstract

Currently, there is a wide variety of resources and tools on the internet for integrating artificial intelligence into languagerelated projects. This article proposes the use of machine learning to address and reduce the communication gap between hearing people and deaf people who use sign language as a form of communication. For this purpose, the Edge Impulse tool, which facilitates the implementation of machine learning models on mobile devices, was employed. The SinSeñas2.0 project emerged as a response to this need and is based on the Extreme Programming (XP) methodology to understand user needs and offer an effective solution. A dataset of 3102 images of Colombian signs was collected, divided into 80% for training and 20% for testing. Convolutional neural networks (CNN) and deep learning techniques were used to train the model, which improved the accuracy in recognizing signs. The results showed that model configuration 1, with an accuracy of 99% and a loss of 0.03%, was the most effective. This configuration used an input size of 96x96 and employed transfer learning with the MobileNet V2 neural network. The tool also included data augmentation techniques to create a balanced and diversified dataset, thereby improving the model’s robustness against different capture conditions. The research demonstrates that SinSeñas2.0 significantly improves the accuracy and efficiency of sign language recognition compared to previous approaches that did not use machine learning. This advancement not only facilitates communication between hearing and deaf people but also represents a significant contribution to sign language translation technology, promoting the social inclusion of deaf individuals.