Colombian Sign Language Interpretation Model using Artificial Intelligence
Abstract
In this work, two interpretation models of Colombian Sign Language (CSL) are presented, using static and dynamic methods that employ artificial intelligence. The CRISP-DM methodology was used as a reference, creating a database with videos from seventy non-expert participants, being preprocessed and subsequently divided into proportions of 70% - 30% for training and testing, respectively. The repository was named LSC-W70 and was used on a pre-trained model of convolutional neural networks and another in combination with LSTM networks. The results reached 67% and 76% accuracy for the static and dynamic models, respectively, where the dynamic model presents improvements in similar signs by identifying the direction of movement to define the type of sign. In this sense, a dynamic Colombian sign language interpretation tool was developed that helps close communication gaps, generating equality between people.
Keywords
colombian sign language;, CNN;, LSTM;, CRISP-DM
Author Biography
Jader Alejandro Muñoz-Galindez
Ingeniero Físico
Rubiel Vargas-Cañas
Ingeniero de Sistemas, PhD. in Biomedical Engineering
References
- Boháček, M., & Hrúz, M. (2022). Sign Pose-based Transformer for Word-level Sign Language Recognition. 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), 182-191. https://doi.org/10.1109/WACVW54805.2022.00024 DOI: https://doi.org/10.1109/WACVW54805.2022.00024
- Díaz, C., Goycoolea, M., & Cardemil, F. (2016). HIPOACUSIA: TRASCENDENCIA, INCIDENCIA Y PREVALENCIA. Revista Médica Clínica Las Condes, 27 (6), 731-739. https://doi.org/10.1016/j.rmclc.2016.11.003 DOI: https://doi.org/10.1016/j.rmclc.2016.11.003
- Galvis-Serrano, E. H., Sánchez-Galvis, I., Flórez, N., & Zabala-Vargas, S. (2019). Clasificación de Gestos de la Lengua de Señas Colombiana a partir del Análisis de Señales Electromiográficas utilizando Redes Neuronales Artificiales. Información Tecnológica, 30 (2), 171-180. https://doi.org/10.4067/S0718-07642019000200171 DOI: https://doi.org/10.4067/S0718-07642019000200171
- IBM. (febrero 27, 2021). Documentation. https://www.ibm.com/docs/en/cloud-paks/cp-data/3.0.1?topic=overview-accuracy
- INSOR, (2021). Informe técnico Estado Goce en Derechos de la Población sorda 2019.
- INSOR. (2011). Diccionario básico de la lengua de señas colombiana. Imprenta nacional de Colombia. http://www.insor.gov.co/descargar/diccionario_basico_completo.pdf
- Jimenez, G., Moreno, E., Guzman, R., & Barrero, J. (2019). Automatic method for Recognition of Colombian Sign Language for vowels and numbers from zero to five by using SVM and KNN. 2019 Congreso Internacional de Innovación y Tendencias En Ingeniería (CONIITI). https://doi.org/10.1109/coniiti48476.2019.8960695 DOI: https://doi.org/10.1109/CONIITI48476.2019.8960695
- Konstantinidis, D., Dimitropoulos, K., & Daras, P. (2018). A deep learning approach for analyzing video and skeletal features in sign language recognition. 2018 IEEE International Conference on Imaging Systems and Techniques (IST). https://doi.org/10.1109/ist.2018.8577085 DOI: https://doi.org/10.1109/IST.2018.8577085
- Martínez, F., Robayo-Betancourt, F., & Arbulú, M. (2020). A gesture recognition system for the Colombian sign language based on convolutional neural networks. Bulletin of Electrical Engineering and Informatics, 9 (5), 2082–2089. https://doi.org/10.11591/eei.v9i5.2440 DOI: https://doi.org/10.11591/eei.v9i5.2440
- Mindlin, I. (2021). Reconocimiento de Lengua de Señas con redes neuronales recurrentes. Universidad Nacional de La Plata. http://sedici.unlp.edu.ar/handle/10915/129853
- Monsalve-Pineda, K. V., & Polo-Álvarez, J. E. (2016). Traductor de símbolos de alfabeto del lenguaje de signos colombiano al lenguaje escrito. Universidad del Valle. https://bibliotecadigital.univalle.edu.co/handle/10893/17348
- Ortiz-Farfán, N., & Camargo-Mendoza, J. E. (2020). Modelo computacional para reconocimiento de lenguaje de señas en un contexto colombiano. TecnoLógicas, 23 (48), 197–232. https://doi.org/10.22430/22565337.1585 DOI: https://doi.org/10.22430/22565337.1585
- Suat-Rojas, N. E., Montoya-Serna, B. S., Pinzón-Velásquez, E. M., & Rodríguez-Galeano, O. S. (2021). Reconocimiento del abecedario de la lengua de señas colombiana con Redes Neuronales Convolucionales. Orinoquia, 25 (1), 25-30. https://doi.org/10.22579/20112629.680 DOI: https://doi.org/10.22579/20112629.680
- Vaghasiya, D. (2021). Dynamic gesture classification of American Sign Language using deep learning. Laurentian University of Sudbury. https://zone.biblio.laurentian.ca/jspui/handle/10219/3843
- Vasilev, I., Slater, D., Spacagna, G., Roelants, P., & Zocca, V. (2019). Python Deep Learning: Exploring Deep Learning Techniques and Neural Network Architectures with PyTorch, Keras, and TensorFlow. Packt Publishing.