AMERICAN SIGN LANGUAGE RECOGNITION BASED ON MACHINE LEARNING AND NEURAL NETWORK

Abstract

Disabilities like deafness and muteness often hinder effective communication with people who do not
share the same condition, making it essential to develop solutions for this issue. One viable approach is Sign
Language Recognition (SLR), which employs pattern recognition techniques. This paper explores the use of
machine learning and deep learning methods to recognize and classify American Sign Language (ASL) gestures,
focusing on 24 English letters, as the letters J and Z involve finger movements that are difficult to capture. Initially,
Principal Component Analysis (PCA) and manifold algorithms are utilized for dimensionality reduction to speed up
the machine learning training process and to facilitate visualization. Subsequently, several machine learning
techniques, including Random Forest Classification (RFC), K-Nearest Neighbour (KNN), Gaussian Naïve Bayes
(GNB), Support Vector Machine (SVM), and Stochastic Gradient Descent (SGD), are employed for pattern
classification. Given that the SVM algorithm has multiple hyperparameters, Grid Search is used to identify the
optimal combination of these parameters for better prediction accuracy. The study finds that different dimensionality
reduction techniques have varying impacts on the performance of each classification model. Specifically, the
manifold algorithm proves to be the most effective for KNN, while PCA generally performs better than the manifold
algorithm for other models. Additionally, two deep learning techniques, Convolutional Neural Networks (CNN) and
Deep Neural Networks (DNN), are tested for classification, with these methods demonstrating the highest accuracy
among the algorithms examined.