The E-Learning Application in A Classroom for Deaf-Mute Students

Abstract

Usage of the E-learning environment has become one of the most important requirements of the current era, and the general trend of students today is to deal more passionately with these environments. Sign Language Recognition (SLR) is the most common way for deaf-mute to communicate among themselves or with others. Deaf-mute students need to communicate with their teachers or their colleagues. By e-learning application, this work is trying to translate the gestures of (SLR) to audible sound. Then use the voice to address Alexa. A widely used control method for smart environments is voice control, which is very difficult for the deaf community. Thus, the proposed system describes an automatic fast, and accurate system adopted for recognizing one-hand static gestures in real-time conditions. The base is a vision-based RGB image in different environments for American Sign Language (ASL). Deep Learning (DL) technique was used by Convolution Neural Networks (CNN) architecture in three models: Alexnet, Googlenet, and Squeezenet. The used dataset in the off-time implementation of two levels of complexity: RGB images of the uniform color background and RGB images with complex backgrounds. The best accuracy achieved for a complex background dataset of (6×150) image by the Squeezenet model, was 99.85% accuracy for off-time testing. Real-time experiments gave the best accuracy of 98.593 % by Squeezenet in 0.001sec recognition time. This achievement has been exploited for smart classrooms by providing a comment for ‘Amazon Alexa’, which is a powerful voice recognition system.