Utilizing hand landmarks and MediaPipe for deep neural network-based recognition of signal for help

Abstract

The unsafe condition is a problem that has greatly affected the quality of life of people in society, especially those people with hearing loss which accounts for the second largest number of all disabled people. The unsafe conditions may take place physically and mentally, as well as problems of family violence that may occur at home, school, or public places, thereby vastly affected people of all groups and all ages. Thus, the issue is attributed to the real-time “signal for help” hand gesture recognition development with the aid of the MediaPipe to track the location. The victim can ask for help in an emergency case without words or shouting out. The objectives of the study include: 1) to detect the “signal for help” hand gestures and provide notifications, and 2) to examine the performance of the “signal for help” hand gesture recognition. The MediaPipe framework is used and practiced with 30,000 images dataset and tested by the obtained video data to detect six “signal for help” hand gestures, each gesture of 100 images, a total of 600 images. The palm and finger coordinates are configured into a Python programming language. When “signal for help” hand gestures are detected, a notification is provided to the administrator. The test results found that the accuracy of hand gestures 1-6 of the dataset ranged between 95-99%, and the accuracy was approximately 97%.