GAT: Gestural Artificial-intelligence Technology is a collection of 2 AI models that can recognize human movements using a camera and perform specific actions based on specific human movements. It is divided into two models, one that is completely optimized in the browser, and one that can be run on your computer as an app. With these models, developers can map hand gestures to specific actions.
Desktop Application(Python): After you download the file, you need to install the dependencies. You can install the dependencies by running the following commands into your
terminal.
pip install openCv
pip install Mediapipe
pip install autopy
pip install numpy
pip install pynput
After you install the dependencies, you run the main.py file, and grant the application access to your camera.
Our First Model(Website Optimized): For our first model, we used the TensorFlow js hand library to track the hand movement of a user. We also used a camera library to track the hand itself. Using the different points associated with the user's hands, we created gesture functionality on our mock HTML webpages. These gestures include moving the mouse and webpage using your hands, and in our example, we liked images on a social media platform using gestures.
Our Second Model(Computer System Optimized): Our second Model was made using python. the Open-Cv python library was used to access the camera of the hosting device. The python Mediapipe library was used to enable hand detection by our AI. Using the Mediapipe Mediapipe library, we were able to create an AI module that recognizes hands, identifies the fingers up, and detects the distance between fingers. Using these methods, we were able to train our AI to associate different hand gestures with different operations. To effectively demonstrate the experience our AI provides, we created a messaging app and made it compatible with our model.
The future for GAT is filled with endless possibilities. We plan to further develop the GAT model to recognize hand gestures representing the American Sign Language figures. We plan to integrate this into an American sign language translator and learning app. This will ease the communication between people with disabilities and teach people how to communicate using this medium. In addition, we plan to create a GAT model that allows users to add their gestures to their version of the GAT and map these gestures to operations. Lastly, we plan to create a web extension version of the GAT model. This will allow users to experience navigating their web browsers with hand gestures. We are certain that the GAT model will evolve into something beautiful. Infinity is the limit for the future of GAT.