How to Control the Dino Game with Hand Gesture?
This tutorial will teach us how to control the popular Dino Game in Google Chrome using hand gestures. We will be using the cvzone library, which is a collection of computer vision functions based on OpenCV and other popular libraries.
Do you like playing the Dino game on Chrome? How cool would it be if you could control the game using just your hand movements? In this tutorial, we will learn how to do just that! We will use a special tool called the “cvzone library” along with the “HandTrackingModule” to detect your hand movements and make the game do what you want it to do. This technology is called “hand gesture recognition”. It is becoming more popular because it lets you control things without touching them. By the end of this tutorial, you’ll know how to control the Dino game with your hand gestures and you can try it out for yourself!
Prerequisites:
- Download and install Python 3.7
- Setup a virtual env in your IDE (pycharm)
- Install the required libraries
- cvzone
- pyautogui
- opencv-python
- A webcam connected to your computer or a laptop with working cam
Step 1: Importing the required modules
We start by importing the necessary modules for our project, which are cvzone, HandDetector from cvzone, cv2, and pyautogui.
import cvzonefrom cvzone.HandTrackingModule import HandDetectorimport cv2import pyautogui as auto
Step 2: Starting the video capture
Next, we need to turn on the laptop camera and start capturing the video stream. We use OpenCV’s VideoCapture function to do this.
cap = cv2.VideoCapture(0)
Step 3: Initializing the HandDetector
Now, we initialize the HandDetector object and set the detection confidence to 0.8 and maximum number of hands to 1.
detector = HandDetector(detectionCon=0.8, maxHands=1)
Step 4: Main loop
The main loop of our program starts here. It continuously captures frames from the video stream and processes them for hand gesture recognition.
while True: # Get image frame success, img = cap.read()
Step 5: Finding hands and landmarks (Hand Gesture Recognition)
We use the detector object to find hands and landmarks in the current frame. We also draw the hand landmarks on the frame.
hands, img = detector.findHands(img, draw=True)
Step 6: Checking for hand gestures
If at least one hand is detected, we check if the index finger and thumb are close enough to trigger the “jump” action in the Dino Game. We use pyautogui to simulate the “up” key press.
if hands: # Hand 1 hand1 = hands[0] HandLandMarkList1 = hand1["lmList"] # List of 21 Landmark points length,info,frame = detector.findDistance(HandLandMarkList1[4][0:2],HandLandMarkList1[8][0:2],img) length = round(length)
if length<25: auto.press('up')
The above code, calculates the distance between the tip of the thumb (index 4) and the tip of the index finger (index 8) using the findDistance() method of the HandDetector class.
How is the distance between tip of thump and index calculated?
The findDistance() method takes two points (specified as (x, y) coordinates) as input and returns the distance between them in pixels
In the given code, the HandLandMarkList1 is a list that contains the (x, y) coordinates of 21 landmark points detected by the hand detector for the first detected hand in the current frame. The landmark points are numbered from 0 to 20, where 0 corresponds to the wrist and the rest of the numbers correspond to the fingertips and joints of the fingers.
Use list index notation to access the (x, y) coordinates of a specific landmark point. For example, HandLandMarkList1[4] refers to the fifth landmark point, which corresponds to the tip of the thumb. Similarly, HandLandMarkList1[8] refers to the ninth landmark point, which corresponds to the tip of the index finger.
However, since each element of the HandLandMarkList1 is a list that contains three values (x, y, z), where z is the depth of the landmark point from the camera, we need to slice the list to only get the (x, y) coordinates. This is done using the slicing notation [0:2], which returns the first two elements of the list.
Therefore, HandLandMarkList1[4][0:2] and HandLandMarkList1[8][0:2] are the (x, y) coordinates of the tip of the thumb and the tip of the index finger respectively for the first detected hand in the current frame. These two points helps in calculating the distance between them.
Pass these two points to the findDistance() method to get the distance between them. Save this distance in a variable (length). This variable will be used later used to control the up key press of the keyboard.
Step 7: Displaying the frame
Finally, we display the current frame on the screen and wait for the user to press the “q” key to quit the program.
cv2.imshow("Image", img)if cv2.waitKey(1) == ord('q'): break
Step 8: Releasing the resources
Once the user quits the program, we release the video capture and destroy all windows.
Best-suited Data Science courses for you
Learn Data Science with these high-rated online courses
Conclusion
In this blog, we explored how to use the cvzone library and the HandTrackingModule to detect hand movements and control a game. We saw how to access the landmark points of the hand and calculate the distance between them. Using this distance, we were able to simulate pressing of the up arrow key of the keyboard. The possibilities of this library are endless. One can use it for various applications such as virtual reality, gesture recognition, and robotics. The cvzone library offers developers and enthusiasts a promising tool for hand tracking and recognition in touchless interfaces.
FAQs
What is the cvzone library, and how does it relate to hand gesture recognition?
The cvzone library is a collection of computer vision tools and utilities for use in Python programming. It includes the HandTrackingModule, which is capable of detecting and tracking hand movements in real-time. This makes it a powerful tool for hand gesture recognition applications.
How does the HandTrackingModule work, and what are its capabilities for detecting hand movements?
The HandTrackingModule uses computer vision techniques to detect and track the key points of a hand in real-time, including the fingertips, palm, and wrist. It can also estimate the orientation and movement of the hand. These capabilities make it useful for a wide range of applications, including hand gesture recognition, virtual reality, and robotics.
Can the techniques used in this tutorial be applied to other games or applications besides the Chrome Dino game?
Absolutely! The techniques used in this tutorial can be applied to any game or application that accepts keyboard inputs. With some modifications, they can also be applied to other input devices, such as a mouse or joystick.
How does hand gesture recognition technology work, and what are some of its potential future applications?
Hand gesture recognition technology works by using a combination of computer vision algorithms and machine learning techniques to detect, track, and classify hand gestures in real-time. The process usually involves capturing an image or video stream of a person's hand, extracting features from the image, and then using machine learning models to classify the features as different hand gestures. It has a wide range of potential applications, including touchless interfaces, virtual reality, gaming, robotics, and even medical applications such as rehabilitation and physical therapy. As the technology continues to improve, we can expect to see even more exciting and innovative applications in the future.
Experienced AI and Machine Learning content creator with a passion for using data to solve real-world challenges. I specialize in Python, SQL, NLP, and Data Visualization. My goal is to make data science engaging an... Read Full Bio
Top Picks & New Arrivals
