Must Dance

Must Dance: Hawk Hacks 2024

Part 1: Inspiration

What was our inspiration?

We were tired of seeing people quit trying to be healthy through exercise, so we devised a robot that helps you exercise - or else.

Part 2: What is it?

What does our game do?

Our game can be split into 3 categories. 

1. Timed

A pose will be shown on screen, and you will need to align yourself to that pose. Getting a pose will give you a point and show you a new pose. Your final score is the total amount of points amassed in 30 seconds.

2. Lives

A pose will be shown on screen, and you will have 3 seconds to match the pose. Failure to do so will result in a life being lost. Losing 3 lives will lose the game, and as a punishment a rubber ball will be shot out of our trusty robot Musty.

3. Pose creation

A simple creation of poses. It will countdown on screen, and the final pose at the end of the countdown will be turned into a new pose that will be added into the Timed and Lives mode.

We hope that this game will incentivize people to put real effort into exercising with the fear of punishment by the hand of Musty. 

Part 3: Software

How does it work?

Pose generator works by taking a camera capture of the screen and processing it with the object Pose from mediapipe. An image is created with a segmentation mask overlay of the human subject. The inside of the subject is rewritten to black while everything else is white. This segmented image is saved as a pose.

Libraries used for pose generator

1. cv2
2. mediapipe
3. numpy
4. os
5. win32api

The main simulation has 6 "modes": start/calibration page, main menu, turret mode, timed mode, make-pose mode, and the ending page. When ran, the program begins at the start page where the user must align their body parts to a predetermined silohoutte (check pose generator). It uses the Pose object from mediapipe to identify landmarks. Blue lines are drawn to show the users figure. The user passes calibration if all landmarks are within the silohouette. The user is then brought to the main menu where they use the nose landmark to select from making a custom pose, playing timed mode, playing turret mode, and exiting.

Custom poses are made using a slightly modified (changed to fit properly in the while loop) version of Pose_Generator (explained previously). This pose is added to Poses folder where the other modes can access the new pose

Timed mode is a mode where the user must match as many positions as they can in 30 seconds. Poses from the pose folder are used as an overlay to show the user what to match to. As a little bit of help, Landmarks turn from red to green as they move from outside to inside the regions of interests. Score is tracked by the number of poses and poses switch when matched correctly.

Turret/lives mode is a mode where users are given 3 seconds to match to a pose. A life is loss for each failed attempt. Once all the lives are used up, you get shot by foam balls. This works very similarly to the Timed mode with a few changes in set up. It is a match when all landmarks are within the regions of interests (displayed as an overlay). "Serial_Communication.py" tells the turret when to fire, details are in hardware.

The end page displays the score and offers a button to return to the main menu.

Libraries used for the game

• cv2
• mediapipe
• numpy
• time
• random
• os
• serial
• playsound
• win32api

Challenges Encountered

1. Landmark coordinates out of range

Landmarks are supposed to be a decimal number from 0 to 1. However we were getting negative numbers and numbers greater than 1, so we experienced many issues with range

When landmarks are off screen, mediapipe has an approximation of where they are leading to results that do not fit the postcondition

2. Serial communication

When trying to get our Arduino circuit and Python program to communicate through USB, we ran into many errors where the port was busy or connected improperly

When we were first trying to send signal from the Python program to the Arduino program, the signal sent improperly as we needed to add a delay after opening the serial port to properly send signals

Part 4: Hardware

How does it work?

Balls are put into the loader, and are fitted to be pushed by the servo motor into the barrel. After a signal from the computer, the Arduino board sends a signal command through the relay module to power on the DC motor. It also powers the servo motor and the dot matrix module to change the facial expression. The mechanism attached to the servo allows one ball to be pushed at a time, and then immediately sequence into the next. After entering the barrel, it is spun forward by the DC motor, being shot at the player.

Materials

• Cardboard
• 1x 3D printer
• 1x DC motor
• 1x 9V battery
• 1x Keyestudio 8x8 Dot Matrix
• Foam golf balls
• Wires (M-M, M-F)
• 1x Arduino UNO board
• 1x Sunfounder Clutch Gear Servo
• 1x Songle relay circuit
• 1x Elegoo breadboard

Challenges Encountered

1. Designing a mechanism to effectively store and launch balls.

Proper DC motor selection and postioning on the barrel, taking into consideration the number of motors to use, maintaining a steady connection with the ball, and optimizing the spin of the ball for travel time.

Loading up multiple balls within the system; designing a mechanism to both push balls into the barrel, and allowing the next one in.

Pushing balls into the barrel with the servo. Had many issues with snugly fitting balls within the barrel, while still being wide enough to not cause jamming.

2. Issues with electrical wiring.

Powering motors with adequate supply of voltage, seperate from the servo motor. Another circuit system was made, while still being connected to the same breadboard/Arduino board.

Implementing a relay module into the circuit, which allows a signal to be sent from the Arduino to power on the DC motor.

Overall issues with sorting out breadboard connections; many wires led to a confusing and unorganized wire system.

Devpost Accounts

• Sunni Xue: sunni-x-2869
• Brian Yin: Brian256
• John Weng: jweng1712
• Eric Rao: n/a