My friend Kevin Yeung and I made a webcam-based version of the Wii's WarioWare using OpenCV and OpenGL for our computer vision class final project. This page is a recreation of the original writeup, which can be found here.
Our final project was a collection of mini-games that are played through a webcam and a visual cue. Current systems that allow for this type of gaming are the Nintendo Wii, Playstation Move, and Microsoft Kinect. The system created relies entirely on visible light, has a minimally cumbersome visual cue, and uses a "cheap" vision system thus making it a simple home-brewed alternative to these commercial systems.
Running this game requires OpenCV and OpenGL. The main function is in cvgl13.cpp. The classes (for the blocks, balls, etc.) are in cvglClasses.h. Don't forget to put it in the same folder as the game.
The game can be compiled using g++ with the following command: g++ `pkg-config --libs opencv --cflags opencv` -lglut -lGLU -lm [filename].cpp -o [output file]
By default, the game tracks orange objects. This can be changed by selecting the brown square. Make sure the desired color is in the middle of the cross-hair when the timer reaches 0.
Move the paddle to bounce the ball and break the blocks at the top of the screen. You get more points for breaking more blocks.
Move into the green zone to start the game. Try to get to the blue zone without hitting the red maze. You will get more points for completing the maze faster.
Move into the green zone to start the game. When the red target appears, move quickly to the target. You will only get points for winning.
Trace the shape on the screen before time runs out. You will only get points for winning. See Bucket video for demo.
The demo from class:
The game is constructed as a finite state machine. Selecting a game changes the state and dictates what frame will be shown on the screen. When a game finishes, the state is reset to the menu state and the players switch. Scores are held in global variables and are updated by each game. The scores are displayed on the menu screen using something like a 7 segment display.
The classes are abstractions of OpenGL classes. There are four classes used
The pointer class. Detects the center of mass of the pixels that fall inside an HSV threshold and draws a square there.
The button class. Can detect when the pointer location falls inside the button. Holds a binary value indicating whether the button is being pressed or not.
The ball class. Has values for location and velocity. Updates position on screen in each frame based on old position and velocity.
The block class. Is very similar to the button class but can interact with balls. Holds a binary value indicating whether it is in contact with a ball.
The game performance is highly dependent on the computer's ability to process frames quickly. It is found that slower computers have a difficult time running the game at a reasonable frame rates. There may be a way to change how the frames are updated to be less computationally straining.
The color tracking algorithm is rather naive and points to the center of mass of pixels that fall inside an HSV threshold. This could be improved through the application of an algorithm that tracks features instead of just colors. This would make the tracking more robust and the games smoother.
We were able to meet most of our objectives for this project. The games were playable during the demo and color calibration was not a hassle. The only thing that was particularly disappointing was the performance on some computers.