1. A sketch of my concept
2. Describe, in at least a couple of paragraphs, your project
- For my final project, I will make use of computer vision to create a controller for my game, which is related to space. In this game, the player has to use a flashlight to detect and kill evil aliens or save trapped astronauts.
- Instead of choosing a particular round or mode of the game on the screen, the player will piece together LEGO bricks of some certain colors to choose the game mode. For example, if a player piece together bricks of white, red and orange, they will go to the game mode called “Mars.” I do this by telling the program to detect certain colors and remember colors that are present in the video, then I use the if statement to direct the program to a particular game mode.
- Then, the player will use the LEGO block he or she has just made as a controller. The LEGO block is actually fixed upon a handheld device that has 2 pushbuttons, one to save astronauts and one to kill aliens. As the player moves the handheld device around, the device acts as a flashlight on screen, allowing them to see who to kill and who to save. I will attach an LED on this handheld device so that the camera can track the brightest pixel and reflect it on the screen.
3. A list of parts I will need
- I will need a screen and camera(s).
- For the camera, I am not sure I need one or two. The camera that detects colors in the LEGO block should be stationary, similar to the one usually attached to the TV, so that the video will not be affected by colors not from the LEGO block. However, the camera that tracks the LED should be on top of the screen so that the user can look at the screen and move the controller at the same time.
4. A detailed list of topics I need to learn:
- 1. How to solder so that I can attach a prototype shield on an Arduino
- 2. How to shorten my code so that my program will run faster: I think for this part I will need to finish my code and then show it to you, so that you will be able to identify which parts can be replaced with a function or a kind of inexplicit calculation, for example, state = 1 – state.
5. A block diagram of my circuit
6. The 3 most difficult parts of my project
- 1. The code for the game:
For the game, I have to work with pixels which are notoriously slow to process. Therefore, my main obstacle in creating the game is to simplify the code as much as possible so that the visuals on the screen will respond instantly to the player’s movements.
- 2. Precision in color tracking
At first, I intended to offer the player 7 colors to choose from, but as I tested the code, it is difficult for the program to infallibly distinguish between orange and red, or yellowish brown and brown. Thus, I am trying to adjust the threshold at which the distance between the current color and the track color is compared to, so that the color detection can be more precise. However, in the end, I might have to use fewer and easily distinguishable colors.
- 3. Creating a user-friendly design for the controller
Initially, I intended to create a box for the controller. Yet it might be uncomfortable for the player has to hold and move a rectangular cube around. Therefore, I am thinking about a controller that is shaped like a rectangular panel bent on its left and right side so that the player has something to grip.