Final Project Atoka Jo

Assignment 19: Final Project

My project was about interacting with the Minion that was displayed on the TV screen using the webcam. Since I like Minions and I am interested in improving education for children, I wanted to create something that involved Minions, and would be appealing to children (whether it was in an educational context or in a recreational context). I used computer vision to accomplish this final project. Since I attended the workshop on AI, machine learning, and computer vision, and I felt that there was so much potential in this field, I wanted to use this technology in my final project. The computer vision was done through a software called “Wekinator,” which acted as the medium between the processing input and the processing output. I decided to use the design principle where the explanation of the final project, the video captured by the camera, and the Minion image, would be displayed all in one big TV screen. Initially, I was not going to have the video captured by the camera to be shown on the screen. However, during the process of building my final project, I found out that I was sometimes unsure of whether the object that I was showing to the camera was actually captured by the camera or not, and so, in the end, I decided to include the video as well.

Default: Unhappy Minion

When a banana is shown: Happy Minion 1

When a snack box is shown: Happy Minion 2

When the camera is covered (dark): Surprise! Pirate Minion


  1. 2 objects (banana and snack box)
  2. TV screen
  3. Webcam
  4. Computer

For this project, I wanted to do something that used minimal materials and involved computer vision. This is because I had already made a game using Arduino and created a box of three buttons where I used my soldering and hardware skills that I learnt in class. Although this was not for any project, since I created this, I wanted to do something completely different for my final project.

Box with buttons

Other photos showing the process of making this

In order to complete the final project, I had to learn how to use the Wekinator software, and sending inputs and outputs using Processing. Some of the useful websites I used were

  1. Wekinator Home
  2. GMU: Tutorials/Performance Platform/Recognizing Gestures with Wekinator

Input Codes (to capture the webcam video):

Output Codes (to display the different Minions and its sounds):

Wekinator Setup:

The three most difficult parts of my project was understanding how the pixels were read in processing, learning how the Open Sound Control (OSC) message worked, and dealing with the sound files. Although we touched on some basic video manipulation in class, it was difficult to understand how the codes for sending the pixels to the Wekinator were managed through processing. Moreover, Open Sound Control was something that we had not learnt in class, and so I had to read a lot of websites and look at tutorials to learn how the OSC worked. Since I wanted to include different Minion sounds for each Minion image, I downloaded video clips of Minions from YouTube, cut the parts that were necessary, and used Audacity to convert them into a readable format. In the beginning, my codes were not reading the sound and even though I tried different kinds of ways to reformat the sound and change my codes, nothing worked. It was only until I asked Professor Pierre for help, and he told me how I had to use Audacity to make the sound file in a mono format to make it recognizable by processing.

Now that I have completed the final project and have seen the reactions of the people who played with my “Make the Minion Happy!” I would use Minion videos instead of Minion images if I could do it again differently. Although using the images came to my mind at first, perhaps, it may have been more amusing if the Minions were in short video clips and the users were able to watch the Minion react/move/dance in different ways to different objects.

Video 1 of the Final Project

Video 2 of the Final Project

Video 3 of the Final Project


Leave a Reply

Your email address will not be published. Required fields are marked *