HomeWork for December 4th (Diego and Nav)

Project Description

This project is a collaboration between Diego and Navya. 

The main idea is to have a SpaceShip that would be controlled by the user. A webcam would read hand gestures of the user and control the SpaceShip accordingly. We would use some machine learning to accomplish this. The spaceship would essentially be a robot which would move around in a Play Area of sorts. It would have 2 wheels and a nice lego spaceship that sits on top of the wheels. It would interact with this play area and follow a certain storyline that we come up with. The storyline, of course is space based. The Play Area, or “Space” would have different “bases”, or locations on it. Each base would have a sensor and would be able to sense if the SpaceShip is at that base or not. We plan to make the Play Area out of cardboard and have some cool space designs on it, with different locations.

The main idea is to give the user a feel that they are on a space mission. By directing them to accomplish certain tasks, it gives that feel. The other aspect is to add a ‘wow’ element by being able to control the spaceship using hand movements or gestures. The two aspects combined should ideally make a fun and enjoyable experience for the user. 

Here’s a diagram that outlines the idea in brief.

Below is a breakdown of the different components of our project.

2 Laptops. 1 runs the spaceship, the other runs the play area – refer diagram above.

SpaceShip – Move around and interacts with the play area. Gets the commands from the XBee connected to the webcam processing sketch.

Items – 1 Arduino, Acrylic sheets, 2 Motors, Motor Shield, 2 Wheels, XBee shield

Play Area (space) Contains sensors that detect the presence of spaceship. Relays this information to the processing sketch.

Items – 1 Very large cardboard piece, 1 Arduino, 8 Large IR RangeFinders, Black paint/cloth

Human Interaction Controls the movement of spaceship by some hand gestures. We would use Wekinator and some very basic machine learning.

StoryLine Gives a purpose to the movement of spaceship in the play area and makes it more interactive. This would be coded in the processing sketch. 

Potential items that we would need (a compiled list)

  • 2 Arduinos (already got)
  • 2 Motors to move bot (already got)
  • MotorShield (already got)
  • 2 Wheels
  • 2 XBees (already got)
  • 1 Very huge cardboard sheet
  • 8 IR rangefinders (already got)
  • Analog Multiplexer. If this is not available, we need an Arduino Mega. This is because we need more than 8 analog inputs whereas Arduino Uno supports only 6 and the motorshield takes up 2. (Thanks Micheal for pointing this out)
  • 1 fairly big screen for displaying the storyline nicely. HDMI cable too.

Things we need to learn!

  • How to use the MotorShield
  • How to use the XBees (done!)
  • How to use the Arduino Mega (if we might need to use it)
  • A little more about the wekinator and the machine learning part

Tools we might need to use

Nothing as of yet.

3 hardest elements of our project!!

  • Construction of the SpaceShip
  • Coding the hand gestures
  • Creating the sequence of events in the play area and coding the storyline

We have not included a Block Diagram of the circuit because we are not sure at this point if we are using a multiplexer or a arduino mega. Once that is clear, we will update this post. 

Final Project Proposal

This project is a collaboration between Diego and I. After a short discussion we realized our projects had a lot in common and collaborating would not only make it more complex, but also more interesting as a final project for the both of us. 

The main idea is to have a SpaceShip that would be controlled by the user. This spaceship would essentially be a robot which would move around in a Play Area of sorts. It would interact with this play area and follow a certain storyline that we come up with. The storyline, of course is space based. The SpaceShip would be controlled using a webcam and some hand gestures which we would read via a webcam. 

Here’s a diagram that outlines the idea in brief, followed by a list of the individual components and potential items needed.



2 Laptops. 1 runs the spaceship, the other runs the play area – refer diagram above.

SpaceShip – Move around and interacts with the play area.

Runs on Arduino code if processing cannot connect to XBee

Items – 1 Arduino, Acrylic sheets, 2 Motors, Motor Shield, 2 Wheels – choose the right ones, XBee shield on it for reception of commands

Play Area (space) Contains sensors that detect the presence of spaceship

Items – 1 Very large cardboard piece, 1 Arduino, Sensors (depending on number of bases), Black paint/cloth

Human Interaction Controls the movement of spaceship

Items – WebCam?

StoryLine Gives a purpose to the movement of spaceship in the play area and makes it more interactive


Homework due Nov 27 – Images

For this assignment we had to manipulate an image. I grabbed this image of a landscape from the internet and chose to work on it. 

I tried many things to figure out what I wanted my image to do. I even tried playing with the P3D, and some 3D manipulation techniques. Yet, I wasn’t very satisfied with how it looked. So I decided to try and manipulate the image dynamically. I used the mouseX and mouseY positions to play with my image, and stuck to using the pixels array. 

I applied a bit of a posterize filter and some amount of blur just to get the effect I wanted. What I aimed for was to be able to move the mouse around in screen and see how the image can be viewed as though we are viewing the scene in different climates. It took a while to get the numbers right, but I think this comes close to what I wanted. 

I also tried to then make some sort of pointillization on the images which would also happen dynamically with the mouseX and mouseY positions. That didn’t work out too well, so I stuck to just changing the weather. 

Here’s my code – 


Computing [due Nov 22]

What Computing Means to Me!

I perceive computing in two broad senses. The first is to think of computing as a tool that enhances the creativity or the power of our mind by the use of a machine. This could mean performing complex calculations or doing some advanced data analysis. The other way is to think of computing is a way of thinking and doing. It might need translation of a real world problem into something that can be worked on and solved (which would use computing in the first sense). However, regardless of how I consider, I strongly believe that computing is making my life easier, and as a result, better. I also believe that this effect that computing has is not restricted to individuals, but to communities and larger bodies as well.
In the state that the world is today, there is hardly anywhere we can go that has not been touched by the magic of technology. Computing sees itself in every corner of the world. The idea is that computes are essentially dumb, meaning they cannot analyse and understand what to do. They perform tasks as we command. This would see us having a very close connection with computing, considering that all the work is assigned by us, but in a way that the machine can understand and perform accordingly. The added advantages are obvious – they can perform tasks faster that our minds can process. Another observation is how we do not realize the complexity of a simple task we assign to the computer – how it essentially breaks everything down to a few signals and acts accordingly. I think there is a certain sense of beauty in it. Computing obviously means a lot me, and I believe that in the future, the possibilities we create with this magical tool are endless.

Two Player Ping Pong – Game with Processing and Arduino (due Nov 6)

For this assignment, I tried to make a ping pong game. I wanted to keep it as simple as possible.


My main challenge was to understand the physics of the game. I found myself repeatedly drawing paddles and balls on pieces of paper, trying to come up with different cases and add them to the code. I also had a lot of trouble with the ending screen. I had to eventually use a trick to make it work. The next challenge was to display the scores. I originally wanted to pause for 2 seconds if someone scored and display both players score in large font on the screen, but that didn’t work out for me. I eventually stuck to having small scoreboards at the corners of the screens – as you’ll see below. I also didn’t want the game to be way too easy. To add a bit of challenge, I decided to use flex sensors as the controllers, which would make the paddles quite unstable. 

The Game

This is how the game looked like – 

The Code



Lastly, my apologies for the late submission. As I’d informed earlier, I wasn’t well. 

Assignment due Nov 1

For today’s assignment we had 2 tasks. The first was to control the movement of a servo motor through processing. The second was to control the LED state using a mouseClick in processing.

Part 1

This assignment was very similar to the Dimmer example. A few modifications and you could control the servo with ease. The key step was the use of the map() function. The code is pasted below: 

Processing – 

Arduino – 


Part 2 

For this part, the key step was identifying whether the mouseX and mouseY values were within the correct rectangle and returning an appropriate value to the port which could then be read by the arduino. 

Processing – 

Arduino – 


Art in Processing (due Oct 30th, 2017)

For this assignment, I did a lot of exploring. I found the topic of “Generative Art” extremely interesting. A few examples can be found here. My inspiration came from here. I really wanted to make my art sync with music, but it seemed too complex for now. So I just stuck to creating some moving graphics. There are many components to it, as you can see in the code below: 

This is how it finally looked: 


Many thanks to Abdullah for giving me a heads up on the Screen Recording software, IceCream.

Self Portrait and Rectangles (due Oct 16th)

For my self portrait, I tried to make a happy bald version of me. It really helped me to draw it out on paper and figure out how to do there first. After that, it was a simple matter of coding and getting the placements right. I think it worked out pretty fine.


Here’s my code – 

And this is what I got out of playing with the for loop!

Here’s the code


Happy HiFive Human (due Oct 11th)

For this assignment, I created a Happy Human (I know, right?) who would spin around and give hi-fives to everyone around! The only issue I faced was not connecting the circuit properly and problems in turning my Happy Human because the motor offered very less torque. Here’s how he looked – 


Materials Used 

  • 1 x 9V DC Motor
  • 1 x Transistor
  • 1 x Breadboard
  • 1 x RedBoard
  • Foam
  • Wood (a small piece)
  • Cardboard
  • Wires




Playing with LED strips (due Oct 9th)

For my whimsical project I intended to make a sort of digital LED sequence and hence I decided to use LED strips.

Few things that I found out after researching and exploring the internet which were very helpful (regarding LED strips)

  • The LED Strips that we use are called ws2812b strips
  • To program them, we need the help of Libraries – the two best ones available right now are Adafruit’s NeoPixel Library and FastLED. After going through both Libraries a bit, I found FastLED to be less complex, hence I have used that. 
  • It is very important to connect the Digital Output pin from the LED strip to a 330 ohm resistor to prevent damage to the LEDs

My would seem quite complex, and I’ve tried to organize it best as I can. I did the rainbow sequence part with the help of the internet, the rest was all me. Here it is

Although the Rainbow part may seem very complex, in essence it is really simple – but to type an explanation of how it works would take quite a while, and I’m not sure if I would be able to do it. So, if anyone needs help in understanding that part, let me know, I’ll be glad to help in whatever way possible. 

Here’s the circuit diagram:

Here’s how it finally worked out: 

Part 2 – Reviewing neighbor’s assignment (Jahnae)

Jahnae built this really cool funky piano which had a mind of its own. Pressing a key would not necessarily mean that the keyboard would play the corresponding note. Sometimes, it would just deny your command of pressing the button to play the note (angry piano). The denial was expressed by the movement of a Servo motor, which looked something like it was shaking its head. To answer all the questions, it was amusing and unexpected and delightful. The signifiers were the letters of the notes written above the keyboard. Of course, they were helpful. The affordances were in my opinion the buttons themselves, clearly meant to be pushed. However, the result, which one would expect that the corresponding note be played, is quite surprising when the keyboard “denies” you. That, in my opinion, is what makes it unexpected and amusing.