Assignment 16: Image Manipulation
For this assignment, what I originally wanted to do was to use computer vision to manipulate the image on the computer (we haven’t learnt computer vision yet so this was too optimistic). Since I like Minions, I used the Minion image that I grabbed from the internet.
For the first step, I decided to make the cursor on the mouse make the flashlight on the Minion image, which is initially dark. For this, I followed Daniel Schiffman’s Processing Tutorial 10.5: Image processing with Pixels.
Code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
PImage minion; void setup(){ size(700,630); minion = loadImage("minion.jpg"); } void draw() { loadPixels(); minion.loadPixels(); for (int x = 0; x < width; x++){ for (int y = 0; y < height; y++){ int loc = x+y*width; float r = red(minion.pixels[loc]); float g = green(minion.pixels[loc]); float b = blue(minion.pixels[loc]); float d = dist(mouseX, mouseY, x,y); float factor = map(d,0,150,2,0); pixels[loc] = color(r*factor, g*factor, b*factor); } } updatePixels(); } |
Output:
For the next step, I wanted my webcam to track the brightest point/the flashlight using computer vision. I watched Daniel Schiffman’s Processing 11.1: Capture and Live Video – Processing Tutorial and 11.2: Using the Movie Object for Displaying Recorded Videos – Processing Tutorial. Since I am new to computer vision, I struggled a lot with making this work. In the end, I was able to find a code that dealt with Brightness Threshholding by Golan Levin.
Code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
import processing.video.*; color black = color(0); color white = color(255); int numPixels; Capture video; void setup() { size(640, 480); // Change size to 320 x 240 if too slow at 640 x 480 strokeWeight(5); // This the default video input, see the GettingStartedCapture // example if it creates an error video = new Capture(this, width, height); // Start capturing the images from the camera video.start(); numPixels = video.width * video.height; noCursor(); smooth(); } void draw() { if (video.available()) { video.read(); video.loadPixels(); int threshold = 127; // Set the threshold value float pixelBrightness; // Declare variable to store a pixel's color // Turn each pixel in the video frame black or white depending on its brightness loadPixels(); for (int i = 0; i < numPixels; i++) { pixelBrightness = brightness(video.pixels[i]); if (pixelBrightness > threshold) { // If the pixel is brighter than the pixels[i] = white; // threshold value, make it white } else { // Otherwise, pixels[i] = black; // make it black } } updatePixels(); // Test a location to see where it is contained. Fetch the pixel at the test // location (the cursor), and compute its brightness int testValue = get(mouseX, mouseY); float testBrightness = brightness(testValue); if (testBrightness > threshold) { // If the test location is brighter than fill(black); // the threshold set the fill to black } else { // Otherwise, fill(white); // set the fill to white } ellipse(mouseX, mouseY, 20, 20); } } |
I tried combining these two codes so that when I turned on the flashlight on my phone, my webcam would detect the light from the flashlight, which would light up the Minion image. However, this code did not work, and since this integration between image manipulation and computer vision is much more complex than I originally intended, I’ll try to achieve this after we learn about computer vision.
Failed Code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
import processing.video.*; Capture video; PImage minion; void setup() { size(700, 630); minion = loadImage("minion.jpg"); // Uses the default video input, see the reference if this causes an error video = new Capture(this, width, height); video.start(); noStroke(); smooth(); } void draw() { int brightestX = 0; // X-coordinate of the brightest video pixel int brightestY = 0; // Y-coordinate of the brightest video pixel float brightestValue = 0; // Brightness of the brightest video pixel if (video.available()) { video.read(); image(video, 0, 0, width, height); // Draw the webcam video onto the screen // Search for the brightest pixel: For each row of pixels in the video image and // for each pixel in the yth row, compute each pixel's index in the video video.loadPixels(); int index = 0; for (int y = 0; y < video.height; y++) { for (int x = 0; x < video.width; x++) { // Get the color stored in the pixel int pixelValue = video.pixels[index]; // Determine the brightness of the pixel float pixelBrightness = brightness(pixelValue); // If that value is brighter than any previous, then store the // brightness of that pixel, as well as its (x,y) location if (pixelBrightness > brightestValue) { brightestValue = pixelBrightness; brightestY = y; brightestX = x; } index++; } } // Draw a large, yellow circle at the brightest pixel fill(255, 204, 0, 128); ellipse(brightestX, brightestY, 150, 150); } loadPixels(); minion.loadPixels(); for (int x = 0; x < width; x++) { for (int y = 0; y < height; y++) { int loc = x+y*width; float r = red(minion.pixels[loc]); float g = green(minion.pixels[loc]); float b = blue(minion.pixels[loc]); float d = dist(brightestX, brightestY, x, y); float factor = map(d, 0, 150, 2, 0); pixels[loc] = color(r*factor, g*factor, b*factor); } } updatePixels(); } |
I found this tutorial on “Tracking the Brightest Point” later on, but perhaps, this can help me achieve my final outcome.