Processing
Thinking about other ways of exhibiting my pictures,
reflections and collection
Here I am showing the process I went through to create the visuals that are showing on my CRT monitor. The processing sketches, the Arduino sketch and the Audition process for the music implemented later on, together with a small video also implemented on the processing sketch.
How it started:
I started thinking about what other ways I could do to explore my pictures. I thought of using also something analog, that conveyed all the ideas I’ve been exploring so far.
I thought of CRT monitors, their image have special grain and texture. I thought about making some visuals that would bring me to the cycles of raving. This was the main focus at the beginning of my project and, although it did contribute to the reflections of my overall idea, I have yet to explore this in a visual sense. I wrote my DH&T on raving as a queer person. The research undertaken for it also informed how I perceive raving: the altered states we reach through music, dancing and lights (sometimes with the use of drugs).
I read a paper on raving as a shamanic experience, McKenzie Wark and her book “Raving” where she narrates very well the states I mentioned above. I wanted to explore colour, light and movement with this processing project so I started thinking what elements I wanted to add to my sketch.
For starters, I wanted to create some visuals similars to those from William Latham that made me think about this 3d visual loops that looked like pc screensavers from the 90s. Below are some reference pictures (and a video) of his artwork.
I set up the code thinking about spheres first and using colour, wanted to potentially make the spheres use some of the pixels from my pictures as the visuals evolved. That created the ground to start thinking on how to implement that feedback loop that characterised the visuals I was talking about. Below I’m showing the code for that and how they looked like.
Processing sketches ++ Arduino:
Trip01:
PImage grab;
void setup() {
fullScreen(P3D);
colorMode(HSB);
grab = createImage(width, height, RGB);
background(0);
//hint(DISABLE_DEPTH_TEST);
//blendMode(ADD);
};
void draw() {
background(0);
translate(width/2, height/2, -200);
//imageMode(CENTER);
//tint(255, 32);
//filter(DILATE);
//image(grab, 0, 0, width/2, height/2);
//stroke(255);
//strokeWeight(3);
noStroke();
//fill(frameCount%255, 255, 255);
lights();
//directionalLight(255, 255, 255, .5, 0, -1);
//ambientLight(153, 102, 0);
//ambient(51, 26, 0);
sphereDetail(8);
float n = 16;
float aStep = 360/n;
for (int i = 0; i < n; i++) {
float a = radians(i*aStep);
pushMatrix();
rotateZ(a);
drawTendril();
popMatrix();
}
//grab = get();
specular(255);
}
void drawTendril() {
pushMatrix();
for (int i = 0; i < 50; i++) {
fill((frameCount + i*4)%255, 255, 255, 255);
float x = sin((frameCount*i)*0.0025)*5;
float z = sin((frameCount*i)*0.0017)*100;
translate(x, -20, z);
float s = (100 - i*10);
rotateY(radians(frameCount*i*0.05)*0.1);
rotateX(radians(frameCount*i*0.03)*0.07);
//box(s*10, s*0.1, s*0.1);
shininess(50.0);
sphere(s);
}
popMatrix();
}
After this, I started thinking on how to implement sound into processing so I made a different sketch to test adding an Ultrasonic sensor. I wanted to implement reactiveness to movement so potentially the ultrasonic sensor could be my best option here.
AudioPitcher01:
import processing.sound.*;
SoundFile soundfile;
LowPass lowPass;
HighPass highPass;
BandPass bandPass;
//import ch.bildspur.vision.*;
//import ch.bildspur.vision.result.*;
//import processing.video.Capture;
//Capture cam;
//DeepVision vision;
//ULFGFaceDetectionNetwork network;
//ResultList detections;
void setup() {
size(640, 480);
setupSerial();
soundfile = new SoundFile(this, "audioTest01_mixdown.wav");
soundfile.loop();
//lowPass = new LowPass(this);
//lowPass.process(soundfile);
//highPass = new HighPass(this);
//highPass.process(soundfile);
//bandPass = new BandPass(this);
//bandPass.process(soundfile);
}
void draw() {
background(0);
doSerialInput();
float playbackSpeed = constrain(map(num, 0, 50, 1, 0.05), 0.05, 1);
soundfile.rate(playbackSpeed);
//float playbackSpeed = map(mouseX, 0, width, 0.05, 1);
//soundfile.rate(playbackSpeed);
//float cutoff = map(mouseY, 0, height, 20, 10000);
//lowPass.freq(cutoff);
//cutoff = map(mouseY, height, 0, 10, 10000);
//highPass.freq(cutoff);
// Map the left/right mouse position to a cutoff frequency between 20 and 10000 Hz
//float frequency = map(mouseX, 0, width, 20, 5000);
//// And the vertical mouse position to the width of the band to be passed through
//float bandwidth = map(mouseY, 0, height, 1000, 100);
//bandPass.freq(frequency);
//bandPass.bw(bandwidth);
}Serial:
// LIBRARIES AND GLOBAL VARIABLES ----------------------
import processing.serial.*; // Import the Processing Serial library
Serial port; // Create a new Serial object
String val; // Create a String to hold the value from the Serial
float num, lNum; // Create a float to hold the number extracted from the String
float di = 10.0; // Create a float to control the size of the circle
// SETUP ------------------------------------------------
void setupSerial() {
printArray(Serial.list()); // Print a list of available Serial ports
port = new Serial(this, "/dev/cu.usbmodem101", 9600); // Set up your Serial port
/* The port you use should be the same as the one your Arduino is connected to. You
will need to change the number in the square brackets to select the right port for
your computer. */
}
// INPUT --------------------------------------------------
void doSerialInput() {
if (port.available() > 0) { // If data is available on the Serial port...
val = port.readStringUntil('\n'); // ... read it in and store it in val
/* Note: The value from the Serial port will always be a String - so you will need
to convert it to an int or float. */
val = trim(val); // Trim whitespace from the beginning and end of the String
}
if (val != null) { // If val does not equal null...
num = float(val); // ... extract the number sent from Arduino
di = map(lNum, 0, 50, 0, width); // Map the number to a useful range and store it in di
/* Note: A lot of sensors won't return a full range of values here, so for best results
you might need to do some calibration. */
//println(di); // Print the new value to the console
}
lNum = lerp(lNum, num, 0.025);
fill(255);
noStroke();
circle(width/2, height/2, di); // Draw a circle scaled based on the data from Arduino
}
After this, I implemented a feedback loop on the code that would make the visuals look more like those I had in mind at the beginning. Those 90s rave visuals // early 2000’s screensaver. This gave more depth to the overall visuals.
tripFeedback01:
PImage grab;
void setup() {
//size(960, 540, P3D);
fullScreen(P3D);
colorMode(HSB);
grab = createImage(width, height, RGB);
background(0);
noCursor();
}
void draw() {
background(0);
drawFeedback();
drawAllTendrils();
grabFrameForFeedback();
}
void grabFrameForFeedback() {
if (frameCount%2 == 0) {
grab = get();
}
}
void drawFeedback() {
blendMode(BLEND);
pushMatrix();
translate(width/2, height/2, -200);
imageMode(CENTER);
tint(255, 248);
//float s = map(mouseX, 0, width, 0, 1.5); // or mouse!
float s = map(noise(frameCount*0.001), 0, 1, 0.5, 1.5); // random feedback based on scale of grabbed image
image(grab, 0, 0, width*s, height*s);
popMatrix();
}
void drawAllTendrils() {
pushMatrix();
translate(width/2, height/2, 0);
noStroke();
lights();
sphereDetail(8);
float n = 16;
float aStep = 360/n;
for (int i = 0; i < n; i++) {
float a = radians(i*aStep);
pushMatrix();
rotateZ(a);
drawTendril();
popMatrix();
}
popMatrix();
}
void drawTendril() {
pushMatrix();
for (int i = 0; i < 50; i++) {
fill((frameCount + i*4)%255, 255, 255, 255);
float x = sin((frameCount*i)*0.0025)*5;
float z = sin((frameCount*i)*0.0017)*100;
translate(x, -20, z);
float s = (25 - i*0.5);
rotateY(radians(frameCount*i*0.05)*0.1);
rotateX(radians(frameCount*i*0.03)*0.07);
//box(s*10, s*0.9, s*4.1);
shininess(50.0);
sphere(s);
}
popMatrix();
}
Then, I started thinking on how to implement my pictures onto the visuals. Thought about making the spheres change using the colours of my pictures, but then I decided I wanted to try using another form. Maybe a cube?
tripFeedbackCube:
int IMAGES = 43;
PImage[] images;
PImage grab;
void setup() {
//size(960, 540, P3D);
fullScreen(P3D);
colorMode(HSB);
grab = createImage(width, height, RGB);
background(0);
noCursor();
setupImages();
}
void draw() {
background(0);
drawFeedback();
drawAllTendrils();
grabFrameForFeedback();
}
void grabFrameForFeedback() {
if (frameCount%2 == 0) {
grab = get();
}
}
void drawFeedback() {
blendMode(BLEND);
pushMatrix();
translate(width/2, height/2, -200);
imageMode(CENTER);
tint(255, 248);
//float s = map(mouseX, 0, width, 0, 1.5); // or mouse!
float s = map(noise(frameCount*0.001), 0, 1, 0.5, 1.5); // random feedback based on scale of grabbed image
image(grab, 0, 0, width*s, height*s);
popMatrix();
}
void drawAllTendrils() {
pushMatrix();
translate(width/2, height/2, 0);
noStroke();
lights();
sphereDetail(8);
float n = 16;
float aStep = 360/n;
for (int i = 0; i < n; i++) {
float a = radians(i*aStep);
pushMatrix();
rotateZ(a);
drawTendril();
popMatrix();
}
popMatrix();
}
void drawTendril() {
pushMatrix();
for (int i = 0; i < 50; i++) {
int fillColour = (int(frameCount * 0.1) + i*4)%255;
//fill(, 255, 255, 255);
float x = sin((frameCount*i)*0.0025)*5;
float z = sin((frameCount*i)*0.0017)*100;
translate(x, -20, z);
float s = (25 - i*0.5);
rotateY(radians(frameCount*i*0.05)*0.1);
rotateX(radians(frameCount*i*0.03)*0.07);
//box(s*10, s*0.9, s*4.1);
shininess(50.0);
//sphere(s);
s *= 5;
float progress = map(fillColour, 0, 255, 0, IMAGES);
PImage currentImage = images[floor(progress)];
PImage nextImage = images[ceil(progress)];
float imageWidth1 = s;
float imageHeight1 = (currentImage.width / currentImage.height)* s;
float imageWidth2 = s;
float imageHeight2 = (nextImage.width / nextImage.height)* s;
//tint(255, (progress % 1) * 255);
//image(currentImage, 0, 0, imageWidth1, imageHeight1);
//tint(255, (1 - progress % 1) * 255);
//image(nextImage, 0, 0, imageWidth2, imageHeight2);
//pushMatrix();
//scale(s);
createTextureCube(currentImage, imageWidth1, imageHeight1);
//popMatrix();
}
popMatrix();
}
void setupImages() {
images = new PImage[IMAGES];
for (int i = 0; i < IMAGES; i++) {
images[i] = loadImage("tripImage" + str(i) + ".png");
}
}
void createTextureCube(PImage textureImage, float cubeWidth, float cubeHeight) {
push();
// noStroke();
// noFill();
textureMode(NORMAL);
int colour = textureImage.get(0, 0);
beginShape();
texture(textureImage);
// +Z "front" face
vertex(-1 * cubeWidth, -1 * cubeHeight, 1 * cubeWidth, 0, 0);
vertex( 1 * cubeWidth, -1 * cubeHeight, 1 * cubeWidth, 1, 0);
vertex( 1 * cubeWidth, 1 * cubeHeight, 1 * cubeWidth, 1, 1);
vertex(-1 * cubeWidth, 1 * cubeHeight, 1 * cubeWidth, 0, 1);
endShape();
fill(colour);
beginShape();
// -Z "back" face
vertex( 1 * cubeWidth, -1 * cubeHeight, -1* cubeWidth, 0, 0);
vertex(-1 * cubeWidth, -1 * cubeHeight, -1* cubeWidth, 1, 0);
vertex(-1 * cubeWidth, 1 * cubeHeight, -1* cubeWidth, 1, 1);
vertex( 1 * cubeWidth, 1 * cubeHeight, -1* cubeWidth, 0, 1);
endShape();
beginShape();
//texture(textureImage);
// +Y "bottom" face
vertex(-1 * cubeWidth, 1 * cubeWidth, 1* cubeWidth, 0, 0);
vertex( 1 * cubeWidth, 1 * cubeWidth, 1* cubeWidth, 1, 0);
vertex( 1 * cubeWidth, 1 * cubeWidth, -1* cubeWidth, 1, 1);
vertex(-1 * cubeWidth, 1 * cubeWidth, -1* cubeWidth, 0, 1);
endShape();
beginShape();
//texture(textureImage);
// -Y "top" face
vertex(-1 * cubeWidth, -1 * cubeWidth, -1* cubeWidth, 0, 0);
vertex( 1 * cubeWidth, -1 * cubeWidth, -1* cubeWidth, 1, 0);
vertex( 1 * cubeWidth, -1 * cubeWidth, 1* cubeWidth, 1, 1);
vertex(-1 * cubeWidth, -1 * cubeWidth, 1* cubeWidth, 0, 1);
endShape();
beginShape();
//texture(textureImage);
// +X "right" face
vertex( 1 * cubeWidth, -1 * cubeHeight, 1* cubeWidth, 0, 0);
vertex( 1 * cubeWidth, -1 * cubeHeight, -1* cubeWidth, 1, 0);
vertex( 1 * cubeWidth, 1 * cubeHeight, -1* cubeWidth, 1, 1);
vertex( 1 * cubeWidth, 1 * cubeHeight, 1* cubeWidth, 0, 1);
endShape();
beginShape();
//texture(textureImage);
// -X "left" face
vertex(-1 * cubeWidth, -1 * cubeHeight, -1* cubeWidth, 0, 0);
vertex(-1 * cubeWidth, -1 * cubeHeight, 1* cubeWidth, 1, 0);
vertex(-1 * cubeWidth, 1 * cubeHeight, 1* cubeWidth, 1, 1);
vertex(-1 * cubeWidth, 1 * cubeHeight, -1* cubeWidth, 0, 1);
endShape();
pop();
}
I don’t know why the cubes weren’t implementing the images in all the faces, so kept trying to tweak it. I thought about losing the cube and just keeping one of the faces only, maybe as a plane? I did that on the first sketch I’m sharing below (left). I then applied some transparency on the pictures to make them blend a bit between them. I realised they looked quite glued together, so tried to make a cube with only one face visible (By keeping the other faces of the cube transparent (I was trying to give it some depth). So tried to do that on the second sketch shared (right).
tripFeedbackPicz:
int IMAGES = 43;
PImage[] images;
PImage grab;
void setup() {
//size(960, 540, P3D);
fullScreen(P3D);
colorMode(HSB);
grab = createImage(width, height, RGB);
background(0);
noCursor();
setupImages();
}
void draw() {
background(0);
drawFeedback();
drawAllTendrils();
grabFrameForFeedback();
}
void grabFrameForFeedback() {
if (frameCount%2 == 0) {
grab = get();
}
}
void drawFeedback() {
blendMode(BLEND);
pushMatrix();
translate(width/2, height/2, -200);
imageMode(CENTER);
tint(255, 248);
//float s = map(mouseX, 0, width, 0, 1.5); // or mouse!
float s = map(noise(frameCount*0.001), 0, 1, 0.5, 1.5); // random feedback based on scale of grabbed image
image(grab, 0, 0, width*s, height*s);
popMatrix();
}
void drawAllTendrils() {
pushMatrix();
translate(width/2, height/2, 0);
noStroke();
lights();
sphereDetail(8);
float n = 16;
float aStep = 360/n;
for (int i = 0; i < n; i++) {
float a = radians(i*aStep);
pushMatrix();
rotateZ(a);
drawTendril();
popMatrix();
}
popMatrix();
}
void drawTendril() {
pushMatrix();
for (int i = 0; i < 50; i++) {
float fillColour = ((millis() * 0.001) + i*4)%255;
//fill(, 255, 255, 255);
float x = sin((frameCount*i)*0.0025)*5;
float z = sin((frameCount*i)*0.0017)*100;
translate(x, -20, z);
float s = (25 - i*0.5);
rotateY(radians(frameCount*i*0.05)*0.1);
rotateX(radians(frameCount*i*0.03)*0.07);
//box(s*10, s*0.9, s*4.1);
shininess(50.0);
//sphere(s);
s *= 5;
float progress = map(fillColour, 0, 255, 0, IMAGES);
PImage currentImage = images[floor(progress)];
PImage nextImage = images[ceil(progress) % IMAGES];
float imageWidth1 = s;
float imageHeight1 = (currentImage.width / currentImage.height)* s;
float imageWidth2 = s;
float imageHeight2 = (nextImage.width / nextImage.height)* s;
tint(255, (1 - progress % 1) * 255);
image(currentImage, 0, 0, imageWidth1, imageHeight1);
tint(255, (progress % 1) * 255);
image(nextImage, 0, 0, imageWidth2, imageHeight2);
//pushMatrix();
//scale(s);
//createTextureCube(currentImage, imageWidth, imageHeight);
//popMatrix();
}
popMatrix();
}
void setupImages() {
images = new PImage[IMAGES];
for (int i = 0; i < IMAGES; i++) {
images[i] = loadImage("tripImage" + str(i) + ".png");
}
}
void createTextureCube(PImage textureImage, float cubeWidth, float cubeHeight) {
push();
noStroke();
noFill();
textureMode(NORMAL);
int colour = textureImage.get(0, 0);
beginShape();
texture(textureImage);
// +Z "front" face
vertex(-1 * cubeWidth, -1 * cubeHeight, 1 * cubeWidth, 0, 0);
vertex( 1 * cubeWidth, -1 * cubeHeight, 1 * cubeWidth, 1, 0);
vertex( 1 * cubeWidth, 1 * cubeHeight, 1 * cubeWidth, 1, 1);
vertex(-1 * cubeWidth, 1 * cubeHeight, 1 * cubeWidth, 0, 1);
endShape();
fill(colour);
beginShape();
// -Z "back" face
vertex( 1 * cubeWidth, -1 * cubeHeight, -1* cubeWidth, 0, 0);
vertex(-1 * cubeWidth, -1 * cubeHeight, -1* cubeWidth, 1, 0);
vertex(-1 * cubeWidth, 1 * cubeHeight, -1* cubeWidth, 1, 1);
vertex( 1 * cubeWidth, 1 * cubeHeight, -1* cubeWidth, 0, 1);
endShape();
beginShape();
//texture(textureImage);
// +Y "bottom" face
vertex(-1 * cubeWidth, 1 * cubeWidth, 1* cubeWidth, 0, 0);
vertex( 1 * cubeWidth, 1 * cubeWidth, 1* cubeWidth, 1, 0);
vertex( 1 * cubeWidth, 1 * cubeWidth, -1* cubeWidth, 1, 1);
vertex(-1 * cubeWidth, 1 * cubeWidth, -1* cubeWidth, 0, 1);
endShape();
beginShape();
//texture(textureImage);
// -Y "top" face
vertex(-1 * cubeWidth, -1 * cubeWidth, -1* cubeWidth, 0, 0);
vertex( 1 * cubeWidth, -1 * cubeWidth, -1* cubeWidth, 1, 0);
vertex( 1 * cubeWidth, -1 * cubeWidth, 1* cubeWidth, 1, 1);
vertex(-1 * cubeWidth, -1 * cubeWidth, 1* cubeWidth, 0, 1);
endShape();
beginShape();
//texture(textureImage);
// +X "right" face
vertex( 1 * cubeWidth, -1 * cubeHeight, 1* cubeWidth, 0, 0);
vertex( 1 * cubeWidth, -1 * cubeHeight, -1* cubeWidth, 1, 0);
vertex( 1 * cubeWidth, 1 * cubeHeight, -1* cubeWidth, 1, 1);
vertex( 1 * cubeWidth, 1 * cubeHeight, 1* cubeWidth, 0, 1);
endShape();
beginShape();
//texture(textureImage);
// -X "left" face
vertex(-1 * cubeWidth, -1 * cubeHeight, -1* cubeWidth, 0, 0);
vertex(-1 * cubeWidth, -1 * cubeHeight, 1* cubeWidth, 1, 0);
vertex(-1 * cubeWidth, 1 * cubeHeight, 1* cubeWidth, 1, 1);
vertex(-1 * cubeWidth, 1 * cubeHeight, -1* cubeWidth, 0, 1);
endShape();
pop();
}tripFeedbackCubeSmooth:
int IMAGES = 43;
PImage[] images;
PImage grab;
void setup() {
//size(960, 540, P3D);
fullScreen(P3D);
colorMode(HSB);
grab = createImage(width, height, RGB);
background(0);
noCursor();
setupImages();
}
void draw() {
background(0);
drawFeedback();
drawAllTendrils();
grabFrameForFeedback();
}
void grabFrameForFeedback() {
if (frameCount%2 == 0) {
grab = get();
}
}
void drawFeedback() {
blendMode(BLEND);
pushMatrix();
translate(width/2, height/2, -200);
imageMode(CENTER);
tint(255, 248);
//float s = map(mouseX, 0, width, 0, 1.5); // or mouse!
float s = map(noise(frameCount*0.001), 0, 1, 0.5, 1.5); // random feedback based on scale of grabbed image
image(grab, 0, 0, width*s, height*s);
popMatrix();
}
void drawAllTendrils() {
pushMatrix();
translate(width/2, height/2, 0);
noStroke();
lights();
sphereDetail(8);
float n = 16;
float aStep = 360/n;
for (int i = 0; i < n; i++) {
float a = radians(i*aStep);
pushMatrix();
rotateZ(a);
drawTendril();
popMatrix();
}
popMatrix();
}
void drawTendril() {
pushMatrix();
for (int i = 0; i < 50; i++) {
float fillColour = ((millis() * 0.001) + i*4)%255;
//fill(, 255, 255, 255);
float x = sin((frameCount*i)*0.0025)*5;
float z = sin((frameCount*i)*0.0017)*100;
translate(x, -20, z);
float s = (25 - i*0.5);
rotateY(radians(frameCount*i*0.05)*0.1);
rotateX(radians(frameCount*i*0.03)*0.07);
//box(s*10, s*0.9, s*4.1);
shininess(50.0);
//sphere(s);
s *= 5;
float progress = map(fillColour, 0, 255, 0, IMAGES);
PImage currentImage = images[floor(progress)];
PImage nextImage = images[ceil(progress) % IMAGES];
float imageWidth1 = s;
float imageHeight1 = (currentImage.width / currentImage.height)* s;
float imageWidth2 = s;
float imageHeight2 = (nextImage.width / nextImage.height)* s;
tint(255, (1 - progress % 1) * 255);
image(currentImage, 0, 0, imageWidth1, imageHeight1);
tint(255, (progress % 1) * 255);
image(nextImage, 0, 0, imageWidth2, imageHeight2);
//pushMatrix();
//scale(s);
//createTextureCube(currentImage, imageWidth, imageHeight);
//popMatrix();
}
popMatrix();
}
void setupImages() {
images = new PImage[IMAGES];
for (int i = 0; i < IMAGES; i++) {
images[i] = loadImage("tripImage" + str(i) + ".png");
}
}
At this point I decided that it had more potential to use a video of my pictures instead of making processing run through them from a folder, so I headed over to After Effects and created a quick video sequence of my pictures that would loop as the cube texture (I decided that a 3D form would look better than only the images as well).
After Effects:
It was quite straightforward. I used the Keyframe Assistant > Sequence layers.
I chose 57 pictures (mix between my own and AI again). And created a sequence by having colour as the main important factor when organising them. I wanted them to no jum from one colour to the opposite one, so tried to create a sort of “colour gradient transition”. I cropped them into a square (because they will go in a cube).
Below I am showing how the sequence looked like:
More processing sketches:
This one is implementing the cubes and the video:
tripFeedbackCubes01:
PImage grab;
float cubeSize = 3;
PImage[] images;
int imageCount = 10;
int imageIndex = 0;
int delay = 180;
void setup() {
//size(960, 540, P3D);
fullScreen(P3D);
colorMode(HSB);
textureMode(NORMAL);
setupVideo();
grab = createImage(width, height, RGB);
background(0);
noCursor();
}
void draw() {
background(0);
drawFeedback();
drawAllTendrils();
grabFrameForFeedback();
}
void grabFrameForFeedback() {
if (frameCount%2 == 0) {
grab = get();
}
}
void drawFeedback() {
blendMode(BLEND);
pushMatrix();
translate(width/2, height/2, -200);
imageMode(CENTER);
tint(255, 248);
//float s = map(mouseX, 0, width, 0, 1.5); // or mouse!
float s = map(noise(frameCount*0.001), 0, 1, 1.25, 1.5); // random feedback based on scale of grabbed image
image(grab, 0, 0, width*s, height*s);
popMatrix();
}
void drawAllTendrils() {
pushMatrix();
translate(width/2, height/2, 0);
noStroke();
noLights();
//sphereDetail(8);
float n = 16;
float aStep = 360/n;
for (int i = 0; i < n; i++) {
float a = radians(i*aStep);
pushMatrix();
rotateZ(a);
drawTendril();
popMatrix();
}
popMatrix();
}
void drawTendril() {
pushMatrix();
for (int i = 0; i < 50; i++) {
//fill((frameCount + i*4)%255, 255, 255, 255);
float x = sin((frameCount*i)*0.0025)*5;
float z = 0;//sin((frameCount*i)*0.0017)*100;
translate(x, -20, z);
//float s = (25 - i*0.5);
rotateY(radians(frameCount*i*0.05)*5);
rotateX(radians(frameCount*i*0.03)*20.07);
scale(cubeSize);
texturedCube(movie);
}
popMatrix();
}
void texturedCube(PImage _p) {
beginShape(QUADS);
texture(_p);
// +Z "front" face
vertex(-1, -1, 1, 0, 0);
vertex( 1, -1, 1, 1, 0);
vertex( 1, 1, 1, 1, 1);
vertex(-1, 1, 1, 0, 1);
// -Z "back" face
vertex( 1, -1, -1, 0, 0);
vertex(-1, -1, -1, 1, 0);
vertex(-1, 1, -1, 1, 1);
vertex( 1, 1, -1, 0, 1);
// +Y "bottom" face
vertex(-1, 1, 1, 0, 0);
vertex( 1, 1, 1, 1, 0);
vertex( 1, 1, -1, 1, 1);
vertex(-1, 1, -1, 0, 1);
// -Y "top" face
vertex(-1, -1, -1, 0, 0);
vertex( 1, -1, -1, 1, 0);
vertex( 1, -1, 1, 1, 1);
vertex(-1, -1, 1, 0, 1);
// +X "right" face
vertex( 1, -1, 1, 0, 0);
vertex( 1, -1, -1, 1, 0);
vertex( 1, 1, -1, 1, 1);
vertex( 1, 1, 1, 0, 1);
// -X "left" face
vertex(-1, -1, -1, 0, 0);
vertex(-1, -1, 1, 1, 0);
vertex(-1, 1, 1, 1, 1);
vertex(-1, 1, -1, 0, 1);
endShape();
}
Video:
import processing.video.*;
Movie movie;
void setupVideo() {
movie = new Movie(this, "cubeVideo.mp4");
movie.loop();
}
void movieEvent(Movie m) {
m.read();
}
I then implemented the audio (I used a tester song) and, the movement and size of the cubes to be dependant on the speed of the song. Here I also resized the screen to match the CRT monitor’s aspect ratio. (There is sound on the below video if you want to turn it on to watch it):
tripFeedbackCubes01:
PImage grab;
float cubeSize = 3;
PImage[] images;
int imageCount = 10;
int imageIndex = 0;
int delay = 180;
//New variable
float playbackSpeed = 1.0; //ADDED
void setup() {
//size(960, 540, P3D);
fullScreen(P3D);
colorMode(HSB);
textureMode(NORMAL);
setupVideo();
setupSound(); //ADDED
grab = createImage(width, height, RGB);
background(0);
noCursor();
}
void draw() {
background(0);
doSound(); //ADDED
drawFeedback();
drawAllTendrils();
grabFrameForFeedback();
//ADDED
PImage q = get();
q.resize(1800, 720);
background(0);
image(q, width/2, height/2);
}
void grabFrameForFeedback() {
if (frameCount%2 == 0) {
grab = get();
}
}
void drawFeedback() {
blendMode(BLEND);
pushMatrix();
translate(width/2, height/2, -200);
imageMode(CENTER);
tint(255, 248);
//float s = map(mouseX, 0, width, 0, 1.5); // or mouse!
float s = map(noise(frameCount*0.001), 0, 1, 1.25, 1.5);
image(grab, 0, 0, width*s, height*s);
popMatrix();
}
void drawAllTendrils() {
pushMatrix();
translate(width/2, height/2, 0);
noStroke();
noLights();
float n = 16;
float aStep = 360/n;
for (int i = 0; i < n; i++) {
float a = radians(i*aStep);
pushMatrix();
rotateZ(a);
drawTendril();
popMatrix();
}
popMatrix();
}
void drawTendril() {
pushMatrix();
for (int i = 0; i < 50; i++) {
//MODIFIED: made audio reactive using playbackSpeed
float x = sin((frameCount*i)*0.0025)*5*playbackSpeed; //CHANGED
float z = 0; // unchanged
translate(x, -20, z);
//MODIFIED: rotation speed scaled by playbackSpeed
rotateY(radians(frameCount*i*0.05)*5*playbackSpeed); //CHANGED
rotateX(radians(frameCount*i*0.03)*20.07*playbackSpeed); //CHANGED
//MODIFIED: scaled cubeSize by playbackSpeed
scale(cubeSize*playbackSpeed); //CHANGED
texturedCube(movie);
}
popMatrix();
}
void texturedCube(PImage _p) {
beginShape(QUADS);
texture(_p);
// Unchanged cube face definitions
vertex(-1, -1, 1, 0, 0);
vertex( 1, -1, 1, 1, 0);
vertex( 1, 1, 1, 1, 1);
vertex(-1, 1, 1, 0, 1);
vertex( 1, -1, -1, 0, 0);
vertex(-1, -1, -1, 1, 0);
vertex(-1, 1, -1, 1, 1);
vertex( 1, 1, -1, 0, 1);
vertex(-1, 1, 1, 0, 0);
vertex( 1, 1, 1, 1, 0);
vertex( 1, 1, -1, 1, 1);
vertex(-1, 1, -1, 0, 1);
vertex(-1, -1, -1, 0, 0);
vertex( 1, -1, -1, 1, 0);
vertex( 1, -1, 1, 1, 1);
vertex(-1, -1, 1, 0, 1);
vertex( 1, -1, 1, 0, 0);
vertex( 1, -1, -1, 1, 0);
vertex( 1, 1, -1, 1, 1);
vertex( 1, 1, 1, 0, 1);
vertex(-1, -1, -1, 0, 0);
vertex(-1, -1, 1, 1, 0);
vertex(-1, 1, 1, 1, 1);
vertex(-1, 1, -1, 0, 1);
endShape();
}
Video:
import processing.video.*;
Movie movie;
void setupVideo() {
movie = new Movie(this, "cubeVideo.mp4");
movie.loop();
}
void movieEvent(Movie m) {
m.read();
}
Audio
import processing.sound.*;
SoundFile soundfile;
float playbackSpeed = 0;
void setupSound() {
soundfile = new SoundFile(this, "audioTest01_mixdown.wav");
soundfile.loop();
//lowPass = new LowPass(this);
//lowPass.process(soundfile);
//highPass = new HighPass(this);
//highPass.process(soundfile);
//bandPass = new BandPass(this);
//bandPass.process(soundfile);
}
void doSound() {
float num = map(mouseX, 0, width, 0, 50);
playbackSpeed = constrain(map(num, 0, 50, 1, 0.05), 0.05, 1);
soundfile.rate(playbackSpeed);
}
Audition // choosing the songs for the processing sketch:
For the music, I used a tester tune that I associate with nostalgia and cycles. I tend to listen to it on repeat and it was my top song on my spotify wrapped of 2024. It brings me feelings of raving as this dream I’ve never reached, or did I and I just can’t remember anymore? This feelings are similar to those that I experience with analog pictures. The nostalgia, the melancholy, the safety. Feelings that feel abstract and big, but very real. That validate my existance and my experience on this world.
I thought about just looping that one song, it goes really well with the visuals and the repetitiveness is a good metaphore of those feelings, but at the same time... a song that is 1 minute long (at 0.5x speed is 2minutes long) looping in the gallery can be annoying.
I made a small selection of songs on a folder, they are dark and hypnotic techno songs. I thought they could work well with the visuals as well, althought the songs are dark and industrial and the visuals are more like 90s trance.
I then made a selection on another folder of more nostalgic songs, they are more acid, trancey techno and are more socially associated with the visuals I made.
I did some tests on how each song would sound slowed down and realised most of the ones from the “DARK” folder sounded like darth vader breathing, and not interesting at all. Didn’t work out with my piece.
DARK music for processing
NOSTALGIC music for processing
I liked most of the ones from my “NOSTALGIC” selection, but some of them just sounded like guitar strings. That sounded a bit silly and didn’t like it. What I liked the moest about my “tester song” was the “high pitched bell sounds”, they reminded me of starts.
** These folders were made through a selection of songs from 2 of my Spotify playlists:
DARK music (some examples):
NOSTALGIC music:
I tested songs from those two playlists because the theme of each one informed the overall idea of my project. Those from the “Extra-terrestrial techno” are songs that I connect directly to this idea of “looking for other worlds” that initially Detroit techno embodied, which is a feeling that I feel quite often when I go raving. And the “Internet-aided nostalgia” are songs that, although most of them are quite recent, they feel like “a blast from the past”. They definetely bring me nostalgia from experiences I never had.
Audition // Creating a “session” for the processing sketch:
As I mentioned above, I started by testing all the different songs I had in each folder to understand better how they sounded from x0.1-0.5 speed, and which ones would help me achieve the feelings I was looking for together with my Processing sketch.
A few examples:
This is the sequence final sequence I made in audition, I basically panned the tracks to create a sequence. If I have a bit of time before submission, I’d like to do more of a “Dj set” but right now, sounds good and it does convey the emotions I talked about, so I am happy with it. I will upgrade it for Degree show for sure, though.
Final result:
For the final result, I implemented the “set” I made in Audition and did a few final changes to the code. Below I added the code with the implementation of the Serial (that connects to the ultrasonic sensor) - so I’ll also add the Arduino code.
FINAL1:
PImage grab;
float cubeSize = 3;
PImage[] images;
int imageCount = 10;
int imageIndex = 0;
int delay = 180;
float time = 0;
void setup() {
size(960, 540, P3D);
fullScreen(P3D, 2);
colorMode(HSB);
textureMode(NORMAL);
setupSerial();
setupVideo();
setupSound();
grab = createImage(width, height, RGB);
background(0);
noCursor();
}
void draw() {
background(0);
time += playbackSpeed;
doSound();
//if (millis() > 3000) {
doSerialInput();
//}
drawFeedback();
drawAllTendrils();
grabFrameForFeedback();
PImage q = get();
background(0);
image(q, width/2, height/2, 1800, 720);
}
void grabFrameForFeedback() {
if (frameCount%2 == 0) {
grab = get();
}
}
void drawFeedback() {
blendMode(BLEND);
pushMatrix();
translate(width/2, height/2, -200);
imageMode(CENTER);
//tint(255, 248);
//float s = map(mouseX, 0, width, 0, 1.5); // or mouse!
float s = map(noise(time*0.001), 0, 1, 1.25, 1.5); // random feedback based on scale of grabbed image
image(grab, 0, 0, width*s-0.5, height*s-0.5);
popMatrix();
}
void drawAllTendrils() {
pushMatrix();
translate(width/2, height/2, 0);
noStroke();
noLights();
//sphereDetail(8);
float n = 16;
float aStep = 360/n;
for (int i = 0; i < n; i++) {
float a = radians(i*aStep);
pushMatrix();
rotateZ(a);
drawTendril();
popMatrix();
}
popMatrix();
}
void drawTendril() {
pushMatrix();
for (int i = 0; i < 50; i++) {
//fill((frameCount + i*4)%255, 255, 255, 255);
float pS = map(playbackSpeed, 0.01, 0.5, 0.5, 1);
float x = sin((time*i)*0.0025)*5*playbackSpeed+0.05;
float z = cos((time*i)*0.0017)*100*playbackSpeed;
translate(x, -20, z);
//float s = (25 - i*0.5);
//float invPlayback = map(playbackSpeed, 0.01, 0.5, 5, 0.5);
rotateY(radians(frameCount*i*0.05)*5*playbackSpeed+0.05);
rotateX(radians(frameCount*i*0.03)*20.07*playbackSpeed+0.05);
scale(cubeSize*pS/1.5);
texturedCube(movie);
}
popMatrix();
}
void texturedCube(PImage _p) {
beginShape(QUADS);
texture(_p);
// +Z "front" face
vertex(-1, -1, 1, 0, 0);
vertex( 1, -1, 1, 1, 0);
vertex( 1, 1, 1, 1, 1);
vertex(-1, 1, 1, 0, 1);
// -Z "back" face
vertex( 1, -1, -1, 0, 0);
vertex(-1, -1, -1, 1, 0);
vertex(-1, 1, -1, 1, 1);
vertex( 1, 1, -1, 0, 1);
// +Y "bottom" face
vertex(-1, 1, 1, 0, 0);
vertex( 1, 1, 1, 1, 0);
vertex( 1, 1, -1, 1, 1);
vertex(-1, 1, -1, 0, 1);
// -Y "top" face
vertex(-1, -1, -1, 0, 0);
vertex( 1, -1, -1, 1, 0);
vertex( 1, -1, 1, 1, 1);
vertex(-1, -1, 1, 0, 1);
// +X "right" face
vertex( 1, -1, 1, 0, 0);
vertex( 1, -1, -1, 1, 0);
vertex( 1, 1, -1, 1, 1);
vertex( 1, 1, 1, 0, 1);
// -X "left" face
vertex(-1, -1, -1, 0, 0);
vertex(-1, -1, 1, 1, 0);
vertex(-1, 1, 1, 1, 1);
vertex(-1, 1, -1, 0, 1);
endShape();
}
Audio:
import processing.sound.*;
SoundFile soundfile;
float playbackSpeed = 0;
float playbackSpeedGoGoGo = 0;
float currentVolume = 0.0;
void setupSound() {
soundfile = new SoundFile(this, "audiocubito3.mp3");
soundfile.loop();
//lowPass = new LowPass(this);
//lowPass.process(soundfile);
//highPass = new HighPass(this);
//highPass.process(soundfile);
//bandPass = new BandPass(this);
//bandPass.process(soundfile);
}
void doSound() {
//playbackSpeed = constrain(map(lNum, 0, 50, 0.5, 0.05), 0.05, 0.2);
playbackSpeed = constrain(map(lNum, 0, 100, 0.05, 0.5), 0.05, 0.2);
//float num = map(mouseX, 0, width, 0, 50);
// playbackSpeed = constrain(map(num, 0, 50, 0.5, 0.05), 0.05, 0.2);
playbackSpeedGoGoGo = lerp(playbackSpeedGoGoGo, playbackSpeed, 0.005);
soundfile.rate(playbackSpeedGoGoGo);
float volume = constrain(map(playbackSpeedGoGoGo, 0.01, 0.5, 1.0, 0.2), 0.2, 1.0);
currentVolume = lerp(currentVolume, volume, 0.005); // 0.05 is the easing factor
soundfile.amp(currentVolume);
//soundfile.amp(volume);
}
Video:
import processing.video.*;
Movie movie;
void setupVideo() {
movie = new Movie(this, "cubevideo2.mp4");
movie.loop();
}
void movieEvent(Movie m) {
m.read();
}
Video:
// ULTRASONIC SENSOR
int TRIG = 3;
int ECHO = 2;
int DURATION;
int DISTANCE;
void setup() {
// ULTRASONIC SENSOR pinMode(TRIG, OUTPUT);
pinMode(ECHO, INPUT);
// SERIAL
Serial.begin(9600);
}
void loop() {
digitalWrite(TRIG, LOW); delayMicroseconds(2); digitalWrite(TRIG,HIGH);
delay(5);
digitalWrite(TRIG,LOW);
DURATION = pulseIn(ECHO,HIGH);
DISTANCE = DURATION / 58.2;
if(DISTANCE > 0 && DISTANCE < 100 ){
Serial.println(DISTANCE);
delay(50);
}
}
Serial:
// LIBRARIES AND GLOBAL VARIABLES ----------------------
import processing.serial.*; // Import the Processing Serial library
Serial port; // Create a new Serial object
String val; // Create a String to hold the value from the Serial
float num = 0;
float lNum = 0;// Create a float to hold the number extracted from the String
//float di = 10.0; // Create a float to control the size of the circle
// SETUP ------------------------------------------------
void setupSerial() {
//printArray(Serial.list()); // Print a list of available Serial ports
port = new Serial(this, "/dev/cu.usbmodem1301", 9600); // Set up your Serial port
/* The port you use should be the same as the one your Arduino is connected to. You
will need to change the number in the square brackets to select the right port for
your computer. */
//delay(100); // Delay to let the Serial settle
}
// INPUT --------------------------------------------------
void doSerialInput() {
if (port.available() > 0) { // If data is available on the Serial port...
val = port.readStringUntil('\n'); // ... read it in and store it in val
//println(val);
/* Note: The value from the Serial port will always be a String - so you will need
to convert it to an int or float. */
val = trim(val); // Trim whitespace from the beginning and end of the String
}
if (val != null) { // If val does not equal null...
num = float(val); // ... extract the number sent from Arduino
//di = map(lNum, 0, 50, 0, width); // Map the number to a useful range and store it in di
/* Note: A lot of sensors won't return a full range of values here, so for best results
you might need to do some calibration. */
//println(di); // Print the new value to the console
}
if (int(num) != 0) {
lNum = num;
}
//lNum = lerp(lNum, num, 0.5);
//lNum = constrain(lNum, 0, 100);
//lNum = lerp(lNum, num, 0.5);
//lNum = int(lNum);
//println(lNum);
//fill(255);
//noStroke();
//circle(width/2, height/2, di); // Draw a circle scaled based on the data from Arduino
}
There is a problem with the “set” I made in Audition at the time of submission. The problem is that the set is too long, so what I will do for Degree Show is to make a shorter set (I was going to improve it anyway).
The final result with the audio tester are below!