Live Coding Blog

Group Members: Aalya, Alia, Xinyue, Yeji

Final Presentation

https://drive.google.com/file/d/14vo1y2KyfkNQ7snYH1QXRpyS5YRDYc8a/view

Video Link

Inspiration

The initial inspiration for our final project was different for the audio and the visuals. For the audio, we were inspired by our previous beat drop assignment and had a small segment that we wanted to use for the final and built on that. For the visuals, we wanted to have a colorful yet minimalistic output and so we started off with a colorful circle with black squiggly lines, and decided to create variations of it. 

Who did what:

• Tidal: Aalya and Xinyue
• Hydra: Alia and Yeji

Project Progress

Tidal Process

For this project, we used a similar approach in developing our sounds. This began by laying out a few different audio samples and listening to them. Then we pieced together what sounds we thought fit best and that eventually turned into a specific theme for our performance. Once we set the foundation, it was then easier to break down the performance into a few main parts:

1. Intro
2. Verse
3. Pre-Chorus / Build Up
4. Chorus / Beat Drop
5. Bridge / Transition to outro
6. Outro

The next step in the music creating process was to figure out the intensity of the beats, and how we wanted specific parts to sound. How do we manipulate the sounds to have a more effective build up & beat drop, how do we make sure that the transitions are smooth and that the sounds aren’t choppy, etc… These are all questions that came up when we were in the brainstorming process.  

Some of the more prominent melodies found in our music consisted of:

 jux rev $ every 1 (slow 2) $ struct "t(3,8) t(5,8)" $ n "<0 1>" # s "supercomparator" # gain 0.9 # krush 1.3 # room 1.1

jux rev $ every 1 (slow 2) $ struct "t(3,8)" $ n "1" # s "supercomparator" # gain 0.9 # krush 1.3 # room 1.1 # up "-2 1 0"  # pan (fast 4 sine) # leslie 8struct "t*4" $ n "1" # s "supercomparator" # gain 1.3 # krush 1.3 # room 1.1 # up "-2 1 0"  # pan (fast 4 sine) # lpf (slow 4 (1000*sine + 100))  n "f'min ~ [df'maj ef'maj] f'min ~ ~ ~ ~" # s "superpiano" # gain 0.8 # room 2 

Next, it was just a matter of developing the sounds further. For example, we had a specific melody that we particularly wanted to stand out, so in order to support it, we layered sounds and effects onto it until it developed into something we liked. That was the case for all the music parts. It was all about balance and smooth transitions, forming a cohesive auditory piece that also complimented and went well with the visuals. 

Hydra Process

We started with a spinning colorful image (based on a color theme that we like) bounded by a circle shape. We then added noise to have moving squiggly lines at the center and decided to have that as our starting visual.

Alia and Yeji then started to experiment with this visual separately on different outputs and had all screens rendered so we can see them all at the same time and decide on what we like and what we don’t. 

At first it was purely based on visuals and finding ones we liked before coordinating with Aalya and Xinyue to match the visuals to the beats in the audio. 

We then started testing out taking different outputs as sources, adding functions to them, and playing around to make sure that the visuals are coherent when changed and the differences are not too drastic but also enough to match the beat change.

Next, we looked at the separate visuals one by one and deleted the ones we weren’t all agreeing on and left the ones we liked. By that time, Aalya and Xinyue were more or less done with the audios and we all had something to work with and put together.

This was more of a trial and error situation where we would improvise as they play the audios, see what matches, and basically map them to one another. Here, we also worked with the cc values to add onto the variations such as increasing the number of circles or blobs on the screen with the beat or creating pulsing effects. 

We wanted the visuals to match up the build up of the audio. To do this, we tried to build up the complexity of the visuals from a simple circle to an oval, a kaleid, a fluid pattern, a zoomed out effect, a fast butterfly effect, ultimately transitioning into the drop visual. To keep the style consistent throughout the composition, we stuck to the same color scheme presented inside the circle from the very beginning. As the beat built up, we used a more intense, saturated color scheme, along with cc values that matched the faster beat. 

now_drop = ()=> src(o0).modulate(noise(()=>cc[4]*5,()=>cc[0]*3)).rotate(()=>cc[4]*5+1, 0.9).repeatX(()=>-ccActual[1]+4).repeatY(()=>-ccActual[1]+4).blend(o0).color(()=>cc[4]*10,5,2)

now_drop().out(o1)

render(o1)

For the outro, we toned down the saturation back to its default, and silenced the cc values as we silenced each melody and beat, returning the visuals back to its original state. We further smoothed out the noise of the circle to indicate the proximity to the ending

src(o0).mult(osc(0,0,()=>Math.sin(time/1.5)+2 ) ).diff(noise(2,.4).brightness(0.1).contrast(1).mult(osc(9,0,13.95).rotate( ()=>time/2 )).rotate( ()=>time/2).mult( shape(100,()=>cc[2]*1,0.2).scale(1,.6,1) ).out(o0)

render(o0)

Evaluation and Challenges

The main challenge that we encountered in the development process of the performance was coordinating a time that worked for everyone and making sure that our ideas were communicated with each other to ensure that everyone was on the same page.

Another issue we had was that the cc values were not reflected properly, or in the same way for everyone, which resulted in inconsistent visual outputs that were not in sync with the audio for some of us while they were for others.

When Aalya and Xinyue were working on the music, although it was fast coming up with different beats, it took some time to put the pieces together into a complete and cohesive composition.

An aspect that worked very well was the distribution of work. Every part of the project was taken care of so that we had balanced visual and sound elements.

Overall, this project made us bond together as a group and fostered creativity. Combining our work to create something new was a rewarding experience, and with everyone  being on the same page and working towards the same goal, it was even more rewarding. For our final performance, we accomplished something that we all could be proud of and were satisfied with the progress we had made. Everyone was in good spirits throughout the process, which helped to create an atmosphere of trust, collaboration, and creativity. This project allowed us to use our individual strengths for the collective benefit and gave us an opportunity to learn from each other in a fun environment.

Overview

The process of creating our final performance was a very hectic but rewarding experience! We had a lot of fun working together and combining all our ideas to create one cohesive piece. We experienced several challenges along the way, such as difficulty matching the visuals with the sounds, and also combining the buildups with the beat drops in a way that makes sense, but we got there in the end! We found it hard to split the work into sections and then each working on our sections, so our approach to creating our final piece was to work on everything together. For example, Louis took charge of creating the music, so Shengyang and Debbie would support Louis during this process and offer feedback instead of working on other things (Debbie took charge of Hydra and midi while Shengyang took charge of p5.js). This definitely prolonged the process of creating the final performance, but we found that working this way made the final result feel more cohesive. If we could work on this project again, we would first choose a theme so that we’d be able to split the work into sections and successfully combine our work to make a cohesive piece. Overall, we are very proud of how far we have come! It was an amazing experience working with the team, everyone in the class, and Professor Aaron! We learned a lot of skills that we will continue to use in the future!

Debbie’s part:

For our final performance, I focused mainly on creating visuals (with Shengyang) and tying the whole piece together. Initially, I created the following visual with p5.js. 

I really liked this visual because I liked how the size of the hole in the centre could be manipulated to mimic the vibrations on a speaker, or even a beating heart. But my issue with it was that I found it difficult visualising what I could do with the visual other than changing the size of the hole. I tried changing the shapes from squares to something else or combining them with the hydra, but nothing felt right. I wanted to create a visual that could be used throughout the whole piece and changed in unique but cohesive ways. So, a week before the performance, I decided to change the visuals from this to the following:

What I loved about this visual is that the code was so straightforward, and little changes to it could change the visual in huge ways that still remained cohesive. This visual was made completely with Hydra. Here is a snippet of the code:

osc(215, 0.1, 20)
.modulate(
  osc(2, -0.3, 100)
  .rotate(15)
)
.color(0.9, 0.0, 0.9 )
.modulate(
  osc(6, -0.1 )
  .rotate(9)
)
.add(
  osc(10, -0.9, 900 )
  .color(1,0,1)
)
.mult(
  shape(900, (()=>cc[1]/4), 0.5)
  .luma()
    .repeatX(()=>cc[3]*100)
    .repeatY(()=>cc[3]*100)
  .colorama(10)
)
.modulate(
  osc(9, -0.3, 900)
  .rotate(()=>cc[0])
)
.out(o0)

Louis’ part:

Like Drum Circle, I basically take charge of the music part of our final work. This is because personally speaking, I am more enthusiastic about making music and have also accumulated some experience creating music via Tidal. In the process of making this final piece, I was inspired by other works, including some works on Garageband (very brilliant works they are but are quite difficult to be realized by using Tidal), the beat-box videos Aaron recommended we watch, and of course our own works (namely the Drum Circle). For me, the most critical parts of this work are the two drops. We spend a lot of time making the two drops to have a good effect of carrying on the top and starting the follow-ups. Take the second drop as an example, we crossed-use the two main melodies (one is the main motif of the main work, and the other one is the melody played at the end. By doing this, I believe we have managed to tie the previous content together and enhance the rhythm of the drop section very effectively (based on the reaction of the audience during the live performance, this section should be quite successful).

I want to thank my two lovely teammates Shengyang and Debbie, they gave a lot of useful advice on how to make the melody as well as make the whole structure of the music part make sense. I also want to thank Aaron for providing great suggestions on improving especially the first drop (at first we have a 16-shot 808 which makes itself more like the drop part than the real one).

Without their help, I won’t be able to realize this work alone.

Shenyang’s part:

In the final project, I am mainly responsible for the production of the p5.js part. I was also involved in some musical improvements and a part of the abandoned visuals. I will mainly talk about the p5.js part we realized and put in the final performance. First, we used laser beams, which are realized in p5.js. This effect is added to the top layer of the original hydra section in the second half of the show. This is a screenshot of a combination of the laser beams from p5 and hydra.

The laser beam effect is actually inspired by the MV of A L I E N S by Coldplay that Louis showed us, you can find the MV here, and the laser beam appears at about 2:30. This is the laser beam in the MV:

I used a class to store the location and direction of each beam:

 class RayLP {
  constructor(x, y, startAngle) {
    this.originX = x;
    this.originY = y;
    this.distance = 0;
    this.angle = startAngle;
    this.stillOnScreen = true;
    this.speed = 1;
    this.length = p2.floor(p2.random(8,17));
    this.color = p2.random(0,1);
  }
  move() {
    this.distance = this.distance + this.speed;
    this.speed = this.speed + this.distance / 350 + ccActual[9];
  }
  check() {
    this.stillOnScreen = (this.distance < width / 2);
  }
  destroy() {
    delete(this);
  }
  show() {
    p2.push();// remember the fill and stroke before
    if(this.color > 0.5){
    p2.stroke(220,65,255, 180 - this.distance);}//, 255 - this.distance * 2);}
     else{p2.stroke(139,0,50);}
    p2.strokeWeight(9);
      if (this.stillOnScreen) {
      for (var i = 0; i < this.length; i++) {
        var x = this.originX + (this.distance + i *2) * p2.cos(this.angle);
        var y = this.originY + (this.distance + i *2) * p2.sin(this.angle);
        var xTo = this.originX + (this.distance + i * 10 ) * p2.cos(this.angle);
        var yTo = this.originY + (this.distance + i * 10 ) * p2.sin(this.angle);
        p2.line(x,y,xTo,yTo);
      }
    }
    else{this.destroy();}
    p2.pop();  //restore fill and stroke
  }
}

And in the p5 draw function, each frame the program will show and check, and push new laser beams into a list. We finally changed the color/stroke to dark purple and dark red just like the background. After building the hydra function on the laser beam effect, it can twist like the background dots, which makes the effect less abrupt.

Notably, I had some silly errors in my p5 code, and Omar gave me some very inspiring advice. Thanks a lot for that! Also, there was no delete(this) part in the class code at the beginning, which doesn’t cause any obvious problems in the p5 editor. But when migrated to hydra, whether used in Atom or in Flok, it quickly fills up memory. This can make the platform stuck or slow to respond.

I was also in charge of realizing the visuals for the surprise Merry Christmas session at the end. This is migrated from the Examples of p5.js that can be found here.

The syntax for declaring classes this way doesn’t seem to be very acceptable in hydra, so I’ve replaced it with a more common way of writing it that is acceptable to the hydra.

I really enjoyed the class, very intellectually stimulating. Thank you so much to professor Aaron, my teammates, and my lovely classmates! I will miss you guys!

You can find videos of our performances here:

Last but not least:

once $ s “xmas” # gain 1.2 # room 1.5

Someone asked us for the code of the visuals. The bulk of it is P5, the height map is inspired by Shiffman’s tutorial. The height values of the terrain are then multiplied by a factor of the distance from the center (hence the flat part in the middle). Square movement is two sine functions, one for their y positions and the other for their rotation. The sun is the shader in the class tutorial.

Hydra is used for feedback and coloring and the final transition. The transition is a Voronoi-modulated oscillator.

Inspiration for the composition of the visual was drawn from this video.

Here’s the full code. Steps:

1. run shader
2. run p5
3. run for loop
4. run draw()
5. run o1
6. run o3
7. run render(o3)
8. tinker with o3 as commented

// glow circle
setFunction({
  name: 'glowCircle',
  type: 'src',
  inputs: [{
      type: 'float',
      name: 'locX',
      default: 0.,
    },
    {
      type: 'float',
      name: 'locY',
      default: 0.,
    },
    {
      type: 'float',
      name: 'glowAmount',
      default: 50.,
    },
    {
      type: 'float',
      name: 'r',
      default: 0.6,
    },
    {
      type: 'float',
      name: 'g',
      default: 0.3,
    },
    {
      type: 'float',
      name: 'b',
      default: 0.5,
    },
  ],
  glsl: `
  vec2 loc = vec2(locX,locY);
  // loc is in screen spaces, but _st is in normalized space
  float dist = glowAmount/distance(_st*resolution, loc);
  return vec4(r*dist,g*dist,b*dist,0.1);
`
})


p5 = new P5({width: window.innerWidth, height:window.innerHeight, mode: 'WEBGL'})
s0.init({src: p5.canvas})
src(s0).out(0)
p5.hide();
scl = 50;
w = 4200;
h = 3000;
//set m as 300
m = 100;
cols = w / scl;
rows = h / scl
flying = 0
terrain = []
spikes = []
toggle = 0
toggle2 = 0
size = 3;
pink = p5.color(255, 34, 240);
blue = p5.color(23, 200, 255);
neon = p5.color(10, 220, 255);
prv = [0,0,0];
ctr = [0,0,0];

p5.remove()

//make electro sound go up with the other one
for (var x = 0; x < cols; x++) {
    terrain[x] = [];
  	spikes[x] = [];
    for (var y = 0; y < rows; y++) {
      terrain[x][y] = 0; //specify a default value for now
      spikes[x][y] = 0;
    }
  }

p5.draw = ()=> {
  blue = p5.color(1, 6, 40);
  m = 100;
  size = p5.random(2,5);
  fade = 0.8;
  //p5.lights();
  p5.background(blue);
  p5.translate(0, 300, -100);
  p5.rotateX(42*p5.PI/72);
  //p5.rotateZ(time*p5.PI / 3);
  //p5.fill(255*p5.noise(1), 190*p5.noise(1), 150 + 200*p5.noise(1), 255);
  p5.translate(-w/2, -h/2);
  p5.noStroke();
  //p5.stroke(255, 34, 240);
  //GRID
  for (var i = 0; i < cols; i++)
  {
    p5.line(i*scl, 0, i*scl, h);
  }
  for (var i = 0; i < rows; i++)
  {
    p5.line(0, i*scl, w, i*scl);
  }
  //p5.noStroke();
  flying -= 0.03;
  var yoff = flying;
  for (var y = 0; y < rows; y++) {
    var xoff = 0;
    for (var x = 0; x < cols; x++) {
      terrain[x][y] = p5.map(p5.noise(xoff, yoff), 0, 1, 0, m) + spikes[x][y];
      spikes[x][y] *= fade;
      //
      xoff += 0.03;
    }
    yoff += 0.04;
  }
  //big blocks
  let cn = 12;
  if (cc[cn] != toggle){
    toggle = cc[cn];
    x = p5.int(p5.random(0.4, 0.6)*cols);
    y = p5.int(p5.random(1)*rows);
    x = p5.constrain(x, 1, cols-size-2);
    y = p5.constrain(y, 1, rows-size-2);
    //spike it up
    for(let i = 1; i < size; i++)
    {
      for(let j =1; j< size; j++)
      {
        spikes[x+i][y] = ccActual[cn]*55;
        spikes[x+i][y] = ccActual[cn]*55;
        spikes[x+i][y+j] = ccActual[cn]*55;
        spikes[x][y+j] = ccActual[cn]*55;
      }
    }
  }
  //sharp spikes
  let cn2 = 10;
  if (cc[cn2] != toggle2){
    toggle2 = cc[cn2];
    x = p5.int(p5.random(0.4, 0.6)*cols);
    y = p5.int(p5.random(1)*rows);
    //spike it up
    spikes[x][y] = 105*ccActual[cn2];
  }
  //terrain
  for (var y = 0; y < rows - 1; y++) {
    //left side
    p5.fill(blue);
    //p5.noFill();
    //p5.stroke(pink);
    p5.noStroke();
    p5.beginShape(p5.TRIANGLE_STRIP);
    for (var x = 0; x < cols-1; x++) {
      let dist = p5.pow(x-cols/2,2)/20;
      p5.vertex(x * scl, y * scl, terrain[x][y]*dist);
      p5.vertex(x * scl, (y + 1) * scl, terrain[x][y + 1]*dist);
    }
  	p5.endShape();
    for (var x = 0; x < cols-1; x++) {
      p5.strokeWeight(10);
      p5.stroke(pink);
      if (x%10==0)
      {
        p5.stroke(neon);
      }
      let dist = p5.pow(x-cols/2,2)/20;
      p5.line(x*scl, y*scl, terrain[x][y]*dist, x*scl, (y+1)*scl, terrain[x][y+1]*dist);
      //p5.line(x*scl, y*scl, terrain[x][y]*dist, (x+1)*scl, (y)*scl, terrain[x+1][y]*dist);
    }
  }
  //translate
  p5.strokeWeight(5);
  p5.stroke(neon);
  p5.fill(pink);
  //central box
  p5.push();
  p5.translate(w/2,2300, 70 + 40*p5.cos(flying*7-3));
  p5.rotateX(-flying*3);
  p5.box(50 + ccActual[13]*0);
  prv[0]=cc[12];
  p5.pop();
  //box left
  p5.push();
  p5.strokeWeight(7);
  p5.translate(w/2-100,1700, 100 + 60*p5.cos(flying*7-1));
  p5.rotateX(-flying*3 - 1);
  p5.box(50 + ccActual[13]*0);
  prv[1]=cc[11];
  p5.pop();
  //box right
  p5.strokeWeight(10);
  p5.push();
  p5.translate(w/2-60,100, 80 + 60*p5.cos(flying*7 - 6));
  p5.rotateX(-flying*3);
  p5.box(50 + ccActual[13]*0);
  p5.pop();
  //box left2
  //box center
}

//o0
src(s0).out(o0)

//MY GPU CANTTT
osc(1,2,0).color(10, 10,10).brightness(-100).modulate(noise()).mask(src(o0).sub(osc(0.9, 0.4).modulate(voronoi(20,10), 0.9))).add(src(o0)).invert().out(o1)

//final output [MAIN VISUAL]
//option1: modulate by o0, become crystalline/transparent: v pretty
//option2: blend multiple times o3
//option3: switch source to o1, this is the main function then blend with o3.
src(o0).layer(glowCircle(31*p5.width/70, 7*p5.height/24, ()=> ccActual[13]*100+2500, ()=>0.3, 0.1 ,0.06)).blend(o3).out(o3)

render(o3)

//black screen
//track 1 => automatically switches
//build up => bring the sun out || sun moves
//drop the beat => sun automatically detects that
//on drop => o1
//modulate o(0) blend o3
//o1 source then hush

hush()