Sound in... My Apartment: Plans for a Quad experiment

The last couple of weeks have been truly bizarre. I was in college in 2008 during the Financial Crisis and thought it was one of the weirder and more momentous experiences I’d lived through. This moment has quickly leapfrogged 2008, and given how terrible this situation is for so many people, I’m very lucky to be in school yet again to participate in/observe how this all unfolds.

Still, it’s been difficult to focus on schoolwork during this first wave of changes to our lives, our world. Prior to going remote, I was very excited to experiment with a quad setup at school; moving from stereo to multichannel was what I signed up for! Going remote, worrying about family and friends, making major adjustments to lifestyle, and just reading the constantly shifting news has drained my motivation tank. So, this week’s assignment is basically a sketch, a set of plans for an experiment, but not what I would have liked to do in a more-normal world.

Idea:

I would like to make a p5 + tone.js sketch that functions like a spatialization test with a twist. I’ve never worked with quad, so this would be a helpful way to start to understand some of the “physics” of different audio setups. In short, the tool will play individual notes in the form of moving balls that, when played together, can form chords. The notes will play through different channels as they collide with the “speakers” on screen.

Here is a rough skeleton of the beginnings:

As I build this out, users will be able to:

• Arrange a multi-channel speaker setup on the canvas

  • The speakers will be rectangles with inputs to indicate the channel number

• Click to create notes visualized as balls that bounce around the screen

  • The balls carry a note that belongs to a particular key

  • The note is randomly chosen from the array of notes when created

    • Default will be to only choose 1,3,5,8 notes from the key to create a major chord

  • As the balls collide with the rectangles, the note associated with the ball plays through the channel associated with the speaker.

• Create many balls, increasing the chance of a chord playing through different speakers

• Use the the right or left arrow to change the key that the notes are drawn from.

• Use the up or down arrows to make balls move faster or slower

• Hit space to clear balls

Execution Plan

I started to wire up the sketch by creating objects for the balls and rectangles. I modified an earlier particle system sketch I made, and kept the system object, which may allow for some more features down the line. I haven’t connected tone yet, which is obviously the meat of this sketch. I plan on working on this over the next 2 weeks to make it work for the binaural assignment. The first set of things I need to to do:

• Finish collision detection between ball and rectangle

• Add tone.js to the html doc

• Hard- assign channels to each speaker

  • Once this work, allow input to assign channel

• When collision is detected, play a sound

  • Make sound draw from an array of possible sounds in a key

  • Create arrays for different keys

  • Allow key to be changed

• Connect slider to ball speed

• Re-configure keys/mouse presses to match a good UX

• Create volume slider and hook up to tone

PComp Midterm: Liquid Sounds

Assignment: Create a project with interactivity that ideally uses serial communication between a microcontroller and computer. Daniel Fries and I collaborated on this assignment.

Idea: Daniel and I chose to collaborate together to build an instrument. Beyond knowing we wanted to make some sort of instrument, we entered the brainstorming phase with a pretty blank slate. Some recurring themes in our brainstorming included the concept of scrubbing through granules of sound, analog tape as either an aesthetic or as an actual mode of playback and playback controlled by walking in physical space. We knew we did not want any screens involved and wanted to make a very physical experience for the musician; we also wanted there to be a process of discovery for the musician to learn the affordances and nuances of the instrument. It would not need to be immediately apparent what produced sound, and what qualities of sound were possible.

We landed on using the idea of mixing liquids to control different parameters of sound. We imagined our users mixing cocktails with different outcomes depending on the particular proportions of spirits and mixers used— among other applications.

Process: Our initial plan was to place 3 glasses in a fabricated wooden enclosure and have a mixing bowl outside of the enclosure. We wanted to put load sensors under each glass/bowl to sense the amount of weight in each one, then program conditions that either changed sounds depending upon hitting particular thresholds of weight or conditions that changed continuously as weight changed.

We placed our order for 4 load sensors on Thursday. The assignment was due the following Wednesday and the sensors were supposed to arrive on Saturday. By Tuesday morning, we still did not have the sensors in-hand. Once we did get them, our worst-case scenario played out: the sensors did not behave as we had expected.

It turns out that 4 load sensors need to be used in-concert to form a Wheatstone Bridge circuit. We expected to be able to use just one load sensor to pull signal from the glass above it, even if that signal was only directionally correct. Once we discovered that this would not work, and it would be both time-and-cost prohibitive to make this approach work, we shifted to a different plan.

We had already fabricated our enclosure and written pseudocode before the sensors arrived; we needed a suitable alternative for the load sensor. Luckily, it turns out Force Sensing Resistors can pick up something akin to weight; and given that we only needed a directional reading, they were a fine alternative. We borrowed FSRs from classmates, soldered longer wires to them, and tested different setups with our enclosures to make sure they could pick up signal as our glasses filled with liquid.

Once we got them working well enough around 9:30pm, we switched to turning our pseudocode to real code in Tone.js. Daniel had experience writing musical instruments in Tone.js; although our initial plan was for me to drive the code section so that I could ramp up my skills, we ultimately had to have Daniel take more control (although we still sat together and talked through all of the code during the process) for the sake of time.

Reflections:

At the end of the day, our instrument worked and drew good responses from users who got to listen to + play the instrument. However, if we had our hands on sensors sooner, we could have finished a working prototype in time to work through details in the enclosure and the sound generation to make this a truly interesting musical device that we could continue to perform with. We’ll take that as a learning lesson moving forward in our physical computing projects.