Whale Theremin Update (still drafting post) 12.10.17

My original idea has been transformed a few times over the past few weeks but has retained the original concept as far as working with organic material, sound and triggers that don’t involve direct touch to be activated.

I worked on the circuitry and code for the Whale Theremin to get the sound and effects triggering with 2 photosensors and a button.  Once the basic code and circuitry were working, I shifted my focus to trying to get the Adafruit Capacitive Touch Shield working to trigger organic materials that are conductive such as a fruit or plant.


For some reason this proved to be very buggy and I spent too much time trying to make it work, it would sporadically read the sensors, I caught it when it momentarily worked below.

I then decided to try to get this to work longer wouldn’t be a wise use of my time and I decided to order the Adafruit Capacitive Touch break out instead, which was relatively smooth to set up and start a reading from.


I decided however that I really wanted to work with plants and midi, so it was more of a reading of the state of the plant’s microelectrical currents than just on and off.

I tried to model the galvonometer from MIDISprout’s circuitry. They have a more complex system that includes a potentiometer that effects specific parameters within MIDI and corresponding LED lights that I wasn’t interested in at this moment, my main objective was to show how plants electrically/energetically have reactions to touch / the environments around them and to have that translated into MIDI and then sound. http://support.midisprout.com/index.php?topic=8.0

At Tom Igoe’s advise, I attempted to replicate just the galvonometer aspect of the circuit and to make a more straightforward MIDI communication. I spent several days on this and was unable to get the galvonometer to work properly.  This is something I definitely want to revisit in the future and get to work.

I then chose to re-focus my attention onto the whale theremin coding and fabrication further and table both of the touch sensors until the theremin felt complete. There were a couple places that I had problems with my code and Jim, Chino and Mathura (*insert last names*) were very helpful in explaining certain hiccups in my code and/or offering help on how to execute certain functions successfully.  I expanded on the visuals in p5 including mapping the 3 color channels to the 2 light sensors. red is mapped to photosensor 1, green to photosensor 2 and blue to both of them. Leon (*insert last name*) helped me execute this properly.


There has been some challenges with the construction of the piece mainly since I chose to project on top of the water. Water is very heavy and obviously damaging around electronics. I envision this project to be at a larger scale projecting onto a larger body of water, but currently I’m using a 12 x 12 x 12 acrylic box for now. I’ve lined the seams with DAP Flexible Clear Sealant and have ordered a project box that will sit underneath the box where the arduino will be housed. The photosensors will be on either side of the box.

I found the best remedy for using photosensors and a light projection that requires a darker room is using 2 flashlights, also making the photosensor readings more accurate.

I might switch to distance sensors, as recommended by Tom, but my priorities now are getting the fabrication and execution running as smoothly as possible. I haven’t had that much difficulty with the photosensors because of the constraint function in the P5 code and the relative ease of calibrating according to the light level in a particular room as well.


I’ve done a lot of research with various waterproof speakers and have ordered 3 now, the smaller one works best so far but is not powerful enough for this amount of water. The most powerful one cuts out when it’s underwater for too long, so I have a 4th one on its way that is supposed to be very powerful and is aesthetically in line with the overall piece.

I figured out how to do a multi speaker out from a macbook and think I’d like to have an audio output inside the water and also outside the water for increased sonic immersion.


I had the plant present during user testing to see the feedback with it included and everyone suggested using the plant for another project. I’m still interested in triggering the plant through underwater sounds and vibrations but probably will not use whale sounds or imagery.

I was attempting to use my own projector for the playtesting but was unable to and the internet kept going out. For my ICM class in the afternoon I was able to successfully project the water onto the surface of a fish tank and have the theremin trigger the sounds and colors, unfortunately I don’t have documentation because i was making sure everything was running smoothly.

The main feedback I got was how to make the speaker could look good and not interfere with the whale projection, Fanchi suggested netting so it would look more aquatic and it wouldn’t muffle the sound. I purchased a white speaker, since I’ll be making the bottom of my box white for optimal projection, so it can blend in as much as possible. That speaker should be arriving tonight. Another Idea I have is to put a layer of white cotton material on top so the sound can go through but the projection still has a smooth white surface.  The speaker also needs to be weighted down so it stays at the bottom of the tank. I think I will be using rocks to do so.


  • wait for final fab materials to come in and make final construction: arduino box, lights for the photosensors, alternate waterproof speaker, netting for speaker,
  • include schematics
  • update blog further with work process
  • test out new speaker and multi speaker configuration
  • test water in box, make sure nothing is leaking
  • apply Maptastic to my p5 sketch so that it projects to the exact dimensions of water surface
  • test projection in new container with Maptastic applied

Alternate water holder if acrylic box doesn’t work:

This would need to be set up near the sink in the kitchen.


<p><a href=”https://vimeo.com/245654112″>ORCHID LAVA 2a</a> from <a href=”https://vimeo.com/user2116950″>Nire</a&gt; on <a href=”https://vimeo.com”>Vimeo</a&gt;.</p>


I chose to work with some of the orchids that mimic animals, insects and people for this After Effects animation.  I’m fascinated and have a great amount of respect for plant medicine and wisdom and feel as though these orchids provide a cosmic joke and secret if we look closely enough. All people worldwide originally had a strong relationship with nature and knowledge of what plants can do for people, that we’re now greatly severed from.  One idea / theme I’m interested in exploring more is the relationship of death and the dead in the US.  It’s very taboo and I think causes a certain type of neurosis in the culture.  This animation is a baby creation story and a small narrative of how living beings are temporal and constantly surrounded by their mortality and frequently a few degrees from death more than we care to think about, yet the orchids (beings) are able to have fun despite this.

Fabrication – enclosure

I chose to make a speaker enclosure that I will be using for PComp out of PVC. I cut small pieces of PVC tube on the miter saw to use as connectors between the curved pieces.

i will be attaching a mirrored acrylic base to it today.

The pipes at Home Depot were called 2″ but the opening is larger than that, so the Bluetooth speakers I bought fall right through, I’m going to purchase larger ones that can click in and out. For now I’m using bubble wrap and have the top pieces tilted up, which looks better aesthetically as well. I’m considering using mod podge and doing some sort of collage on it or try covering it in cement, i do also like it white, so thinking of somehow making it a glossy finish.

Im going to incorporate this into my pcomp final, where there will be two audio outputs, one will be inside a bowl of water – whose output is triggered by a theremin, that controls whale sounds, pitch and delay through p5js. A plant will be suspended in the middle of the water that has a biofeedback system that gets translated to MIDI and will be run through ableton. This pipe enclosure will be the output for the plant. There’s a place in the middle on top of the speakers, for an air plant.

PComp ICM Final Update – Plant-Water/You Duet

For my final project I’m building off of the midterm project light theremin, having it now trigger whale sounds in p5, with the theremin affecting the pitch.

The theremin will trigger sound in p5 whose audio output is routed to a waterproof speaker that will be in a clear plastic bowl. There will be a plant suspended in the middle of the bowl with the roots inside the water – the plant will be connected to MIDISprout which uses electrodes to read micro-electrical shifts in the plant and then translates it to MIDI data. This will be connected ultimately to Ableton Live.
Touching the plant is usually the trigger for Midisprout’s change in notes but I’m going to have the audio from the theremin trigger the shifts,  because the water in the bowl will make various vibrations according to the pitch played.
For my midterm the audio traveling through the water created cymatic patterns that corresponded to the frequency – the hope is that the patterns will still form around the plant as well, but is not essential for this concept at this point.
This will enable the user to trigger a range of sound from p5 that could be used to duet with the plant since the plant will be generating sound from the vibrations of the MIDIsprout.
MiDISprout connected to Ableton:

Whale sounds and pitch:




Whale Sound Example:


The 2 light theremins would control the pitch and delay – currently corresponding to the bubble images.

The background of the sketch will be whale video footage whose speed will correspond with the pitch, that I’d like to project overhead onto the water.

Intro to Fabrication Week #1 – Selenite Flashlight

I decided to use a Selenite crystal for the flashlight and drilled the crystal starting with the smallest drill bit, increasing to the size of the LED. I connected it with two 1.5V batteries and a battery holder with a switch. The base of the flashlight is made out of a plastic vitamin bottle that I glued for security with a bottom hole cut out for the switch.



PComp #5 – Garden Box Theremin

I made this sound and light theremin following the Arduino Theremin instructions in the booklet. I used recycled materials from my house including a small cardboard box that housed nut cheese and some magazine photos and tape. I had a bunch of leaves in a vase that were on their way out, and decided to use them to camouflage the inside of the box, the arduino and breadboard. It also creates a little bit of stability for them and making movement less inside the box, the piece is still pretty delicate. I placed it so it still looks wild, the only piece of the circuit that’s visible is the photoresistor. One of the leaves is acting like a lever to open and close the box, triggering the scale in Arduino. There’s a whole in the side of the box to connect USB from Arudino to computer. The photoresistor is triggering the Theremin in the Arduino.

Arduino Theremin Garden Box from Nire ITP on Vimeo.

This is the code I used:
int sensorValue;
int sensorLow = 1023;
int sensorHigh = 0;
const int ledPin = 13;

void setup() {
digitalWrite(ledPin, HIGH);
while (millis() sensorHigh){
sensorHigh = sensorValue;
if (sensorValue < sensorLow) {
sensorLow = sensorValue;

digitalWrite(ledPin, LOW);

void loop() {
sensorValue = analogRead(A0);
int pitch =
map(sensorValue,sensorLow,sensorHigh, 50, 4000);

PComp #4 – Keyboard Instrument

When I first set up the circuit board and wrote the code, the first note didn’t play, I tried switching out a button but that didn’t make a difference. Then I unplugged the wire going into the button and the other buttons were still playing notes so I figured maybe that wire needed to be switched out, I think even though it clicked in on both sides, it was a touch too short. Once I put a new wire in, the 1st button started playing.

1st Key Not Working from Nire ITP on Vimeo.

At first there were code errors, I eventually realized that there multiple curly brackets at the end of the code, when I erased them, it ran smoothly.

Playing 15 seconds of “Lean on Me”:

15 seconds Lean On Me Arduino from Nire ITP on Vimeo.