Category: COMPUTATIONAL MEDIA

Posted on

There-Liminal Network Final Blog

 

placeholder://

 

 

 

The final test was a success!

Everything went smoothly but I made 2 changes in the final play test.

I made the light source for the photosensors candles instead of lamps which have a more focused light emanation and add to the overall sensory experience.

I tested 4 waterproof speakers and none of them sounded good enough so I’m using speakers outside the box. Visually I couldn’t make it look good with the speaker submerged this time around.

Schematic & Diagram:

Configuration & Construction:

I made a light theremin with arduino, 2 photosensors and a switch. The analog readings of the sensors were serially communicated to P5.JS via p5 serial port app. I loaded whale sounds and video into p5 along with pitch & delay effects and red, green, blue color channels to overlay on the video. All of these components are controlled by the photosensors. Pitch on one side, delay on the other, red on the right, green on the left and blue on both.

The video was run through an overhead projector and I used Maptastic in p5 that allowed me to change the shape of the video so it fit squarely into the water basen.

The circuitry is all underneath in the white box. The 2 sensors are on the first bubbles on the right and left of the box.

The audio is running out to a blue tooth speaker.

I bought an acrylic box at Canal Plastics and lined the seams with a clear waterproof sealant to make sure it could hold water.

Posted on

PComp ICM Final Update – Plant-Water/You Duet

For my final project I’m building off of the midterm project light theremin, having it now trigger whale sounds in p5, with the theremin affecting the pitch.

The theremin will trigger sound in p5 whose audio output is routed to a waterproof speaker that will be in a clear plastic bowl. There will be a plant suspended in the middle of the bowl with the roots inside the water – the plant will be connected to MIDISprout which uses electrodes to read micro-electrical shifts in the plant and then translates it to MIDI data. This will be connected ultimately to Ableton Live.
Touching the plant is usually the trigger for Midisprout’s change in notes but I’m going to have the audio from the theremin trigger the shifts,  because the water in the bowl will make various vibrations according to the pitch played.
For my midterm the audio traveling through the water created cymatic patterns that corresponded to the frequency – the hope is that the patterns will still form around the plant as well, but is not essential for this concept at this point.
This will enable the user to trigger a range of sound from p5 that could be used to duet with the plant since the plant will be generating sound from the vibrations of the MIDIsprout.
MiDISprout connected to Ableton:

Whale sounds and pitch:

Sketch:

IMG_7736IMG_7735

 

Whale Sound Example:

http://alpha.editor.p5js.org/Nire/sketches/ByCr4XKxz

The 2 light theremins would control the pitch and delay – currently corresponding to the bubble images.

The background of the sketch will be whale video footage whose speed will correspond with the pitch, that I’d like to project overhead onto the water.

Posted on

ICM Final Proposal – Whale Song Sensory Integration

Screen Shot 2017-11-08 at 1.44.38 PM

For my ICM final, I’m collaborating with Jaqi Garcia, a teacher who works with children on the Autism spectrum and incorporates social emotional learning in her curriculum. We’re building on the “Whale Song” piece for children with sensory integration challenges, referred to by some as “Sensory Processing Disorder”.  SPD is a condition where multi-sensory integration is not adequately processed in order to provide appropriate responses to the demands of an environment. This effects some people on the autism spectrum, but also includes neurologically “typical” children/adults that have mild multi-sensory integration challenges.

Processing systems work differently for every individual, we all have 1 or 2 senses that are more acute than others.

Occupational therapists find ways to anchor children with SPD using sensory stimulation that is right for the individual, working with tools such as weighted blankets, LED blankets or trampolines.

Various sensory systems include tactile, auditory, visual and proprioception, which is the sense of your gravitational center.

We would like to sync the Whale sounds and visuals with a gel pad with a magnetic that children can push around, triggering the effects on the sound.

 

Screen Shot 2017-11-15 at 2.24.36 PM.png

Screen Shot 2017-11-15 at 3.41.13 PMScreen Shot 2017-11-15 at 3.40.11 PM

This might seem that it would overwhelm a child with sensory sensitivities but it actually  syncs the multi-sensory input (sonic, tactile and visual) in a way that is cohesive and the child has control over.  This increases the communication between the processing systems, helping with integration and regulation of their nervous system.

Constructive cause and effect activities anchor all children.  The tactile, auditory, visual interactivity of the “Whale Song” can also help children with attention challenges as well.

I’m going to expand on the current Whale Song:

http://alpha.editor.p5js.org/Nire/sketches/H1MHZFD0Z

and animate the whale and bubbles so they look more apart of the environment. I’ll map the bubbles to the marbles within the gel pad with sensors.

Uses: in home, therapeutic environments, classrooms, any place.