Sound&Vision

I’m interested in the role of the artist as the encoder and the observer as the decoder. This relationship has always existed as the observer has to perceive the artist’s work with their senses and try to make sense of it; they make their own translation, determine their own subjective meaning. I take it a bit further by encoding objective data in the visuals.

Le Wonkatania v3.0, v1.0, v2.0, v4.0

My current work is about encoding sound data visually. I have developed four methods of visual notation or visual encoding of sound, specifically music. The first works in this series are based on music from the movie Willy Wonka & The Chocolate Factory (1971). After deciding on an algorithm for conveying the sound data I use color, shape and texture to convey the rest of the soundscape.

This has been my favorite movie since I was a little kid–we were born the same year. Time and again it inspires me with all of its wonderful, dark weirdness. When I was 15 I watched it every day for a summer and could recite the dialog of the entire movie. I’ve used it many times over the years for sampling sounds and visuals.

The first series of four shown above are, Le Wonkatania are the Wonkatania Song. The second series of four, Rachmaninoff, are the Overture of the Marriage of Figaro by Mozart, which is used as a musical lock in the movie (and is mis-identified by one of the characters as Rachmaninoff). This piece of music is also described by some random guy on the internet as “holy shit, the main tiddly in this song is what willy wonka used as a password to get into that candy room where you can eat the trees and shit!”

Further series based on other music and sounds from the movie are planned. Additionally, I am interested in painting songs by local (or other) musicians.

**Spoiler Alert**

If you want to decode these visuals on your own stop reading here. Otherwise, proceed with caution to delve into my thought processes below.

Visualization 1.0

This visualization uses an algorithm of my own devising. Measures are concentric loops along which notes are placed and time moves clockwise starting from the left edge, starting with the outermost loop, moving inward. Note pitch is represented by the size of the circle. Larger circles are lower in pitch, smaller circles higher.

In Rachmaninoff v1.0 harmonizing notes are also part of the visualization.

Visualization 1.0: Rachmaninoff v1.0

Visualization 2.0

Try not to go blind reading the following explanation. It’s ok to just look at the pretty colors and shapes and skip ahead.

Parsons Code is used to make music searchable in music search engines. The data conveyed by Parsons Code is whether the next note is higher than, lower than, or the same as the previous note. Time and beats per measure are not conveyed.

Visualization 2.0: Rachmaninoff v2.0

The Wonkatania Song encoded as Parsons Code is

*DUDUDUddDUDUDUdsDUDUDUddDUDUDUduudDUudDuduDUduD or *v^v^v^\\v^v^v^\-v^v^v^\\v^v^v^\//\v^/\v/\/v^\/v where:

• * is the starting point of the musical phrase that is being encoded
• d or \ is a descending step (1-2 notes down)
• D or v is a descending leap (3 or more notes down)
• u or / is an ascending step (1-2 notes up)
• U or ^ is an ascending leap (3 or more notes up)
• r or – is a repeating note

These ascensions and descensions can further be visualized with the ASCII characters *, /, \ and – which show the rises, falls, and sustained notes. This is the basis for visualization 2.0.

Visualization 3.0

First, I start with Javanese Kepatihan notation (yes, seriously, that is a real thing, I know it sounds like gibberish) which, more simply, is just one method that assigns numbers to notes on a musical scale. Low notes are low numbers, higher notes are higher numbers. If you wanted to play Le Wonkatania v3.0 using a piano keyboard, note 1 is D.  Note 1 for Rachmaninoff v3.0 is C#.When mapping numbers to notes, do include the black keys.

Next, I’ve developed a color layering scheme for the numbers that reflects Parsons Code. For example, in Le Wonkatania v3.0 purple-red-purple is the *, yellow-orange-blue is a descending leap, blue-green-yellow is an ascending leap, etc.

Visualization 3.0: Rachmaninoff v3.0

Visualization 4.0

This is based on the Fisher Price Music Box Record Player from the 1970s (hmmm another mention of the 70s). Each concentric ring on the record has blocks along them which are struck by the teeth of the music box mechanism, which vibrate and make a tone.

I ♥ that this is an analog machine with a spring-loaded motor and a conical piece of plastic that amplifies its sound.

One circuit of the disc plays the whole song, read counter-clockwise starting in the bottom-left-ish quadrant on Le Wonkatania v4.0; starting in the top-right-ish quadrant on Rachmaninoff v4.0. Some makers on the internet have developed software that allows notes to be plotted and a CAD file created which can be used to 3D print custom records for this record player. I used the software to create an image of a record of the song, then I added geometric shapes and colors to represent the quality of the tones created by the music box record player.

Visualization 4.0: Rachmaninoff v4.0

Rachmaninoff v3.0, v1.0, v2.0, v4.0

I also accepted a commission for cover artwork for the release of an EP cassette, called Light Leak by Bendu. I’d like to do more of this sort of thing.

Light Leak v4.0; Cassette by Bendu

And in case you were wondering, Sound&Vision is a reference to the 1977 David Bowie song.