by Dominique Fougere

You see sound. You taste colors. You experience numbers spatially.

No, you aren’t dreaming. You’re a synesthete. Welcome to the world of synesthesia.

To those new to the word, synesthesia is loosely defined as a “sensory/ perception experience” in which one “sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway.” In other words, you can experience the intangible with concrete senses.  There is, of course, more to the science behind this “ability”, but I’ll leave the rest to Wikipedia to explain.

For those of us, like myself, who experience this, creative fields allow for full expression of synesthetic experiences. Design, I believe, is supposed to show different modes of perception. Thus, the principles of user experience design are often utilized to expose a non-synesthetic audience to different, unique sensory perceptions.

There are abundant examples of this in various media. This teaser video for “Noise” allows the audience to view a glimpse of an apartment dweller’s visual perceptions of auditory stimuli, designed with compelling visual effects.

Videos like these are creative, yet passive. The audience is only an observer. It means more to have a truly memorable, if artificial, synesthetic experience if the program or media one views is designed for an immersive and engaging user experience.

For that, computer game developers have created astounding programs for just this purpose. Audio-Surf is one shining example. The simple formula of the game is to guide a rocketship through a racetrack while collecting floating bars for points.

Audio-Surf

Gives new meaning to the term "joyride."

The formula, however, is worthless without one key element – the individual song you supply. This is the game’s true synesthetic design. You literally see or “ride” your music. Thus, the game is designed from a synesthete’s playable perception of music: bars in the game are specific beats, the bass in the song is visualized in the spaceship’s pulsing thrusters, the tempo changes make the road either bob happily in energetic green colors, rise up in serene purple-blue, or plummet quickly in searing shades of yellow and red.

Songs that are personal to the user show off the success of playing this game, as you become fully immersed in the experience. In a way, it enhances your perception of music. Personally, I feel this game is the closest in fully realizing how one can “see” sound as colors. The following video is just one example (and a favorite of mine):

[youtube http://www.youtube.com/watch?v=2IbMrkooFus&w=627&h=353]

Other games that are designed with this same experience in mind are Groove Coaster and Supersonic HD (both available for download in the iTunes store).

Groove Coaster: The Grid never looked so cool.

While there might be the feeling that games like Audio-Surf are just games and not worthwhile, I believe it is very valuable to expose oneself to new modes of perceptions. Playing games is an easy and enjoyable way of doing so.

I look forward to the treasure hunt of discovering innovative media, especially games, which effectively utilize the principles of good user experience design with synesthetic elements. Often you find gold. (Some times you can even hear the gold.)