How synaesthesia has shaped the music world for generations

Australia Broadcast Center

2 days ago

For most of us, music is mainly a feast for our ears. But people with synaesthesia experience music with more than their sense of hearing.
Some see musical harmony in vivid colours, others have visceral physical reactions such as pins and needles.
Some even experience music in textures like jelly or cloth.
For people with synaesthesia, a neurological condition where stimuli can be experienced by multiple senses, music can invoke something profound or disruptive.
"The main thing about synaesthesia is that it is an added sense that your brain uses to interpret whatever information is going on around your body," says Stéphanie Kabanyana Kanyandekwe, ABC Classic's Passenger presenter and synaesthete.
It's not just seeing colours when hearing sounds. "The permutation is almost endless," Kabanyana Kanyandekwe says.
We talk to musicians who are living and working with synaesthesia, its upsides and downsides and how musicians have harnessed this brain quirk for generations.
The most well-known form of synaesthesia in the music world is chromesthesia: seeing sounds as colours and patterns.
French composer Olivier Messiaen associated specific chords with vivid colours, and jazz legend Duke Ellington experienced orchestral textures in shifting shades of light and fabric.
Jessica Cottis, Canberra Symphony Orchestra's artistic director, also sees instrument groups as colours.
"The woodwinds are bright blue and the brass section is spring green," Cottis says. Meanwhile, "the string section is always a deep red."
But no two experiences are alike. Composers Rimsky-Korsakov and Scriabin assigned different colour-charts to musical key signatures.
Kabanyana Kanyandekwe found out her experience was different when she sat with another synaesthete at university.
"I said this chord was blue, and they said it was yellow," she recalls.
It's common for people to experience a few forms of synaesthesia, Kabanyana Kanyandekwe says.
Although she primarily experiences chromesthesia, some sounds can also give her physical sensations. Sometimes music makes her feel lighter than air; other times a squatting weight. This physical reaction to music can impact her concert-going experience.
"My synaesthesia also makes me experience the concept of time as a massive spiral in space," Kabanyana Kanyandekwe says.
"Playing or listening to music is the only thing that helps me process the movement of time in a linear fashion."
Kabanyana Kanyandekwe says that synaesthesia is considered a form of neurodiversity.
"Neurodiversity is an umbrella term that covers many different conditions and the ways the brain works," she says.
Kabanyana Kanyandekwe believes synaesthesia is common in young children, but the experience can fade as people get older.
"As people's brains work out what information is important, most shave off the extra [sensory] information," she says.
Kabanyana Kanyandekwe describes: "Synaesthesia may interfere or impact people's understanding of music, their ability to appreciate music, even capacity to sit and listen to music."
Although most people think of synaesthesia as a harmless party trick, for some it can be profound or disruptive.
Initiatives like providing sonic stories which describe what sounds people will hear during a concert can help synaesthetes and people with other forms of neurodiversity to make decisions to manage their experience.
"[In some concerts], I wear certain earplugs that help me attenuate the sound in a way that doesn't harm me and allow me to appreciate the music," Kabanyana Kanyandekwe shares.
Being able to choose where to sit is another way for people with synaesthesia to enjoy concerts.
Some like to be seated right at the front to increase the intensity of their experience. Others prefer to sit at the back so they can step out if the music gets overwhelming.
The musical potential of synaesthesia has long been explored by composers such as Messiaen and Scriabin, among others.
Messiaen's Couleurs de la Cité Céleste (Colours Of The Celestial City) is a depiction of a multi-coloured, heavenly city based on his experience of synaesthesia.
Cottis is using her gift in a more practical way: to help her rehearse with the orchestra. She experiences harmony as "mists of colour that are always in flux".
"If I'm rehearsing and [the orchestra] is out of tune, the colours become less clear in my mind in addition to hearing it," Cottis says.
"Having these two senses fused can help me to understand what's happening very quickly."
In 1915, Scriabin invented a musical instrument called the "Clavier à lumières": a keyboard which could project coloured lights based on the composer's specifications. Each colour was assigned to different notes of the scale, for instance C is red and D is yellow. The instrument was invented for the performance of his Prometheus: The Poem Of Fire.
Researchers have cast doubt on whether Scriabin's musical colour system was an expression of synaesthesia or something he created for pure aesthetic value. But associating music and colours isn't confined to composers with synaesthesia. Scriabin's instrument invention remains one of the earliest attempts to create an immersive concert experience.
An Australian performance artist and researcher who has also used the multi-sensory nature of synaesthesia is Jocelyn Ho.
The Synaesthesia Playground project, which Ho ran in 2016 and 2017, brought together musicians, computer scientists, visual artists, and fashion designers to create an immersive and interactive concert.
"This project was based on my research in how we experience music with our bodies, such as spine-tingles and goosebumps," Ho says.
One of the ways Ho made these physical reactions visual was by wearing a fibre-optic dress that pulsed in response to what was going on inside her body.
"People could see when my heartrate quickened because I might be feeling a bit stressed, for instance" Ho says. "It stripped off my performer's mask because they could see my vulnerability."
What musicians who live or work with synaesthesia show is that music is more than a feast for our ears but a full-body experience. And you don't necessarily have to live with synaesthesia to experience a taste of it.
Ho says: "In musical memory, for instance, all of these senses such as sight, sound, smell and touch are integrated, they are not separated."