A synaesthesia-like effect in which people ‘hear’ silent flashes or movement, such as in popular ‘noisy GIFs’ and memes, could be due to a reduction of inhibition of signals that travel between visual and auditory areas of the brain, according to a new study led by researchers at City, University of London.
The study is the first to provide insight into the brain mechanisms underpinning such auditory sensations also known as a ‘visually-evoked auditory response’ (aka vEAR or ‘visual ear’).
Whilst one theory is that areas of the brain responsible for visual and auditory processing normally compete, this research suggests that they may actually cooperate in people who report visual ear.
It was also found that musicians taking part in the study were significantly more likely to report experiencing visual ear than non-musician participants. This could be because musical training may promote joint attention to both the sound of music and the sight of the coordinated movements of the conductor or other musicians.
Dr Elliot Freeman, Principal Investigator on the study and a Senior Lecturer in Psychology at the University said:
“We already knew that some people hear what they see. Car indicator lights, flashing neon shop signs, and people’s movements as they walk may all trigger an auditory sensation.
“Our latest study reveals normally-occurring individual differences in how our senses of vision and hearing interact.
“We found that people with ‘visual ears’ can use both senses together to see and also ‘hear’ silent motion, while for others hearing is inhibited when watching such visual sequences.”
Some neuroscientists believe visual-ear may be a type of synaesthesia, with other examples including music, letters or numbers that can evoke perceptions of colour. However, visual ear appears to be the most prevalent, with as many as 20% of people reporting some experiences of it compared to 4.4 per cent for other types.
The condition has received more attention due to the recent, viral popularity of the ‘skipping pylon GIF‘, and other ‘noisy GIFs’ depicting silent motion, which in some people evoke very vivid visual ear sensations.
To shed light on what may be going on in the brain when people view such content, the researchers applied a weak alternating current to participants’ scalps, using a technique called transcranial Alternating Current Stimulation (tACS), to explore how the visual and auditory parts of the brain interact in those who experience visual ear and those who don’t.
The first experiment of the study included 36 healthy participants, including 16 classical musicians from the London Royal College of Music. All were shown auditory and visual ‘Morse code’ sequences, while electrical simulation (tACS) was applied to either the back of the head (visual areas of the brain) or the sides (auditory areas) using ‘alpha-frequency’ tACS stimulation. Participants were then classified as visual or non-visual ear depending on whether they reported ‘hearing’ the silent flashes.
The researchers found that in non-visual ear participants, alpha-frequency stimulation to auditory areas significantly reduced auditory performance but improved visual performance, while the opposite pattern was found for the same frequency of stimulation to visual areas (poorer vision, better audition).
This reciprocal pattern suggests a competitive interaction between visual and auditory brain areas with each normally inhibiting the performance of the other.
However these interactions were strikingly absent in visual ear participants, suggesting that their auditory and visual areas were not competing but cooperating with each other.
A second experiment was conducted to see whether even people without conscious awareness of ‘visual ear’ sometimes use their auditory brain areas to make purely visual judgements. It found that this might indeed be the case for some, where stimulation to auditory areas of the brain affected accuracy of visual judgements almost as much as stimulating visual areas.
Taken together, the results of these experiments support a popular theory that some kinds of synaesthesia may depend on a disinhibition of pre-existing neural cross-connections between sensory brain areas that are normally inactive. When these connections are disinhibited this may result in conscious awareness of visual ear and other synaesthetic phenomena.
Dr Freeman said:
“We were also interested to find that, on average, participants with visual ear performed better on both visual and auditory tasks than those without. Perhaps their audio-visual cooperation benefits performance because more of the brain is engaged in processing visual stimuli.
“Such cooperation might also benefit musical performance, explaining why so many of the musicians we tested reported experiencing visual ear.”
