In 1976, Harry McGurk and his research assistant John MacDonald were researching how infants perceive speech as they develop. Some of their experiments involved separating visual and auditory stimuli and seeing how the children learned, such as playing the video of a mother speaking in one area and having the audio play in another. In this same vein they took two phonemes, “ba” and “ga”, and then merged the audio of “ba” with the video of “ga”. And just like a mad science experiment gone wrong, they created a third sound: “da”.
At first McGurk and MacDonald thought there was some audio error at play or technical mixup, but further testing confirmed that they discovered something new. Listening to the audio alone lead the person to hear “ba”, but watching the video with the wrong audio made them hear “da”. This phenomenon is called the McGurk Effect and you can see and hear it in this BBC2 video, pictured above. To really appreciate the effect though, watch this video and then replay it with your eyes closed.
What’s going on? In order to perceive speech, our minds merge both auditory and visual information, overlaying them in an effort to decipher words. When “ba” and “ga” are merged we hear the middle sound of “da”, or as in the BBC2 video, “ba” and “va” become “fa”. Interestingly, people in Japan experience a weaker McGurk Effect than English speakers because the former don’t look at each other as often during conversation. Similarly, people who watch dubbed movies are less susceptible to the McGurk Effect because they’ve trained their brains to dissociate audio and video speech processing. This indicates that the effect is learned instead of something we’re born with.
Today’s Tangent: The McGurk Effect is something that can’t be easily overcome; even those who have researched it for years and understand how it works are still affected by it. In contrast, many optical illusions can be easily flipped between by knowing that there are two options, such as Ruben’s Vase (a vase with two faces surrounding it). More interesting is the Spinning Dancer as the silhouette is seen spinning clockwise twice as often as counter-clockwise, though the animation never changes. The reason? There’s a slight vertical angle and more people imagine looking down at the dancer instead of looking up. In fact, the original animation was taken at a slightly downward angle, making clockwise the true rotation.