McGurk effect
McGurk effect, an audiovisual speech illusion that demonstrates the impact of visual cues on speech perception, particularly the identification of spoken syllables.
The effect was named after cognitive psychologist Harry McGurk, who, along with cognitive psychologist John MacDonald, discovered the phenomenon when they were studying the perception of language by infants. They created a video showing a person articulating the syllable gah dubbed with audio of the syllable bah. Upon playback, they both perceived the syllable dah. They then designed a new study to investigate the phenomenon and found that visual input played a role in speech perception. McGurk and MacDonald reported their findings in the paper “Hearing Lips and Seeing Voices” (1976), which was published in the scientific journal Nature.
When the brain is given incomplete or incongruent input from the senses, it fills in the blanks or makes best guesses to construct reality. In terms of the McGurk effect, when the visual information received from mouth movement does not correspond to auditory information received from spoken sounds, the brain makes a best guess by perceiving a third sound. This third sound consists of a blend of the sounds perceived from visual and auditory cues.
According to research, not everyone is susceptible to the McGurk effect, and the range of susceptibility varies. Studies have shown that women generally experience it more often than men, because visual influences play a bigger role in their speech perception, which also makes them better at understanding speech. Older adults are also more likely to experience the effect because they too are heavily influenced by visual cues when interpreting speech. Children younger than 10 are not as likely to experience the effect because they do not rely as much on the visual aspects of speech perception.
Susceptibility to the McGurk effect also varies from language to language. Research indicates that German, Dutch, Spanish, Italian, and Turkish speakers experience levels of the effect comparable to those of English speakers, whereas Japanese and Chinese speakers are not as prone to experiencing the effect. Cultural and linguistic differences, including the preference for indirect eye contact and simple syllable structures, may account for reduced susceptibility.
When there is a visual distraction, such as a leaf moving across the speaker’s face, the listener is less likely to experience the effect. A listener’s familiarity or unfamiliarity with the speaker also influences the effect. Listeners are less likely to experience the effect when they are familiar with the speaker’s face and are more likely to experience the effect when they are unfamiliar with the speaker’s face. When a listener has clear visibility of the right side of the speaker’s mouth, the listener is more likely to experience the effect, because the right side of the mouth moves more during speech than does the left.
Since its discovery, the McGurk effect has become a useful neuroscience research tool in studies on how the different senses work together, and it has been used to examine audiovisual speech integration among people with certain diseases and disorders. There is less likelihood for individuals with Alzheimer disease, aphasia, autism, dyslexia, schizophrenia, and specific language impairment to experience the effect.
The McGurk effect has also been used to examine the audiovisual speech integration of people with brain damage. Multiple variables determine whether individuals with brain damage experience the effect and, if they do, to what extent. Some variables include the hemisphere in which the damage is located, the handedness of the individual, and whether visual stimuli are used.