Abstract
The McGurk effect is a sensory phenomenon based on the ability of the brain to process and interpret a certain type of information by relying on the data from other sensory means; in particular, to correct auditory data in accordance with received visual clues. Despite being extensively studied in terms of the relevant factors, very little research exists on the McGurk effect outside the English-speaking audience. This project aims at confirming the hypothesis that the effect is observable, and retains the same robustness among Arabic speakers, to the same extent as in the English-speaking population.
Introduction
Language recognition is a key factor that allows the existence of the complex phenomenon of language in human culture. While some mammals demonstrate similar abilities, none of them approach the level of complexity characteristic of humans. In part, this result is possible, thanks to the fact that sensory perception usually involves more than one sense in obtaining and interpreting information.
In the case of spoken language, the information is derived from the auditory source. However, as oral communication has been traditionally conducted face-to-face, the sensory system usually accounts for other available signals, primarily the visual information. Thus, the comprehension of the speech is augmented by visual factors like the lip movement. This may lead to the emergence of illusions, like the McGurk effect, a perceptual phenomenon that results from the difference in auditory and visual feedback.
When the lip movement is inconsistent with the received sound, the brain produces information in a different way, leading to the receiving of the sound that is different from the actual recording. The effect has been extensively studied since its accidental discovery in 1976 (McGurk & MacDonald, 1976) and has shown to work in different age groups (Rosenblum, Schmuckler, & Johnson, 1997), in varying brain-prestimulated conditions (Keil, Müller, Ihssen, & Weisz, 2012), and independently from the involvement of the motor regions of the nervous system (Matchin, Groulx, & Hickok, 2014).
The intensity of the effect has also been shown to be influenced by the multitude of conditions, like the previous exposure to similar discrepancies (Boersma, 2012), the varying levels of the reliance on auditory information (Massaro & Cohen, 2000), and various brain disorders. The majority of the results suggest that the effect is cognitive in origin and is inherent to the basic functioning of humans’ brains. However, the vast bulk of the research was conducted on the English-speaking audience. The aim of this project is thus to replicate the McGurk experiment using the Arabic language. As the effect is presumably not influenced by a specific culture, the initial hypothesis is that the results would be the same for the Arabic speakers.
Method
Participants
Ten female participants have been selected for the participation form a group of fifty by performing simple randomization using the online randomization tool Random.org. All of the participants are native Arabic speakers of the age stratum 20 through 24. The close age range should not be a source of bias as the previous research has concluded the age does not play a significant role in the strength of the response (Rosenblum et al., 1997).
Stimuli
Two videos were created to be presented to the participants. The first video shows the close-up sequence of lips articulating the word “Bar” repeatedly while the corresponding sound is played synchronously. The second video uses the same audio track but features the lips articulating the word “Far”. The sound in both videos was carefully synchronized. Both videos were two minutes in length.
Apparatus
Both videos were presented on a 15.6-inch screen of a laptop as no high-fidelity output is required for the experiment.
Design
The participants were shown the videos in three successions of three, four, and three, respectively. Each show involved one participant at a time.
Procedure
Each of the participants was shown the first video, which contained the “Bar” articulation. After that, she was asked which sound she had heard. After that, the same participant was shown in the second video. After 20 to 30 seconds into the video, she was asked about the sound she heard in the second video. After receiving the response, the instructor has asked the participant to close her eyes and report if the sound has remained the same. In total, three responses were collected from each participant: one from the first video and two from listening to the second one, both with and without visual aid.
Results
All of the participants have recognized the word from the first video as “Bar.” After watching the first 20 to 30 seconds of the second video, all ten participants reported hearing the word “Far.” After being instructed to close their eyes, all ten has reported the change of the sound to “Bar.” Thus, all of the participants have shown identical results without a single deviation. Additionally, none of the participants has shown doubt regarding the perceived sound. The time between the question and the response in all three instances was no more than two seconds.
Discussion
The results align with the initial assumption that the participants will be susceptible to McGurk effect regardless of the language. The unanimity of results also confirms the established understanding of the robust nature of the effect, as it is influenced little by the awareness of the participant. In this particular case, no specific disclaimer was made to the listeners. Nevertheless, the transparent nature of the experiment did not conceal the change occurring in the second session.
Despite this, the responses remained the same. Nevertheless, it is important to stress that according to the findings of Boersma (2012), the effect heavily depends on the respondent’s previous experience with the asynchronous presentation of auditory and visual material.
While the said findings mostly deal with individual occurrences, like the study of people who are used to watching dubbed films, it can arguably be extended to a national scale, for example to the cultures that tend to rely less on the visual contact. This, again, can not be considered a language issue, but rather cultural in origin. In this regard, the result also suggests that the participants belong largely to the same cultural stratum as the majority of the respondents from the previous studies.
Limitations
The study has used a limited number of participants, which was sufficient to confirm the generalized hypothesis but not enough to make a detailed assessment. Besides, the sampling process did not account for the possible differences in the physical condition of the participants, which have been proven to visibly alter the results (Massaro & Cohen, 2000). Thus, the research should be viewed only as a confirmation of repeatability of the observable effect.
References
Boersma, P. (2012). A constraint-based explanation of the McGurk effect. Phonological Explorations: Empirical, Theoretical and Diachronic Issues, 548, 299-312.
Keil, J., Muller, N., Ihssen, N., & Weisz, N. (2012). On the variability of the McGurk effect: audiovisual integration depends on prestimulus brain states. Cerebral Cortex, 22(1), 221-231.
Massaro, D. W., & Cohen, M. M. (2000). Tests of auditory–visual integration efficiency within the framework of the fuzzy logical model of perception. The Journal of the Acoustical Society of America, 108(2), 784-789.
Matchin, W., Groulx, K., & Hickok, G. (2014). Audiovisual speech integration does not rely on the motor system: evidence from articulatory suppression, the McGurk effect, and fMRI. Journal of Cognitive Neuroscience, 26(3), 606-620.
McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
Rosenblum, L. D., Schmuckler, M. A., & Johnson, J. A. (1997). The McGurk effect in infants. Perception & Psychophysics, 59(3), 347-357.