Studies on the brain responses depending on the congruency between video and odor

Abstract

As attempts to expand audiovisual multimedia using other sensory stimuli increases, olfaction-enhanced multimedia has surfaced. The effects of combining odor with multimedia differ depending on the congruency between odor and video. However, there have been few studies to examine these questions at the neuronal level, which could explain different effects of combining odors with video depending on the congruency. Here, EEG was used to explore frequency bands related to congruency to examine how video and odors are processed in the brain depending on congruency of the pairing. Two video clips were used in this experiment: a flower video and a coffee video. Four odors were used in this experiment, which were divided into two categories: two flower odors (Lavender (FO1) and geranial (FO2)) and two coffee odors (2-furanmethanethiol (CO1) and 2-ethyl-3,5-dimethylpyrazine (CO2)). Two congruent odors and one incongruent odor were paired and shown with each video. Odors were presented with a video clip simultaneously while measuring brain activity by EEG. Congruency between odor and video were confirmed by a congruency survey presented to participants after the EEG experiment. EEG data were analyzed by ERSP and clustered depending on the odor using agglomerative hierarchical clustering (AHC), which was repeated 1000 times. The number of clusterings for each pair (e.g., congruent odors pairs: FO1-FO2, congruent odor-incongruent odor pairs: FO1-CO1, FO2-CO1 in flower video) was counted. The results revealed that flower odors (FO1 and FO2) were similarly congruent to the flower video rather than coffee odor (CO1), and coffee odors (CO1 and CO2) were similarly congruent to the coffee video rather than flower odor (FO1). The number of clusterings of the congruent odor pairs was higher than the clustering of congruent odor-incongruent odor pairs in the theta band for both videos, especially in frontal channels. The current findings suggest that the theta band in the frontal area may be related to processing odor and video depending on the congruency.