Musical wine glasses

By Bruno Mesz, Juan Cruz Amusategui and Sebastián Tedesco

A number of recent studies have analyzed the influence of music and sound on food or drink evaluation (Spence, C. ,2016). Crossmodal taste-sound congruence has been shown to have a perceptual impact on taste intensity and other qualities such as bitterness or sweetness (Reinoso Carvalho, F., et al. ,2017).

In the case of wine, temporal methods in sensorial analysis have revealed an impact of music on sweetness and sourness of a white wine (Wang et al., 2017).

As multisensorial integration is more likely to occur in conditions of spatial and temporal proximity between the stimuli in different modalities (Chen, L., and Vroomen, J. (2013).), and music and taste are time-varying in nature, it seems interesting to develop technologies that favor these situations of synchronization and co-location, while allowing to keep track of multisensory interactive effects along time.

Augmented glass (Figure 1 and 2): This device consists in a wine glass with sensors that are able to detect three main gestures of the user: when the cup is taken by the hand, airing the wine and when the liquid contacts the mouth while drinking. The sensing is performed through electrodes attached to the surface of the cup, next to the edge, and connected to capacitive sensors. A soundtrack is activated for each gesture, intended to emphasize sweetness, sourness or bitterness of the wine, each soundtrack being congruent to the intended taste according to sound-taste crossmodal correspondences (Mesz et al., 2011).

Methods: These glasses are intended as exploratory devices to be used outside the laboratory, for instance at wine bars, or as part of artistic installations. We try to gather evidence for (or against) the conclusions that we draw from formal experiments at the lab, by asking people about the perceived effects of music on wine at informal settings.

So far, we used the glasses in 3 different settings:

1) Piano Bar: We implemented a pilot exploration of the Augmented Glass by making it activate motors and transducers placed in an upright piano (Figure 3) (video: https://vimeo.com/245263256) and asking users to drink red and white wine with different sounds (low and high pitched) (Figure 4). From this initial experience we collected some opinions of the users, who in all cases enjoyed the interaction with the glasses: “this sound makes the wine feel like a Rutini [an expensive Argentine red wine], with strong body and expanding taste”, “the low pitched sounds seem to give weight to the wine”, “the pizzicatos [probably referring to the motors bouncing over the strings] bring forward sour fruit notes”. Also, high pitched sounds were judged to be more congruent to white wine, and low-pitched sounds to red wine.

2) At the Brain and Wine Symposium in Barcelona (2018): the glasses activated, with each sip, electroacoustic sounds presented over headphones (Figure 5), intended to be congruent with young and aged wines served in the glasses. Natural soundscapes were played when the public grebbed the glasses, while windy sounds were produced by the act of airing the wine. In this event we didn´t ask for specific shifts in taste produced for the music, we just collected general impressions of the people who interacted with the device, who manifested enjoying the experience.

3)Recently at an event organized at a wine bar in Buenos Aires for Malbec´s Day (17 April), we presented again the glasses, but now with the sounds emerging from loudspeakers so everyone could listen them. We took care that, at a given time, only sounds congruent to a given taste (sweet, bitter or sour) were present, so there was no “taste mixing” in the music.

People at the event (N=23 participants) reported noticing changes of wine taste depending on the music. All participants at the event except one (N=22) reported intensification of acidity in the wine with high pitched and dissonant sounds, and no effect on sweetness or bitterness. One participant specified that the changes were on taste and not on smell perception. The only participant not reporting effects on acidity reported an effect of music on sweetness perception and no effect on acidity of bitterness.

Conclusions:

• Tasting wine while listening to different soundtracks may lead to different perceptions of flavours. This supports our quantitative study on the impact of music on wine taste, presented also in this congress.
• Support for the theory that what we hear can (perhaps automatically) shift our attention to specific taste/flavours that correspond to the sound.
• Support for anecdotal report that changes in auditory stimulus can quickly influence taste evaluation.

BRUNO MESZ: musician and mathematician. As a researcher, he works on sound perception, musical semantics, multisensory perception and mathematical models of musical evolution at the Centro de Investigación en Arte, Materia y Cultura (IIAC) / MUNTREF Centro de Arte y Ciencia of the Universidad Nacional de Tres de Febrero, Buenos Aires, Argentina, where he directs the projects “Transmodal associations between audition, taste and olfaction” and “Musical Evolution”. He is a pianist and clarinetist, specialized in contemporary music. He has played in Argentina, Italy, Germany, Scotland, Finland, Spain and France as a soloist and with several ensembles such as the Ensemble Modern, KNM Berlin, Perceum, among others. He has participated in world and Latin American premieres of works by Steve Reich, Walter Zimmermann, Peter Ablinger, among others.

SEBASTIAN TEDESCO: Visual artist and industrial designer. He researches the relationships between old and new technologies and between art and other disciplines. He is currently Academic Coordinator and Professor of the postgraduate specialization in Conceptual Design at UNTREF (Universidad Nacional de Tres de Febrero, Buenos Aires, Argentina); Coordinator of the Suspended Borders project at the Museum of Contemporary Art of Mar del Plata, and researcher at the Centro de Investigación en Arte, Materia y Cultura (IIAC) / MUNTREF Centro de Arte y Ciencia of the Universidad Nacional de Tres de Febrero where he co-directs the project “Transmodal associations between audition, taste and olfaction” together with Bruno Mesz. Since 2013 he exhibits his work in individual and collective exhibitions in Argentina and abroad, highlighting: Itaú Award for Visual Arts / FILE 2015 / Agency for Unrealized Projects, e-flux and Serpentine Gallery / The Wrong New Digital Biennale / TEDx Riodelaplata / #ProgramaLaPlaza 2014, Medialab-Prado / Prize Faena a las Artes, Faena Arts Center / MACBA / PROJECTOR 9no. International Video-art Festival of Madrid / CCK – Kirchner Cultural Center / 3rd Istanbul Design Biennial / MALBA.

JUAN CRUZ AMUSATEGUI: Lives and works in Buenos Aires. He is currently a student of Electronic Arts at the National University of Tres de Febrero, and is a research student of the “Transmodal associations between audition, taste and olfaction” project at the Centro de Investigación en Arte, Materia y Cultura (IIAC) / MUNTREF Centro de Arte y Ciencia of the Universidad Nacional de Tres de Febrero. He participated in several projects related to arts, technology and software.

References

1. Spence, C. (2016). Sound–the forgotten flavour sense. Multisensory Flavor Perception: From Fundamental Neuroscience Through to the Marketplace, 81.
2. Reinoso Carvalho, F., et al. (2017). “Smooth operator”: Music modulates the perceived creaminess, sweetness, and bitterness of chocolate. Appetite, 108, 383-390.
3. Wang, Q. J., Mesz, B., & Spence, C. (2017, November). Assessing the impact of music on basic taste perception using time intensity analysis. In Proceedings of the 2nd ACM SIGCHI International Workshop on Multisensory Approaches to Human-Food Interaction (pp. 18-22). ACM.
4. Chen, L., and Vroomen, J. (2013). Intersensory binding across space and time: a tutorial review. Attention, Perception, and Psychophysics, 75(5), 790-811.
5. Mesz, B., et al. (2011). The taste of music. Perception, 40(2), 209-219.