This is an interactive exploration of the data from Cuskley, C.*, Dingemanse, M.*, van Leeuwen, T. & Kirby, S. 2019. Cross-modal associations and synaesthesia: Categorical perception and structure in vowel-colour mappings in a large online sample. Behaviour Research Methods, doi: 10.3758/s13428-019-01203-7.
In the broadest terms, the experiment reported in the paper asked participants to listen to vowels and then choose colors for them over three trials. If they were consistent across trials, they were classified as synaesthetes. We also tested whether participants structured the colours they chose to mirror structure in the vowel space - in other words, if they chose similar colours for similar sounding vowels. The vowel space can be described in acoustic terms (i.e., in terms of measuring the acoustic properties of a vowel) or articulatory terms (i.e., in terms of how they are formed with the mouth. You can view the data from both these perspectives: choose "Acoustic View" or "Mouth View".
This interactive visualisation allows you to explore the associations of individual particiapnts from both perspectives. To see a participant's data, hover over a circle in the structure/consistency plot (on the left or top, depending on your screen dimensions). The participant's associations will appear next to or below the main structure consistency plot (depending on your screen dimensions). Below provides a more detailed description of the Mouth View and Acoustic view.
- Mouth View (default) shows the approximate tongue positions associated with each vowel sound, coloured according to a random trial from the participant. This view shows more about how associations are structured (i.e., vowels with similar articulation have similar colours) than whether participants are consistent across trials - however, this is reflected in consistency on the y axis of the plot to the left, and participants who were more consistent (lower values on the y axis) tended to have more structured associations. To hear vowel sounds: roll over the circle associated with the vowel/color. When it gets larger, click to play the sound file.
- Acoustic View shows the vowel sounds in acoustic space, which linguists may be more used to seeing. This corresponds roughly to the outer edges of a traditional vowel trapezoid. In this view, you can see all three trials for each participant, so inconsistency (higher values on the y-axis) is more obvious. This also shows the phoneme groupings from Dutch which seem to underlie the observed associations.To hear vowel sounds: roll over any of the circles associated with the vowel. When they get larger, click to play the sound file.
Participants to the right of the blue line on the x axis have significantly structured associations (distances in color space correlate with distances in phonetic space). Participants below the green line on the y axis have consistent color choices across trials sufficient to be considered synaesthetic, after criteria laid out in Rothen, Seth, Witzel & Ward (2013). Vowel stimuli were kindly provided by Anja Moos, and were first used in Moos, A., Smith, R., Miller, S.R., & Simmons, D.R. (2014).
For more information on the precise methods and values, see the full paper. The code used to run the experiment and the resulting data are both freely available on GitHub.