Despite its key position in the term ‘multimodality’, the concept of ‘mode’ continues to be elusive. This is in part because some people use the term ‘modality’ instead of ‘mode’, and because ‘modality’ also has another meaning, denoting the semiotic systems which, in different modes, facilitate the expression of the validity of texts and semiotic artefacts, or parts thereof.
In the context of multimodality, ‘mode’ has come to refer to the organized use of material resources for purposes of meaning making. Often the term is simply defined through examples (“modes, e.g. language, images, music”), and even when it is defined more closely, the definitions may leave open how modes are semiotically organized, as in some of Kress’ definitions, e.g. “Mode is a socially shaped and culturally given semiotic resource for making meaning” (2010: 79), and “Mode is that which a community, a group of people who work in similar ways around similar issues, has decided to treat as a mode” (in Andersen et al 2015: 77). The stress on social ‘shaping’ in these definitions is very important, but as mode has generally been taken to include language, should definitions also specify in which respects other modes are like language, or, to put it another way, what aspects of semiotic organization all modes have in common? This is done, for instance, by Lemke (in Andersen et al 2015: 126), who defines mode as “a system of meaningful contrasts between forms in a community that has conventions for the interpretation of those forms and contrasts as paradigms and syntagms”.
In Multimodal Discourse (2001) Kress and Van Leeuwen distinguish between ‘mode’ and ‘medium’. ‘Modes’ are defined as abstract ways of organizing meaning making which can realize ideational, interpersonal and textual meanings, and which can do so, in principle, in materially different media, with sound, graphically, or in both media. Thus language is a mode because it can realize all three metafunctions and do so either in the form of speech or in the form of writing. And according to Kress and Van Leeuwen (2006), visual communication is also a mode because it can also realize all three metafunctions and do so in materially different ways – as drawings, photographs, paintings, etc.
‘Media’ they define as the resources (including bodily articulation through speech, gestures etc) that materialize meaning but that themselves can also produce meaning, whether directly, on the basis of a creative use of the affordances of the materials, or, less creatively, on the basis of an unsystematic ‘lexicon’ of such creative uses that have become clichés. As Van Leeuwen (2011) has argued, these additional meanings today focus especially on the expression of identity and on textual cohesion.
Kress and Van Leeuwen then distinguish between production media and distribution media. The latter serve mainly to record or distribute, but the way in which they do so can itself express cultural values, and the distinction between production media and distribution media is not watertight – technical distribution media often become involved in the creative process.
The term ‘resources’, as used, by Van Leeuwen (2005) is an over-arching category, encompassing both non-material ‘mode’ resources such as knowing how to construct a sentence or a narrative, and material media such as our body, or pen and paper, cameras and computers, etc. In the age of artificial intelligence the distinction between material and immaterial resources can be blurred as semiotic knowledge may be built into material tools.
The term ‘mode’ does not by itself imply multimodality and accounts of specific modes often assume that modes can be used independently, ‘mono-modally’. Multimodality then becomes the joining of already articulated ‘messages’, and the study of multimodality a matter of, for instance, ‘text-image relations’. But today, ‘modes’ often integrate much more closely and interdependently. In diagrams, for instance, the linguistic elements may be ‘participants’ and the visual elements (e.g. arrows) the processes in complex multimodal transitivity structures. In this case neither the words (nouns or nominal groups in boxes) nor the arrows constitute a complete ‘message’ on their own.
Also, many of the ‘modes’ which play a key role in contemporary communication are, to use Christian Johannessen’s term, ‘supra-modal’, used in connection with many other ‘modes’ but not capable of being used independently. Colour, for instance, is used in typography, graphic design, image making, dress, architecture, and so on, but rarely on its own (perhaps we should refer to such ‘modes’ as semiotic practices which use semiotic resources in specific ways, rather than as modes).
Inevitably all this should shift the emphasis from individual modes to principles of multimodal integration. Perhaps this is the reason that ‘mode’, which only yesterday seemed such a clear concept that it hardly needed definition, has today become so elusive.
Citing this entry
Holsting, Alexandra and Van Leeuwen, Theo. 2016. “Mode.” In Key Terms in Multimodality: Definitions, Issues, Discussions, edited by Nina Nørgaard. www.sdu.dk/multimodalkeyterms. Retrieved dd.mm.yyyy.
References
Andersen, T.A., Boeriis, M., Maagerø, E. and Seip Tønnesen, E., eds. 2015. Social Semiotics – Key Figures, New Directions. London: Routledge.
Kress, G. 2010. Multimodality – A social semiotic approach to contemporary communication. London: Routledge.
Kress, G. and Van Leeuwen, T. 2001. Multimodal Discourse – The Modes and Media of Contemporary Communication. London: Arnold.
Kress, G. and Van Leeuwen, T. 2006. Reading Images – The Grammar of Visual Design. 2nd Edition. London: Routledge.
Van Leeuwen, T. 2011. The Language of Colour. London: Routledge.