In social semiotics, the concept of modality as understood in its grammatical sense refers to the degree to which a sign or a text claims to be a representation of something true or real. This use of “modality” is distinguished from the use of the term in the concept of “multimodality,” which refers to the existence of multiple modes in textual meaning-making. (See entry on multimodal meaning making.)
Kress and van Leeuwen propose that modality is “to some extent” realized “in the form of the message itself,” as the text provides cues about the reliability of messages, guiding receivers in deciding whether what is seen or heard is “true, factual, real, or […] a lie, a fiction, something outside reality” (Kress and van Leeuwen 2006: 154). According to Hodge and Kress (1988: 124), modality “refers to the status, authority and reality of a message, or to its ontological status, or to its value as truth or fact.” Yet Kress and van Leeuwen emphasize that modality judgments are not assessments of the truth or ontological status of a “proposition” as such, but the degree to which the proposition is represented as true or real (2006: 154-55)
In verbal texts, the concept of modality ”refers to the truth value or credibility of (linguistically realized) statements about the world” (Kress and van Leeuwen 2006: 155). In Hallidayen systemic functional grammar, modality, as part of the interpersonal metafunction, is the speaker’s expression of an evaluation of the certainty or probability of what he or she is saying; it is ”the speaker’s judgement, or request of the judgement of the listener, on the status of what is being said” (Halliday and Matthiessen 2004: 172). Modality expresses probability in intermediate degrees between positive and negative polarity: ”What the modality system does is to construe the region of uncertainty that lies between ’yes’ and ’no’” (Halliday and Matthiessen 2004: 176) by expressing, for example, degrees of probability (”different degrees of likelihood”) and usuality (”different degrees of oftenness”) (Halliday and Matthiessen 2004: 177). Verbally, modality is expressed by such resources as modal auxiliary verbs, such as may, will, and must, modal adverbs such as perhaps, probably and certainly and modal adjectives and nouns like probable and probability (see Halliday and Matthiessen 2004 sec. 4.5.2 and 4.5.3; Kress and Van Leeuwen 2006: 155; Van Leeuwen 2005: 162-63). In linguistic modality, according to Halliday (2004:177), “even a high value modal (‘certainly’, ‘always’) is less determinate than a polar form: that’s certainly John is less certain than that’s John; it always rains in summer is less invariable than it rains in summer. In other words, you only say you are certain when you are not.”
In social semiotics, which adapts Hallidayen grammar to the analysis of other sign systems, modality refers to the degree to which things are represented “as though they are real, as though they actually exist in this way, or as though they do not — as though they are imaginings, fantasies, caricatures, etc.” (Kress and Van Leeuwen 2006: 156). Modality is expressed visually through resources indicating increases or decreases in ’”as how real” the image should be taken’ (Van Leeuwen 2005: 166) and includes the following gradable modality markers (Kress and Van Leeuwen 2006: 160-63):
‘(1) Colour saturation, a scale running from full colour saturation to the absence of colour; that is, to black and white.’
‘(2) Colour differentiation, a scale running from a maximally diversified range of colours to monochrome.’
‘(3) Colour modulation, a scale running from fully modulated colour … to plain, unmodulated colour.’
‘(4) Contextualization, a scale running from the absence of background to the most fully articulated and detailed background.’
‘(5) Representation, a scale running from maximum abstraction to maximum representation of pictorial detail.’
‘(6) Depth, a scale running from the absence of depth to maximally deep perspective.’
‘(7) Illumination, a scale running from the fullest representation of the play of light and shade to its absence.’
‘(8) Brightness, a scale running from a maximum number of different degrees of brightness to just two degrees [such as] black and white, or dark grey and lighter grey’.
The sender’s representation and the receiver’s assessment of modality cues are “social, dependent on what is considered real (or true, or sacred) in the social group for which the representation is primarily intended” (Kress and Van Leeuwen 2006: 158). That is, the “issues of representation – fact versus fiction, reality versus fantasy, real versus artificial, authentic versus fake” (van Leeuwen 2005: 160) — with which modality is involved are based on criteria that are socially created and negotiated. Accordingly, as Kress and van Leeuwen point out, visual truth is not absolute: “[r]eality may be in the eye of the beholder, but the eye has had a cultural training and is located in a social setting and a history” (Kress and Van Leeuwen 2006: 158).
Representations and evaluations of modality therefore depend upon the purpose of the representation, the needs of the users and conventions of representation as they are constrained by genre and medium. Thus “the modality value of a given configuration depends on the kind of visual truth which is preferred in the given context” (van Leeuwen 2005:167). To account for this, Kress and Van Leeuwen, adopt the concept of “coding orientation” (Bernstein 1971), and propose four coding orientations as principles deriving from the needs of specific social groups within specific situations (Kress and van Leeuwen 2006: 165-66). The naturalistic, typically found in photographs, relies on the correspondence between the object of representation and how it would be viewed by the naked eye. The technological evaluates modality on the basis of the representation’s effectiveness as a ‘blueprint’ for the user. Sensory modality depends upon the ability of the image to awaken sensory responses, and the abstract, as in works of art or and science, is evaluated on the basis of its fidelity to abstract ideas or essential qualities of phenomena. Coding orientations mean that modality markers (mentioned above) in isolation are not sufficient to evaluate truth value or credibility, but that purpose and context are necessary factors. For example, a high degree of color saturation may mark low modality in the naturalistic orientation, but a higher one in sensory or abstract texts.
Realism and naturalism
Realism and naturalism are terms employed by Kress and van Leeuwen in their discussion of visual modality. In a departure from the understanding of “realism” as denoting a style of mimetic representation, Kress and van Leeuwen define “realism” as representation that is produced by a particular group, on the basis of the practices by which the group is defined and characterized (158); that is, “a motivated sign in which the values, beliefs and interests of that group find their expression” (Kress and van Leeuwen 2006: 158). Thus the four coding orientations above each represent a form of realism. Similarly, in a departure from “naturalism” as a term for a style of depiction of detail in art and literature, it is in Kress and van Leeuwen’s formulation the degree of correspondence “between the visual representation of an object and what we normally see of that object with the naked eye” (Kress and van Leeuwen 2006: 158).
Naturalism is a dominant standard by which visual realism, and hence visual modality, is judged (Kress and van Leeuwen 2006: 161). For example, a photograph that presents itself as “naturalistic, unmediated, uncoded representation” foregrounds content rather than calling attention to its production; thus it will likely be considered to have a high degree of modality and to be interpretable as a representation of something real. Nevertheless, more conspicuously constructed representations, including abstract ones, can have high modality when evaluated according to viewers’ expectations and knowledge of conventions of representation. As Hodge and Kress assert, “Different genres, whether classified by medium (e.g. comic, cartoon, film, TV, painting) or by content (e.g. Western, Science Fiction, Romance, news) establish sets of modality markers and an overall value which acts as the baseline for the genre” (1988:142). Thus the colors in Matisse’s paintings or of toys are not naturalistic, but they can have a high modality on a consideration of their purpose (of sensory perception, or pleasure), just as a line drawing or diagram can have a high modality depending on the criteria established by the group of users for representing something reliably.
Discussion of usefulness of the term
Factoring modality into an analysis can be useful, because it can heighten attention to what the visual or verbal text is an attempt to achieve through its manipulations of the world by means of representation; a receiver may, for example, be able to discern differences between something represented as ”true” or ”real” or as ”too good to be ’true’ or ’real’” which can be a step towards thinking critically about the purpose of a particular representation. Similarly, modality can help us understand how truth or authenticity can be an effect of verisimilitude created textually, without actually having a referential relationship with reality outside it. The effect of authenticity can be achieved through stylistic devices, but authenticity itself is a complex phenomenon relating to the origins of text, its production and its use.
It is useful, therefore, to define modality as as an evaluation of the semiotic construction of truth or realness rather than truth or reality per se. It is possible to make evaluations about how real or true the world is represented in the work, but evaluations of truth/reality status per se require techniques of validation that go beyond the image or text to its actual referentiality, or to the contexts and techniques of production of the work, or even to the intention of the producer and/or sender. Similarly, it is useful to account for how representations that are not naturalistic do not necessarily represent things as untrue or unreal, but can use the conventions of specific genres and affordances of medium to represent meanings that are interpreted as true or real within the scope of the work’s context and purpose.
But it can be relevant to question what there is to be gained from modality judgements as such, especially when we depart from more universal or naturalistic criteria for realism. If we accept modality as one key to the reliability of the message, as Kress and Van Leeuwen assert, we have to ask, for example, what there is to be gained by asking whether an abstract painting or a work of imagination represents the world as true or real, or more precisely, what perception of reality such a work is true to. Further, if we apply criteria based on coding orientations, we must have thorough knowledge of the conventions accepted by the group of users as well as how social and cultural training influences the truth criteria and understandings of the group. Instead of using a limited set of coding orientations, perhaps attention to modality entails investigating whether and/or how a text represents the world in the way expected for at particular genre or purpose, and/or how a text convinces its receiver of reliability. Similarly, a relevant question for investigation would be how a text achieves the effect expected of a particular type of text or genre(?)
Citing this entry
Maagaard, Cindie Aaen. 2015. “Modality.” In Key Terms in Multimodality: Definitions, Issues, Discussions, edited by Nina Nørgaard. www.sdu.dk/multimodalkeyterms. Retrieved dd.mm.yyyy.
References
Bernstein, Basil. 1971. Class, Codes and Control: Volume 1 – Theoretical Studies Towards A Sociology Of Language. London and New York: Routledge.
Halliday, Michael and Christian M.I.M. Matthiessen. 2014. Introduction to Functional Grammar. Fourth Edition. London and New York: Routledge.
Hodge, Roger and Gunther Kress. 1988. Social Semiotics. Cambridge: Polity.
Kress, Gunther and Theo van Leeuwen. 2006. Reading Images. The Grammar of Visual Design. 2nd ed. London: Routledge.
van Leeuwen, Theo. 2005. Introducing Social Semiotics. Abingdon, Oxon.: Routledge.