This entry focuses mainly on narrative in visual images, particularly monophase (single frame) images.
As a mental activity, narrative refers to the operation of constructing or re-constructing a meaningful structure out of a sequence of disparate events that occur over time. As an object, a narrative is the construction (virtual or material) that arises from this activity. The narrative operation thus has the potential to create “both an abstract cognitive structure and the material trace of that structure left in writing, speech, sign-language, three-dimensional visual images or some other representational medium” (Herman 2003, 170). Therefore, narratives can be represented and communicated.
Regardless of approach, temporality is understood to be a defining feature of narrative, both
in the event or events being narrated (the “story” or “fabula”) and in the narrative construction into which events are organized (the “discourse” or “sjuzet”). A single, still image lacks the capacity for narrating a sequence of events (as is possible, for example, in the verbal clause), and it arranges its objects primarily spatially and thus does not unfold temporally in the same way as speech, writing and film (see Ryan 2012; Wolf 2005). Moreover, in the absence of verbal narration, the work does not report what people think and feel, although it does show outward physical signs of inner life, such as emotional states or responses.
Although still visual images in themselves possess no temporal progression, they may have narrative potential — that is, they can posses what narratologists refer to as “narrativity,” the ability to stimulate a narrative script in the mind of the receiver (Ryan 2012, 2005). But how does narrativity arise?
To understand this, narratologists tend to try to apply concepts from analyses of verbal narrative, emphasizing the role of contextual factors, background knowledge and cognition in guiding narrative interpretations. Focus is on how the frozen “pregnant moment” of the image (Lessing 1766/1984) may compel the viewer to construct a larger narrative arc — an inferred “story” or “fabula” (see Kafalenos 2001; Ryan 2012) that temporally precedes and follows the depiction of the image.
While also considering contextual and communicative factors, a social semiotician approaches the analysis of the image through its visual grammar, and the socially constructed meanings of image-internal features, locating there representations of unfolding actions, movements and relations. This approach does not necessarily view the lack of verbal narration as an obstacle to the expression of narrative.
Kress and van Leeuwen’s discussion of narrative in Reading Images
A social semiotic approach to visual analysis recognizes the narrative potentials in images lacking temporal progression. The approach appropriates the categories of linguistic description and analysis used by Hallidayan systemic functional grammar. Importantly, this does not mean equating what visual and verbal representations can achieve narratively, as images and verbal language are “not simply alternative means of representing ‘the same thing’” (2006: 76). Verbal and visual representations can express similar meanings, but not identical ones, and they achieve this differently. It is thus not always possible to translate grammatical structures of the clause to visual ones and vice versa. (Halliday 1994; Kress and van Leeuwen 2006).
Kress and van Leeuwen locate the narrative meaning of visual images primarily within the ideational metafunction. More specifically, it is understood to be the capacity of images to represent the world “’narratively’ — that is, in terms of ‘doing’ and ‘happening’” (Kress and van Leeuwen, 2006: 73). They distinguish directionality, movement and interaction through the representation of participants, human actors, vectors and eyelines.
This definition of narrative proceeds from the assumption that images make “meaningful propositions by means of visual syntax” (2006: 47), which differs from narratologists’ view that “pictures cannot make propositions.” Put differently, while verbal texts use language both to “show” and to “tell,” that is, to make propositions, visual images “show.” Accordingly, still images can represent “narrative patterns [that] serve to present unfolding actions and events, processes of change, transitory spatial arrangements” (Kress and van Leeuwen 2006: 59).
According to Kress and van Leeuwen, a narrative visual “proposition” is realized by the presence of a visible oblique vector (Kress and van Leeuwen 2006: 59) emanating from a participant in the image. The vector constitutes a “visual verb,” a representation of realized action or transaction. Thus a vector prompts the viewer to infer direction, motion or action that may or may not have a receiver (a “goal” or “phenomenon”), with the more or less explicit head of the arrow indicating in which direction and at whom or what or the action is aimed. This is reflected in the terminology, in which “actor” refers to the “participant from which the vector emanates, or which itself, in whole or in part, forms the vector” (Kress and van Leeuwen 2006: 63). The “goal” is the “participant to whom or which the action is done, or at whom or which the action is aimed” (2006: 64). Vectors can indicate action that is transactional, or transitive, and non-transactional, or intransitive (2006: 63), so that non-transactional action “has no ‘goal’, is not ‘done to’ or ‘aimed at’ anyone or anything” (2006: 63).
Further distinctions are made when implicit vectors represent the act of looking: “when the vector is formed by an eyeline, by the direction of the glance of one or more represented participants, the process is reactional” (2006: 67). In such cases, actors are termed “reactors” and goals “phenomena.” The phenomena themselves may be other narrative scenarios (2006: 67).
Kress and van Leeuwen exemplify their narrative analysis with representations that range from abstract art to academic diagrams, to more realist images in drawings and photographs. Kress and van Leeuwen recognize the role of contextual knowledge in guiding the viewer’s interpretation: “in ‘realist’ images [as opposed, say, to abstract art], the context usually makes clear what kind of action the vectors represent” (2006: 60).
Thus, like narratologists, they recognize that the field of possible meanings is widest and least determinate where the coding is abstract or diagrammatic and provides few contextual clues; such meanings can be “broad, abstract and hence difficult to put into words” (2006: 61).
In sum, still images can represent “narrative patterns [that] serve to present unfolding actions and events, processes of change, transitory spatial arrangements” (Kress and van Leeuwen 2006: 59). Nevertheless, “non visual phenomena cannot directly be related in the visual semiotic” (2006: 77). This is also where analytical focus on narrative as action and process approaches limits, particularly in light of the fact that narration itself is often the result of the narrator’s felt need to relate a highly subjective experience of actions and processes as well as their emotional significance (for more on the narrator’s evaluation of significance see Labov 1972; Fludernik 2003).
Painter, Martin and Unsworth on Reading Visual Narrative
Claire Painter, Jim Martin and Len Unsworth account for aspects of such subjective experience in their book Reading Visual Narrative (2013), in which they analyze children’s picture books. Their focus is not specifically on the monophase image alone, but on images participating in sequences, part of the narrative whole of a book that combines words and images. In addition to ideational meanings of narrative, they are interested in how interpersonal meanings contribute to narrative, for example their role in creating social relations through engaging the reader and the work, and they examine the role of textual, or compositional, meanings — such as framing or creating bound or unbound images — in creating the sense in the reader of the storyworld being open and accessible or closed and at a remove.
Their work supplements the social semiotic approach through a consideration of the role of interpersonal meanings, for example in their discussion of “affect” and their consideration of “focalization,” a term from narratology, proposed by Gerard Genette (1972) and reconceptualized both by him and by many other narratologists. Both affect and focalization are important to narrative meaning-making, as they are part of how receivers are positioned by the image, and thus invited to respond to it. Both contribute to an understanding of how visual depictions prompt emotional reactions in receivers, and thus also the degree to which receivers are stimulated to shape a narrative around the image. They assert that “our response to the affect of a depicted character is a relevant dimension in our positioning, regardless of whether we are gazed at” (Painter, martin and Unsworth, 2013: 19).
Positioning of the viewer also takes place by means of focalization, a concept from narratology that grows out of the analytical distinction between two different narrative functions: that of on the one hand telling and on the other seeing (Genette, 1972: 186) or perceiving (see e.g. Bal 1997). Painter, Martin and Unsworth use this distinction analytically, examining how visible representations create the illusion of characters and viewers positioned physically in relation to the image, and as if they are actually seeing, or even interacting with, the objects which it represents.
To sum up, narratological concerns about narrative across media are largely concerned not only with their content — their themes and messages — but also with understanding how narrative inferences are stimulated through the narrativity of signs. Social semiotic analysis is concerned to understand how functional, social meanings are realized by the grammar of the signs of the image.
While this entry has focused on visual narrative in the discussion of combined narratological and social semiotic approaches, the issues of narrativity also arise in other sign systems and media, and has been studied by narratologists interested in narratological concepts as applied to areas including film (see for example Bordwell 1985; Chatman 1978; Grodal 2005; Kozloff 1988; Kuhn 2011; Thompson 2003) music (see for example Nattiez 1990; Rabinowitz 2004; Seaton 2005; Tarasti 2004) and dance (see for example Erken 2012; Foster 1966; Midgelow 2007).
Citing this entry
Maagaard, Cindie Aaen. 2016. “Narrative and Narrativity.” In Key Terms in Multimodality: Definitions, Issues, Discussions, edited by Nina Nørgaard. www.sdu.dk/multimodalkeyterms. Retrieved dd.mm.yyyy.
References
Bal, Mieke. 1997. Narratology: Introduction to the Theory of Narrative. Toronto: U of Toronto P.
Bordwell, David. 1985. Narration in the Fiction Film. Madison: Univ. of Wisconsin Press.
Chatman, Seymour. 1978. Story and Discourse. Narrative Structure in Fiction and Film. Ithaca: Cornell Univ. Press.
Emily Alane Erken. 2012. “Narrative Ballet as Multimedial Art: John Neumeier’s The Seagull.”
19th-Century Music. 36. 2. 159-171.
Fludernik, Monika. 2003. “Natural Narratology and Cognitive Parameters.” In D. Herman (ed.) Narrative Theory and the Cognitive Sciences. Stanford: CSLI, 243-67.
Foster, Susan Leigh. 1996. Choreography and Narrative. Bloomington: Univ. of Indiana Press.
Genette, Gerard. 1972/1980. Narrative Discourse. An Essay in Method. Trans. Jane E. Lewin. Foreword Jonathan Culler. Ithaca: Cornell Univ. Press.
Grodal, Torben. 2005. “Film Narrative.” D. Herman et al. (eds). Routledge Encyclopedia of Narrative Theory. London: Routledge, 168–72.
Halliday, Michael. 1994. An Introduction to Functional Grammar. London/Sydney: Edward Arnold.
Herman, David. 2003. “Stories as tools for thinking.” Narrative Theory and the Cognitive Sciences. Ed. David Herman. Stanford, CA: CSLI. 163-192.
Kafalenos, Emma. 2001. “Reading Visual Art, Making — and Forgetting — Fabulas.” Narrative 9.2. 138-145.
Kafalenos Emma. 2004. “Overview of the Music and Narrative Field.” In M-L Ryan (ed). Narrative Across Media: The Languages of Storytelling. Lincoln: Univ. of Nebraska Press. 275-282.
Koloff, Sarah. 1992. Invisible Storytellers. Voice-Over Narration in American Fiction Film. Berkeley: Univ. of California Press.
Kress, Gunther and Theo Van Leeuwen. 2006. Reading Images. The Grammar of Visual Design. 2nd ed. London: Routledge.
Kuhn, Markus. 2011. Filmnarratologie. Ein erzähltheoretisches Analysemodell. Berlin: de Gruyter.
Labov, William. 1972. Language in the Inner City. Studies in the Black English Vernacular. Philadelphia: Univ. of Philadelphia Press, 1972.
Lessing, Gotthold E. 1766/1984. Loacoon. An Essay on the Limits of Painting and Poetry. Baltimore: Johns Hopkins University Press.
Midgelow, Vida L. (ed.) 2007. Reworking the Ballet. Counter-narratives and alternative bodies. London and New York: Routledge.
Nattiez, Jean-Jacques. 1990. “Can One Speak of Narrativity in Music?” Journal of the Roya Musical Association 115. 240-257.
Painter, Clare, Jim Martin and Len Unsworth. 2013. Reading Visual Narratives: Image Analysis of Children’s Picture Books. Sheffield and Bristol: Equinox.
Rabinowitz, Peter. 2004. “Music, Genre, and Narrative Theory.” In M-L Ryan (ed). Narrative Across Media: The Languages of Storytelling. Lincoln: Univ. of Nebraska Press. 305-328.
Ryan, Marie-Laure. 2012. Narration in Various Media. In P. Hühn et al. (ed). The Living Handbook of Narratology. Hamburg: Hamburg University Press. Accessed 17 Jan. 2012.
Ryan, Marie-Laure. 2005. “On the Theoretical Foundations of Transmedial Narratology.” In Jan Chr. Meister (ed.) Narratology Beyond Literary Criticism. Mediality, Disciplinarity. Berlin: de Gruyter. 1-23.
Ryan, Marie-Laure. (ed.) 2004. Narrative Across Media: The Languages of Storytelling. Lincoln: Univ. of Nebraska Press. 283-304.
Seaton, Douglass. 2005. “Narrative in Music: The Case of Beethoven’s ‘Tempest’ Sonata.” J. C. Meister (ed.). Narratology Beyond Literary Criticism. Mediality, Disciplinarity. Berlin: de Gruyter, 65-82.
Tarasti, Eero. 2004. “Music as Narrative Art. In M-L Ryan (ed). Narrative Across Media: The Languages of Storytelling. Lincoln: Univ. of Nebraska Press. 283-304.
Thompson, Kristin. 2003. Storytelling in Film and Television. Cambridge: Harvard Univ. Press.
Wolf, Werner. 2005. “Pictorial Narrativity.” In D. Herman et al. (ed.) The Routledge Encyclopedia of Narrative Theory. London: Routledge. 431-35.