Abstract
Software both powers and constrains our ability to engage in an ever widening range of meaning-making practices across diverse social domains. In contrast to earlier semiotic technologies (e.g. a pencil or a typewriter), software enables users to select from a range of different semiotic resources, and incorporates and represents knowledge about what constitutes effective communication with these resources in particular contexts. This knowledge reveals and serves the interests of certain social groups (their communicative goals, norms, values and aesthetics), even though the dominance of many software products extends well beyond the institutions and practices for which they were originally designed (e.g. PowerPoint was originally designed as a tool for corporate marketing presentations). This is because, although their redesigns and updates steadily increase the arsenal of semiotic resources and choices they offer, software products rarely cater to expansions in their use (e.g. the use of PowerPoint for research presentations or teaching). This creates tensions in the interaction between a software’s design and its use, and can lead users to either conform to the communicative norms built into a given software or rebel against them, using the semiotic resources provided by the software creatively, against the grain of its design. Analysing software as a semiotic technology – and studying its design, use and their interplay – is thus a promising new research direction for multimodal semiotics and critical discourse analysis. It can help us understand the role of software in testing the limits and enriching the meaning potential of semiotic resources and their interaction, as well as in engendering, maintaining and subverting social boundaries and inequality.
In this paper, we introduce a social semiotic framework for analysing software as semiotic technology that we have developed using PowerPoint as a key example (Djonov & Van Leeuwen, 2012; Van Leeuwen & Djonov, 2013; Zhao, Djonov, & Van Leeuwen, 2014). Specifically, we examine the challenges and benefits associated with directions that have been taken in developing and applying the framework, and emphasize the value of comparative and historical perspectives for studies of semiotic software. These include:
- starting with a specific semiotic resource (e.g. colour, texture, layout, smart art), by first mapping the meaning potential of that resource as instantiated in a range of semiotic artefacts and interactions, including ones that are enabled through a given software tool as well as ones that are not, and then relating this potential to the way this resource is represented in the software’s interface (Djonov & Van Leeuwen, 2011, 2013) or vice versa (Kvåle, in press)
- comparing the same resources in the interface and use of different software products (e.g. animation in PowerPoint vs. Adobe After Effects; Djonov & Van Leeuwen, in progress)
- focusing on specific semiotic practices that employ (and reflect on) a software tool (e.g. online shopping or software art) (Van Leeuwen, Djonov, & O’Halloran, 2013), and on how they differ from comparable non-digital practices
- comparing the use of a software tool in different semiotic practices (e.g. PowerPoint in corporate presentations vs. university lectures), or in the representation of knowledge in different disciplinessuch as cultural studies vs. science (Zhao & Van Leeuwen, 2014)
- taking a historical perspective and examining shifts in a software’s design, its use and related discourses about it (Djonov & Van Leeuwen, 2011, 2013; Kvåle, in press).
In the final section, we will examine how the artificial intelligence built into social media technologies distributes or off-loads semiotic knowledge and work (e.g. through layout templates) and shapes how people share information and negotiate their identities, in the light of recent developments in ‘distributed cognition’ theory (cf. Cowley & Vallée-Tourangeau, 2013). This discussion employs ResearchGate as a case study of the ways “the platformization of the web” (Helmond, 2015) powered by social media pre- programs the construal of academic identity and networks, benchmarking both against the global corporate culture principles that underpin the information age.
References
Cowley, S. J., & Vallée-Tourangeau, F. (Eds.). (2013). Cognition Beyond the Brain: Computation, Interactivity and Human Artifice. London: Springer.
Djonov, E., & Van Leeuwen, T. (2011). The semiotics of texture: from tactile to visual. Visual Communication, 10(4), 541-564. doi: 10.1177/1470357211415786
Djonov, E., & Van Leeuwen, T. (2012). Normativity and software: A multimodal social semiotic approach. In S. Norris (Ed.), Multimodality and Practice: Investigating Theory-in-Practice-through-Method (pp. 119-137). New York: Routledge.
Djonov, E., & Van Leeuwen, T. (2013). Between the grid and composition: Layout in PowerPoint’s design and use. Semiotica, 2013(197), 1-34. doi: 10.1515/sem-2013-0078
Helmond, A. (2015). The Platformization of the Web: Making Web Data Platform Ready. Social Media + Society, 1(2). doi: 10.1177/2056305115603080
Kvåle, G. (in press). Software as Ideology. A Multimodal Critical Discourse Analysis of Microsoft Word and SmartArt. Journal of Language and Politics, (Special Issue on Critical Multimodal Discourse Analysis).
Van Leeuwen, T., & Djonov, E. (2013). Multimodality and software. In C. Chapelle (Ed.), Encyclopedia of Applied Linguistics. Oxford: Wiley-Blackwell (Area Editor: Sigrid Norris).
Van Leeuwen, T., Djonov, E., & O’Halloran, K. L. (2013). “David Byrne really does love PowerPoint”: art as research on semiotics and semiotic technology. Social Semiotics, 23(3), 409-423. doi: 10.1080/10350330.2012.738998
Zhao, S., Djonov, E., & Van Leeuwen, T. (2014). Semiotic technology and practice: a multimodal social semiotic approach to PowerPoint. Text & Talk, 34(3), 349-375. doi: 10.1515/text-2014-0005
Zhao, S., & Van Leeuwen, T. (2014). Understanding semiotic technology in university classrooms: a social semiotic approach to PowerPoint-assisted cultural studies lectures. Classroom Discourse, 5(1), 71- 90. doi: 10.1080/19463014.2013.859848