Determination and agency in the work of Gunther Kress

1 Introduction

In his book Linguistic Processes in Sociocultural Practice (1985), Gunther Kress commented on Saussure’s “unresolved opposition of diachrony and synchrony (…), of language as a socially and historically determined phenomenon and language as an autonomous system” (p. 86). He noted “a constant tension between these contradictory tendencies, sufficient to suggest that these matters were not settled for Saussure” (p. 86). But “what seemed to have been very much a live dialogue for Saussure became to his readers a settled unidimensional text” in which “individual language users meet the system as a monolithic, immutable given which they may use but cannot alter” (p. 86). One reading, Kress said, that of language as a synchronous autonomous system, outside of time and outside of its social historical context, became dominant and Saussure’s awareness of language as always in a process of change was airbrushed out.

Such unresolved contradictions and struggles exist also in Kress’s own work. Characteristically, many of his books and articles end with open questions, doubts about assertions just made, accounts of what had not been achieved. At the end of the first edition of Reading Images (Kress and Van Leeuwen 1996), he wrote:

We have travelled a certain distance along the road, but, as we conclude the book, we realize that we have only just begun, and find ourselves thinking more about the limitations of what we have done and the amount of work that still remains than about what we have achieved so far.

Already in the early 2000s, he began to question aspects of what we had done in Reading Images (Kress and Van Leeuwen 2001: 124):

For us there is also now a slight question about the fundamental issue of whether grammars of distinct modes are quite so uncontentiously ‘there’ as our own efforts in relation to images, for instance, suggest.

Reading Images is indeed a “grammar of a distinct mode”. It was inspired by Halliday’s linguistics and it used the system networks that have become characteristic of his ‘systemic-functional’ grammar. But while, for us, there were still questions, many others accepted Reading Images as a finished analytical framework, ready for use, rather than as a tentative beginning needing testing, and further work.

Later, especially in Kress (2010), he distanced himself altogether from the term ‘grammar’. Instead of grammar, “a fixed and highly constrained regularity”, he preferred ‘resources’ which are “never fixed, let alone rigidly fixed” (Kress 2010: 8) and in his own analyses he rarely followed a strict analytical protocol and rarely used a fixed terminological framework, even the one we had developed ourselves in Reading Images – I will say more later about his ‘exploratory’ way of analysing images and other texts. In working on the third edition of Reading Images, a task which Kress was initially reluctant to undertake, that ‘slight question’ was taken up again, and still not settled. Grammar remained ‘grammar’, between inverted commas, not to be confused with ‘rules for correct usage’ and certainly not universally applicable. Yet Kress still believed that Reading Images was a valid way of describing “major compositional structures which have become established as conventions in the course of Western visual semiotics” (Kress and Van Leeuwen 2021: 1).

Other writings, too, often ended with questions rather than conclusions, for instance his essay “Gains and losses: New forms of texts, knowledge, and learning” (Kress 2005: 21) – what follows is just a selection:

Can I say that depiction is a better means of dealing with much in the world than writing or speech could be? Could we have a better physics if image became more dominant? Would the next generation of children actually be much more attuned to truth through the specificity of depiction rather than the vagueness of words?

In my view it is fundamental that such questions remain questions, that the work of Gunther Kress is taken up as an invitation to keep asking them anew as the world keeps changing. In an early essay (Kress 1989a), he contrasted narrative and argument, seeing narrative as a conservative cultural form which produces stability and reproduces culture, argument as a progressive cultural form “concerned essentially with the effective production of new cultural values and knowledge” (p. 9) and always unresolved, at least in texts – “if there is closure in argument it comes from without the text, in the dialogues that surround it” (p. 9). In this, he stressed, arguments differ from the kind of ‘reports’ which “achieve resolution by the device of assigning the phenomena described to pre-existing theoretical categories, in conformity with a pre-existing schema” (1989b: 11) – a form of discourse which is perhaps all too common in contemporary humanities and social sciences research, including in the field of discourse studies where increasingly many research papers adopt specific analytical frameworks without asking what these frameworks can perhaps not reveal about the data. It is precisely for this reason that Kress began to have second thoughts about the analytical frameworks we had introduced in Reading Images.

There are many such unresolved questions in Kress’s work. In this paper I will discuss two. The first is the relation between the social and the individual, a theme which resonates throughout his work. It might seem, at first sight, that Kress gradually moved from a focus on social determination to a focus on individual agency, but closer analysis shows that he continued to struggle with this question. The second is his move from ‘critique’ to ‘design’ which, in the end, is perhaps not so much a rejection of critique but a subtle account of the relation between convention, critique and design. I will then attempt to explain Kress’s ‘exploratory’ and dialogic approach to text analysis and theory making.

Throughout I will attempt to quote as much as possible from Kress’s work, letting him speak for himself. But I will occasionally refer to the many discussions we have had, mostly between 1987 and 2001 and, more recently, when we were revising Reading Images. Dialogue was an absolutely fundamental part of what I will call Kress’s ‘exploratory’ approach. Knowledge, for him, came about in and through conversation, and he was on the one hand remarkably open to his partners in conversation, while on the other hand always stubbornly following his own course. For this reason, I make no distinction between his single-authored and co-authored work – both had their origin in the kind of creative conversations of which he was such a master. Finally, knowledge, for Kress, also came about by reflecting on sometimes seemingly banal everyday personal experiences, as I will show in detail in Section 4. For this reason, I follow, in this paper, the same approach Kress used in his writings – a dialogic approach in which personal experience is not edited out, allowing readers to see just how Kress’s innovative ideas actually came about.

2 The social and the individual

In the 1980s Gunther Kress worked in an Australian media and communication department. The field was split between American social psychological theories of ‘mass communication’ and ‘interpersonal communication’ in which, as we used to say, ‘meanings are in people’ and British-mediated European semiotic theories in which ‘meanings are in texts’ and based on ‘codes’. At the time, Kress (1985: 3) rejected “psychologically-based theories” which “place the individual (a-social) psyche at the centre” and favoured “an account which places the social at the centre”. “Discourse”, he wrote “colonises the social world imperialistically” and texts “construct the ideal reader” by “providing reader positions”, which in turn create “subject positions” (1985: 36):

If the domination of a particular area by discourse is successful, it provides an integrated and plausible account of that area, which allows no room for thought; the social will have been turned into the natural” (Kress 1985: 10)

In this context education becomes “an institution focused on the reproduction of culture” (1985: 12) where students are instructed and constructed through the resources of language to learn “what kind of social being they would need to be in order to be a member of a community which insists on a denial of the significance of the individual member” (1985: 40).

But there was a counter melody:

A theory which makes no allowance for the social determination of linguistic practice is obviously deficient; at the same time a theory which ignores individual difference in linguistic practice is also deficient (Kress 1985: 12).

Clearly, individual difference was on Kress’s agenda from the start, even at a time when structuralist semiotics and deterministic theories such as those of Althusser still held sway in Australia.

Even though, in this period, he often foregrounded that “the formation of individuals takes place in discourse and genre” (1985: 33), he also acknowledged the possibility of resistance, for instance in a memorable interpretation of graffitists altering the message of a billboard advertising cigarettes (Hodge and Kress 1988: 12): “Meaning is always negotiated in the semiotic process, never simply imposed by an omnipotent author through an absolute code”, and education should therefore “aim at training effective readers who are active in relation to the text, able to resist, able to construct the text to their benefit” (Kress 1985: 40) “In my view”, he said, even then, “the learner is active and agentive, rather than a merely passive recipient. Although the final outcome is socialisation into the rules, values and meanings of the social group, during the learning, the learner is active” (1985: 90).

In the 1990s, Kress began to focus more fully on the individual, on “the making of signs now, in this environment, for this occasion” (Kress 2010: 13) rather than on discourses and genres, and on the way the ‘affordances’ of an object open up possibilities for ‘always new’ meaning making, rather than on the pre-given and hierarchically organized options of Hallidayan ‘meaning potentials’ whose use is socially determined. His crucial example was a drawing by his three-year-old son (Figure 1).

Figure 1:

Drawing by a 3-year-old child (“This is a car”).

The child had just learnt to draw circles, and this had opened up the multiple representational affordances of circles. From these the child selected the affordances of circles to represent wheels, and hence, metonymically, cars, thus choosing an “apt signifier” to represent something which he was, at that moment, interested in drawing, rather than locking into an already existing ‘code’. Generalising from this, Kress concluded that material signifiers “carry a set of affordances from which sign-makers and interpreters select according to their communicative needs and interests at a given moment” (Kress and Van Leeuwen 2006: 232). The result of that process, “like all signs and sign-complexes, is a metaphor, newly made” (Kress 2010: 55).

This new theory was often formulated subjectively, in the first person. Subjects were now no longer seen as formed by ideologically constructed ‘subject positions’, they formed themselves, using discourses as a resource, in ways that were always to some degree new:

Interest directs my attention to something that now engages me at this moment … my interest is shaped by my history, by my experience over time in a set of communities and their cultures, and my interest is (also) shaped by my sense of what is relevant to attend to in my social environment right here and now, in relation to this phenomenon or object. (Kress 2010: 51)

And while the social remained “the source, the origin and generator of meaning”, individual differences, stemming from biographies, now became meaningful

My biography shares much and yet is never identical with the biography of any other member of my or any other social groups. (Kress 2010: 13)

It was in this context that grammar, as “fixed and highly constrained regularity” was replaced by “resources” which are “constantly remade, never wilfully, arbitrarily anarchically, but precisely in line with what I need, in response to some demand, some ‘prompt’ now, whether in conversation, in writing, in silent engagement with some framed aspect of the world, or in inner debate” (Kress 2010: 8). Such resources are developed in communities which “instead of sharing an innate linguistic competence” share social, semiotic, communicational principles and dispositions which “… are articulated … in the ceaseless process of social (inter)action” (2010: 10).

Education should therefore no longer be based on ‘competence’, on “the outcome of processes regulated by power and authority”, but on the active process of design. As he explained in an interview (Andersen et al. 2015: 84):

Competence is always limited by what is socially regarded as competence … whereas design is not limited by a framing. You can make new things and my notion of the sign is that signs are always newly made, and they are used in designs that will always be, in some ways different.

And, again, in the same interview (p. 86):

I remake the world in my making of signs from the world and in doing that I change the resources I have. In changing the resources I have, I change my potential for action in the world. I call that learning, a constant change in my resources, and my capacity for action.

This empowers the individual and the ‘community’. While, in Language and Control (Fowler et al. 1979: 195), Kress had written that “language not only encodes power differences but is also instrumental in reinforcing them, and does so “without any conscious choice on the part of a writer or speaker”, he now restored the possibility of intent and saw power as “redistributed from hierarchical to (at least seemingly) more open, participatory relations”: “No degree of power can act against the socially transformative force of interaction” (Kress 2010: 8). Yet, in that same book, he acknowledged the continuing role of power hierarchies where “the participant with lesser power has no choice – unless he is prepared to risk giving offence” (p. 130) and reaffirmed that “the distribution of power and agencies of control is generative of and crucial for an understanding of communication =(p. 21).

We debated these changes in his work. I initially argued that, while his account of semiosis clearly and plausibly described how young children explore meaning-making, it cannot be a general theory of meaning-making. It would not apply, for instance, to contexts in which individuals have to follow contextually specific rules imposed by powerful institutions and technologies. But in time I came to see that it does not only apply to the way children or artists make meaning, but also to forms of public communication that have become increasingly dominant in our neo-liberal age, forms of communication in which innovation and constant change are fundamental, and in which meaning, like consumer goods, must be customized and individualized, yet also shared by communities, so as to serve the needs of identity design and branding. Such forms of communication, ultimately stemming from the practice of advertising, start with something that needs to be expressed (a ‘brief’), and then choose apt signifiers – colour palettes, fonts, materials, etc. – to express it, in ways that must always seem creative and innovative. Yet they exist to fulfil recognizable social functions within the context of powerful institutions. And Kress (2003: 49) knew this. As he explains:

Design asks, ‘what is needed now, in this one situation, with this configuration of purposes, aims, audience, and with these resources, and given my interests in this situation’. This corresponds in any case to the dominant – that is, mythically leading – social, cultural and economic environment at the moment.

This does not mean that the theory is only valid for contemporary neoliberal forms of meaning-making. There can be no doubt that the form of meaning-making it describes has always been able to be a source of innovation. It only means, as Kress noted in the above quote, that it is at the moment socially, culturally and economically dominant.

In sum, the issue of agency and determination may not have been resolved, but it is of continuing importance in an age where we are both called upon to design our own learnings, lifestyles and identities, and inescapably subject to the designs of big tech, app makers and other forms of power. It is therefore also of continuing importance to keep a clear focus on both these sides of the coin, as, I believe, Kress has done throughout his intellectual career.

3 Critique and design

My second theme is Gunther Kress’s ‘critique of critique’. Kress was one of the originators of critical discourse analysis. An early paper, co-authored with Roger Fowler, introduced the term ‘critical linguistics’ and concluded “The need, then, is for a linguistics which is critical (…) prepared to reflect critically about the underlying causes of the phenomenon it studies, and the nature of the society whose language it is” (Fowler et al. 1979: 186–187). Critical linguistics would eventually develop into critical discourse analysis, which has now become a major sub-field of linguistics, with its own conferences, journals, anthologies, handbooks and so on.

Unlike “the theoretically more diffuse project of critical discourse analysis” (Kress 2010: 73), the original ‘critical linguistics’ project was founded on a Marxist concept of ideology. It saw language as a set of stable conventions, “expressed in the form of laws and rules” (Kress 2005: 16). These conventions, “as a result of power over time” (p. 16), eventually come to be seen as natural, yet form a “theory of reality” which functions to maintain power and involves “systematic distortion in the service of class interest” (Kress and Hodge 1979: 6). In this context, learning becomes a matter of acquiring grammatical competence, competence in complying with the rules of language and therefore also with the theory of reality which has given language its shape. For this reason, critical linguistics focused not just on analysing the discourses that maintain power, but also, and above all, on analysing the system of language itself. It saw linguistics, not as a descriptive account of the system, but as an instrument of discovery, clarification and insight, able to reveal “not a coherent metaphysical, as Whorf assumed, but the opposite: confusions, contradictions, incoherence, traces of the schizoid universe of a class society” (Kress and Hodge 1979: 7).

These confusions resulted from the transformations which the grammar enables:

The typical function of transformations is distortion and mystification, through the characteristic disjunction between surface form and implicit meanings. Since it is usually a help in reversing transformations to know the content independently, transformations can act as a code, fully interpretable only by initiates, safeguarding their privileged knowledge. But transformations can also create the illusion of such knowledge for both speaker and hearer masking contradictions or confusions and imposing an unexamined consensus (Kress and Hodge 1979: 35).

Such transformations famously included agent deletion in passive constructions which can obscure agency and nominalization which can construe processes as stable entities, again with agent deletion, thus representing the actions of power as natural events or as entities for which no-one can be held accountable (Kress and Hodge 1979: 26–27), so “altering the way in which the reader meets the material and structuring his interpretation in specific ways” (Kress and Hodge 1979: 28). Critical linguistics aimed to undo these transformations, so as to “unsettle the naturalization of the social (…) through showing the working of power, whether in representation and communication or elsewhere” (Kress 2005: 17). For education this meant a move away from competence, from mastering the system, to a critical attitude, to fostering ‘critical readers’ (Kress 1985: 42) who “need not comply” and are “active in relation to the text, able to take distance, able to resist, able to construct the text to their benefit” (Kress 1985: 40).

However, in later work, Kress (2010) began to distance himself from the concept of critique in favour of the concept of ‘design’: “Critique is oriented backwards and towards superior power, concerned with the present effects of the past actions of others”, he argued, while “design is prospective, looking forward. Design focuses on my interest now in relation to the likely future effect of my actions” (p. 6). Just as ‘convention’ had to be replaced by ‘critique’, so ‘critique’ now had to be replaced by ‘design’. Critical linguistics, he argued, had aimed to “bring systems and structures into crisis” but “now in the early part of the 21st century, there is no need for bringing the social into crisis: it evidently is” – we evidently live in an “era of radical instability” (Kress 2010: 6):

Things are provisional. For every occasion of communication and interaction, social relations need to be newly assessed (…) and the resources of representation freshly considered in their utility for this instance.

In this context, the stable genres of yesterday, and the stable relationships they imply, have come into crisis. While genres had been “fully determined in all essential characteristics and therefore as outside the scope of effective individual action” (Kress 1989a: 4), they now became resources, able to be used in fluid and flexible ways, “subject to the actions of socially located individual agents” (Kress 1989a: 4). Hence the need for the notion of design which “assumes that resources are never entirely apt but will need to be transformed in relation to all the contingencies of this environment now and the demands made” (Kress 2005: 20). Hence also the emphasis on rhetoric rather than grammar. The work of the ‘rhetor’ is not a competent performance. The rhetor must “make an assessment of all aspects of the communicational situation: of his or her interest; of the characteristics of the audience; the semiotic requirements of the issue at stake and the resources available for making an apt representation; together with establishing the best means for its dissemination” (Kress 2010: 26). This differs from critique: “Where critique unsettled, design shapes, or has the potential to always shape. It makes individual action central”, but he added “though always in a field saturated with the past work of others and the present existence of power” (Kress 2010: 26).

This new direction in his work also included a different take on power. Although rhetoric is still “the politics of communication” and politics “the attempt to shape and regulate social relations by means of power” (Kress 2010: 45), there is now, Kress argued, a redistribution of power in communication, an effect jointly of the social conditions just mentioned and of the way digital devices afford participation, user-created content, accessibility, connectivity, mobility, communities of like-minded people, and so on.

As we revised Reading Images, in 2018, we debated these issues at length. I felt that critiquing the workings of power is always necessary, and has existed, in many different guises, ever since the Biblical prophets admonished the Kings of Israel, and perhaps even earlier. And I felt that new conventions are already emerging, related to new forms of power, for instance in social media and in the standardized templates, formats and themes provided by semiotic resources such as PowerPoint and SmartArt. Such new resources, I argued, need to be critically analysed, as recent critical discourse analysts have done (see e.g. Björkvall and Nyström Höög 2019; Djonov and Van Leeuwen 2018; Kvåle 2016; Poulsen et al. 2018).

Kress knew, of course, that convention, critique and design are interdependent (Kress 2010: 6).

The understanding which inheres in competence was essential to carry out critique, just as the understanding developed through critique is essential in the practice of design. Design draws on both of these, carries their insights forwards and develops them, focused in a social semiotic theory of multimodality.

Innovation always changes what went on before and critique always accompanies change, as can be seen, for instance, in the early 20th century avantgarde manifestos that heralded the move from art to design, e.g. in this call for innovation in fashion (Giacomo Balla 1913, quoted in Apollonio 2009: 132), which at once critiques ‘neutral colours’ and ‘patterns composed of lines, checks and spots’ as ‘depressing’ and ‘gloomy’, and proposed that new designs should use ‘brilliant colours and dynamic lines’ to be ‘daring’ and ‘provide novel enjoyment for our bodies:

We must abolish gloomy and neutral colours, along with patterns composed of lines, checks and spots. (…) Our crowded streets, our theatres and cafés are all imbued with a depressingly funeral tonality, because clothes are made only to reflect the gloomy and dismal moods of today’s passéists (…) We must invent (…) daring clothes with brilliant colours and dynamic lines. They must be made to last for a short time only in order to encourage industrial activity and to provide constant and novel enjoyment for our bodies.

And Kress also knew that the current breakdown of conventions is functional in establishing the new social order of market control, which “fosters fragmentation – lifestyles, the subjectivity of the consumer as opposed to worker or citizen” (Kress 2010: 20). As for power, I have already indicated how he, in his final single-authored book, recognized both newly emerging forms of empowerment, participation and agency and continuing forms of power as an agency of control, the latter of which, he said, should continue to be “generative of and crucial for an understanding of communication” (Kress 2010: 21).

As we worked on the revision of Reading Images we included critiques of the emerging conventions of new forms of visualization. But Kress nevertheless remained adamant about the primacy of design which his analysis of ‘This is a car’ (Figure 1) had introduced in Reading Images (Kress and Van Leeuwen 1996). And I agreed. Critique should lead to, be linked with action, a point also made by the Dutch historian Mak (2004) in relation to the 1968 Paris May revolution. The leaders of this revolution, Mak argued, had a chance to take power, but did not so (Mak 2004: 877, my translation):

They were not really after power. Yes, the power of critique, of being right, but not the power of taking control, of getting your hands dirty. Today I meet the students of 1968 as project leaders, politicians, entrepreneurs. But their leaders, in this respect, felt themselves too good for ordinary work, for ordinary power.

Theorizing the interrelation between critique and design is and will remain crucial. It will require both a critique of design and a re-design of critique. With this project Kress has made an important beginning.

4 Exploratory research

I finally want to discuss Gunther Kress’s approach to text analysis and theory-making. Texts, understood in the widest sense of the term, as artefacts and performances for making meaning, were always the starting point of his thinking, and there was a set of images and texts he used over and over in his various articles and books. Others do this too, but their recurrent images are often diagrams, models through which they seek to visualize the essence of their theoretical approaches. Kress’s texts were different. The examples he used over and over were actual images and texts he had encountered in his everyday personal life as a father, a traveller, a teacher, a researcher – images and texts that had drawn his attention as he walked through galleries, leafed through magazines, read stories to his children. And they included many mundane, everyday texts, such as no smoking signs, a promotional card in a newly purchased leather purse, a handwritten notice with do’s and don’ts for the occupants of a holiday cottage and so on. Such texts had for him a privileged place in developing his ideas: “The banal, the everyday and the unremarkable is always the best site to anchor theory” (Kress 2010: 67) because, he said, “the hurly burly of social life is the generative force which constantly reshapes a society’s semiotic resources and in doing so documents and ratifies new social given” (2010: 35).

Kress did not use these texts as examples of ideas he had formed earlier. They were the very source of his ideas. They embodied his ideas and would, over time, continue to express them better, more immediately, and often more visually, than the various paraphrases that accompanied them. During the years I worked with him, we could never talk without having such examples in front of us. Kress’s approach was exploratory. He generated ideas through conversations, with himself or others, centred, literally, around texts. He was an explorer.

A few concrete examples can perhaps give insight into his way of working, a look in the kitchen, so to speak. Walking through the Tate Gallery in 1995, we stopped at Rodin’s The Kiss, which was prominently displayed in the centre of one of the gallery’s rooms and suddenly realized that in three-dimensional texts of this kind, the visual Given-and-New structure we had introduced in Reading Images, depended on where you stood. From one angle the man was the New, the key focus of attention, from another the woman. While in two-dimensional images Given-New structures are always an interpretation imposed on the represented phenomenon, in three-dimensional texts, the relation is never inherent, never ‘coded’ in the text itself. It always depends on the viewer, unless the positioning of the sculpture makes some points of view less accessible or not accessible at all.

The point is, we were right there, in the Gallery, looking at the sculpture, walking around it and making notes of what we observed and speculated. The notes, with the writing of which we took turns, would slowly build up from week to week until the point we felt we had enough to start writing.

Figure 2 shows a short extract from four pages of notes, in Kress’s own handwriting. They were written after we visited the Victoria and Albert Museum in 1996, where we had somehow decided to linger in a room with bottles – perfume bottles, snuff bottles and so on.

Figure 2:

Work notes about vases.

What is characteristic of these notes, and hence of Kress’s way of thinking? First of all, he always began by describing the signifier in all its physicality and without any pre-constructed sets of parameters or analytical frameworks:

A slightly irregularly formed flat rectangular dish with a rim about 1.5 cm high.

French Sevres porcelain, 1760, pink and bone-coloured (with, I think, gold etchings).

When, much later, he described the Annapelle card (see Figure 3), in Literacy in the New Media Age (Kress 2003: 103), he started in much the same way:

It is a small card, about 5 × 10 cm on quite firm cardboard, olive-eucalypt green (…)

The text is laid out in what I take to be an aesthetically pleasing fashion – it is ‘sculpted’ (…)

Then there is the logo. Is it Renaissance Italy or Australian art deco?

Figure 3:

Annapelle card.

Note that there were also always cultural and historical references – ‘Renaissance Italy’, ‘Australian art deco’. These we would later check, just as art historians do when they try to work out the functions and meanings of objects shown in Renaissance art.

Next, the object would become a source of ideas and of questions, some of them left open, to be returned to later, as in the notes reproduced in Figure 2: “In how many domains is language excluded or inappropriate? Where practices and objects speak for themselves?” In other words, the ‘research questions’ emerged from the texts, rather than that ‘data’ were found after formulating research questions.

One text, or rather set of two texts, has special significance to me because it played a central role in our first work session together, and I recount it here, not to amuse the reader with a personal anecdote, but to demonstrate the way of doing research which Kress taught me. We had decided to write about the image. I had said that I had always wanted to study ‘the language of the image’. He had said that he had long realized that it was no longer viable to analyse media texts without paying attention to the visual – at the time he worked in a media and communication school. Social Semiotics (Hodge and Kress 1988) was already in the making and would contain many visual examples. So the visual it was to be.

Once this was established, he fetched two little books from the bedroom of his young son – Ladybird’s Baby’s First Book and Dick Bruna’s On My Walk, the one explaining images with words, the other entirely visual. In other words, we could not even begin talking without having a text on the table. And the texts he chose were part of his life, books he had read to his young son. But there was another thing. In picking these two books from his son’s room, without any prior planning, he selected, intuitively, and unfailingly, one in which images were closely tied to language, a clear example of what Barthes (1977: 38–41) had called the ‘anchorage’ of visual meaning by language, and another which included no language at all. The double page from Baby’s First Book we focused on in particular was titled ‘My Bath’. The left page contained text (“Every night I have my bath before I go to bed”) and the right page a fairly realistic drawing of a bath. The page from On My Walk we focused on used highly stylized drawings in bold colours – a drawing of a bird in a tree in the centre of the page, and four smaller drawings, one in each of the corners of the page – an aeroplane, another bird in a tree, a cat and a pear (see Kress and Van Leeuwen 2006: 24–25 or 2021: 21 and 22 for reproductions of these images).

So how did that second page make meaning, we asked each other? And why was it no longer a naturalistic image but a composition of rather diverse, and by comparison to ‘My Bath’, rather abstract images? Did the text foreshadow new forms of textuality in which images were sufficient and language was no longer needed? Are images suited to pluriform, multicultural societies? As we talked, we began to realize, back in 1987, that On My Walk was a non-linear text which could be read in different ways in different contexts, yet always on the basis of what was there, on the page. Parents, we surmised, could tell a story about a bird and a cat, and that story could for instance be, as we put it in Reading Images (Kress and Van Leeuwen 2006: 27), “a political story, a story of powerful predators coming from another continent and native birds killed and threatened with extinction”, a story perhaps appropriate in Australia, or “a story that legitimized the survival of the fittest”. In short, we began to see, in On My Walk, the beginnings of a new form of control over meaning, no longer located only in the text itself, but also and importantly in the embedding of the text in its environment, here in the practice of ‘reading a bedtime story, where it becomes, as Kress and I would later write, “less a text than an organized resource for making texts” (Kress and Van Leeuwen 2021: 28).

The Annapelle card (see Figure 3) I already mentioned also figured in many of Kress’s writings, and it, too, was encountered in the course of his everyday personal life. “It came”, as he wrote in Literacy in the New Media Age (2003) “together with two other cards exactly the same, in a leather purse, sent from Australia in 1996, on the occasion of a birthday, sent as a gift and as a memento from ‘home’ to us, living now in England” (p. 103). He then described the card, characteristically not restricting himself to analytical frameworks that might limit what should and what should not be looked at. This created room, as we have seen, for aesthetics, something which we had not touched on in Reading Images – the card “is laid out in what I take to be an aesthetically pleasing fashion. It is ‘sculpted (…) The makers of the card address me as someone who is interested in something beautiful, as someone who has taste” (p. 103) And it also created room for an emphasis on materiality (p. 104):

In the text we need to focus on the discourse of nationalism, of (weakly articulated) racism, of taste and of contemporary economics (‘quality inspection’). But beyond the written text in that narrow sense, we need to focus on quite physical material features of the card. Generically, as card, it relates to business cards and to social relations of that type of card. In its colour it invokes both a certain idea of Italy, and the colour of the Australian bush.

From this he distilled a critique of conceptualizations of genre that link it uniquely to temporal sequence (Kress 2003: 105):

Generic meanings are carried as much in the prepositional usage as in the thickness of the card and its glossiness, as much as in the type of card as in the written text. All these point to social meanings that are realized in genre.

And we have already seen how the drawing of his 3-year-old son (see Figure 1) was crucial in the development of his new approach to sign-making, in which sign-makers have something they want to express, and then choose from the resources they have available what they consider the most apt signifiers for doing so – just as his son, having just learned to draw circles, worked with the affordances of circles to draw a wheel, which he then used to represent a car. Thus Kress saw sign-making as, each time, a creative act, and conventions as merely “placing the pressure of constant limitations of conformity in sign-making” (Kress and Van Leeuwen 2006: 12).

One last example, from Kress’s work with his colleagues at the Institute of Education in London. I will introduce it in his own words:

The teacher had, in the one instance, asked the children to write a ‘story’ of the journey of a red blood cell around the body. In the other case the teacher had asked groups of children, two or three in each case to construct ‘concept maps’, also of blood circulation.

One of these concept maps is reproduced in Figure 4. And (part of) the ‘story’ is reproduced below:

Figure 4:

Concept map of blood circulation.

In Reading Images, Kress and I had departed from the idea that the underlying semantic systems of language and the image overlap, so that, in many cases, image and text can make the same meanings, albeit with different means. Transitivity, we argued for instance, is realized in language by the relations between certain kinds of nominal groups and certain kinds of verbal groups, in images, following Arnheim (1972), by vectors and volumes, but these different signifiers realize the same kinds of meaning. But the blood circulation texts showed that his is not necessarily the case. Different modes have different ‘epistemological commitments’ (Kress 2003: 57). Language can express causality: “If, in the diary, we read ‘I … squeezed my way through’, we know that the agent that did the squeezing is the agent who caused this action” (Kress 2003: 57). However, the visual expresses a different meaning (ibid): “In the case of the concept map, movement suggested by the arrows means ‘point of origin’ and ‘point of destination’ (…) Causality is not, or very weakly implied.”

And, in another, similar example (Kress 2003: 57):

To describe cells linguistically it is necessary to name the relation between cell and nucleus in terms of possession, ‘have’. In a visual representation this is not necessary, but it is necessary to decide exactly where in the cell that nucleus is located.

Such differences have large implications. When we read a linguistic text, we imagine things differently from other people. When we see a film that is adapted from a book we have read, we may be disappointed that the characters are not as we imagined them. As Kress wrote (2003: 59):

Imagination, in the sense that it was produced by engagement with the written text, was a move towards an inner world; imagination in the sense that is required by the demands of design – the imposition of order on the representational world – is a move towards action in the outer world. One was the move towards contemplation; the other is a move towards outward action.

5 Conclusion

This paper has centred on conceptual oppositions – the social versus the individual; critique versus design; exploratory research versus using tight analytical frameworks. Such oppositions are, and will always remain, central in social and semiotic theory. But all too often they are resolved by foregrounding only one side of the equation in our reasoning and reading.

Gunther Kress was an explorer, early to sense changes in the semiotic landscape that have now become part of everyday reality – new forms of writing, new forms of learning, new kinds of social relations. It would be easy to read him as an advocate of the new media with their affordances for participation, connectivity and access to information, and their ability to redistribute certain kinds of power and create communities of interest. This would background or eliminate that they are also new and powerful forms of social control, just as it is easy for critics such as Zuboff (2019) to foreground only power and control, and not the many ways in which the new media can and do support a wide range of grassroots movements and forms of resistance.

Reading Kress’s work attentively shows that it has not been a linear trajectory to ‘where we are now’, a trajectory from asserting the power of institutions, including language, to re-discovering the agency and the creativity of individuals and communities. Throughout his work both continue to be considered, as when, in Linguistic Processes in Social and Cultural Practice, he states that a theory which “makes no allowance for the social determination of linguistic practice” is equally deficient as “a theory which ignores individual difference in linguistic practice” (Kress 1985: 12). And throughout his work, strong assertions are often followed by qualifications, as when he argues that design makes individual action central, but then adds “though always saturated with the past of others and the present existence of power” (Kress 2005: 20).

It is important to keep his work as open as he would have liked it to be, to see it as a set of challenges for social semiotics and multimodality that will continue to be relevant, and as a theory in need of constant transformation and re-making as it is put to work in different contexts, where it becomes itself an “apt” source of “design”, for instance in educational practices of different kinds. So long as we do not use Kress’s name as a stamp of approval for making one of these conceptual oppositions into a universal truth at the expense of the other, as Kress argued has been done with the work of Saussure, the conversation can and should continue. It is Kress’s spirit of exploration, of dialogue, of open-ended argumentation and of personal involvement that I hope will be his lasting legacy and a continuing inspiration for us all.