Using ATLAS.ti for constructing and analysing multimodal social media corpora

2.1 Multimodal corpus studies of social media discourse

Social media posts are inherently multimodal, combining semiotic resources such as emojis, memes, videos, and external links. Their diverse modes, shaped by platform-specific properties, have increasingly become the focus of many multimodal discourse studies in recent years (Bouvier and Machin 2020; Djonov and Zhao 2013). For example, Han and Zappavigna (2024) examined TikTok videos, conducting a qualitative analysis of rhythm across speech, bodily action, gesture, and music. Andersson (2024) employed a qualitative, social semiotic discourse analysis of YouTube reaction videos on transgender activism, demonstrating how the integration of different modes can express impoliteness towards non-targeted viewers. Both studies focus on video-based platforms; however, for social networking platforms such as Twitter and Instagram, most social media discourse studies emphasize images and written language in posts (Jovanovic and van Leeuwen 2018).

While research on social media discourse has grown, it remains largely divided into two methodological camps: qualitative (e.g., Chałupnik and Brookes 2022; Hansson and Page 2023; Jovanovic and van Leeuwen 2018; Ng 2018) and quantitative (e.g., Christiansen et al. 2020; Collins 2020) approaches, each with inherent limitations.

Quantitative or statistical methods, particularly those using computational tools like Google Cloud Vision (e.g., Christiansen et al. 2020), enhance efficiency when handling large datasets. However, these automated tools sometimes struggle to capture the nuance and granularity of visual expressions (Baker and Collins 2023). Also, though valuable for identifying large-scale patterns, purely quantitative approaches often overlook the intricate semantic and functional relationships across modes.

In contrast, qualitative discourse studies with human interpretation (e.g., Chałupnik and Brookes 2022; Ng 2018), as Christiansen et al. (2020) assert, remain crucial for understanding the complexities of multimodal social media content, especially for uncovering discourses not explicitly represented in language. However, purely qualitative multimodal discourse studies also present challenges. For instance, Widdowson (2000) highlights the potential for cherry-picking, where qualitative analysts selectively focus on atypical data to support preconceived arguments. Additionally, the manual analysis process often struggles to reveal cross-modal relationships across posts, limiting researchers’ ability to objectively identify prominent patterns.

Consequently, there is a clear need for research which combines the benefits of quantitative and qualitative methods for multimodal discourse analysis on social media. Quantitative methods can enhance pattern recognition and mitigate the risks of selective interpretation (Baker and Levon 2015). Such methods can reveal patterns in language use, especially co-occurrences, that may not be immediately evident to human intuition (Widdowson 2000). Meanwhile, qualitative methods allow exploration of the nuanced relationships across modes and facilitate interpretation of patterns within broader social, political, and cultural contexts in a way that is not possible using quantitative methods (Baker et al. 2008). Together, these approaches can offer a more comprehensive framework for understanding multimodal communication on social media platforms.

Efforts within corpus linguistics to study multimodal social media discourse are relatively new, and, to our knowledge, no corpus software has the capability yet to fully interpret multimodal data in the same way it handles linguistic monomodal data, for example by identifying co-occurring patterns and demonstrating associated concordances across different modes. Researchers have attempted to bridge this gap by adapting existing corpus tools, though these adaptations are mostly applicable to specific target semiotic resources, such as emojis, which are readable in some already existing linguistic corpus software (e.g., Sketch Engine). Some studies have encoded emojis with Unicode values to make them more machine-readable (e.g., Collins 2020; Zappavigna and Logi 2021); for instance, Collins (2020) focuses on emojis and images by converting emojis in Facebook posts to their Unicode values and categorizing images into predefined groups, enabling searches comparable to linguistic queries. Other approaches attempting to take account of multimodality, such as Bednarek and Caple’s (2017) research, combine results from separate corpus analysis of linguistic data and qualitative analysis of sampled visual data. The brevity of much social media data presents an additional challenge for traditional corpus software, since fragmented meaning-making units and contextual interpretations (Clarke 2022) necessitate a different approach than would be taken with traditional media discourses. This highlights the ongoing need for more advanced, integrated analysis tools and approaches capable of addressing the intricacies of multimodal social media data.

2.2 Introducing ATLAS.ti for multimodal corpus studies of social media discourse

To address the existing gaps in multimodal corpus studies of social media discourse, this paper introduces the application of ATLAS.ti (version 24.2.0), an interpretive piece of software originally developed for computer-assisted qualitative data analysis (CAQDAS) in 1993. Aligned with grounded theory principles, CAQDAS tools, such as ATLAS.ti, allow the iterative development of data categories (Page 2022), enabling more transparent, systematic, and replicable analytical processes (Woods et al. 2016).

ATLAS.ti (version 24.2.0) is available in both desktop (offline) and web (online) versions, with a feature comparison provided on the official website (ATLAS.ti 2024b). Researchers can access a free trial of the software, though continued use typically requires a paid licence. The software includes an inter-coder agreement mode to support collaboration among multiple users. Additionally, a technical document detailing ATLAS.ti’s concepts and functionalities is available to assist researchers with limited prior knowledge (ATLAS.ti 2024a).

ATLAS.ti has a strong track record of application in linguistic monomodal discourse and conversation studies (e.g., Friese et al. 2018; Karera et al. 2024; O’Halloran 2011; Potgieter 2014). Its reach also extends beyond linguistics to fields such as business strategy analysis; for example, Karera et al. (2024) used the software to conduct content analysis of semi-structured interviews with medical officers, and Potgieter (2014) applied it to summarize customer interaction strategies on social media in classroom settings. One of ATLAS.ti’s key advantages is its ability to incorporate multimodal datasets (Antoniadou 2017), which has led to its use in a limited number of qualitative multimodal discourse studies. For instance, Rossolatos (2014) employed ATLAS.ti (version 7) to analyse the multimodal rhetorical structure of TV advertisements. The researcher identified their minimal units of analysis (i.e., “ad filmic segments”) by focusing on five groups of pro-filmic elements, such as actors, setting changes, and slogans, which were then coded for rhetorical figures. However, given the comparatively limited capacity of ATLAS.ti at that time, the summary replied primarily on descriptive statistics (e.g., frequency), and the segmentation process presented some challenges, particularly regarding the overlap of some coded elements – for example, setting changes and slogans – within the timeline.

In contrast, ATLAS.ti (version 24.2.0) is particularly well suited for analysis of data from text-and-image-based social media platforms. The short-form nature of posts, often limited by character constraints, allows researchers to treat each post as a discrete analytical unit. This enables ATLAS.ti’s functionalities to be applied in constructing and analysing multimodal social media texts, much like in monomodal corpus studies. Researchers can use these functionalities to identify both intra- and inter-mode co-occurrences, which can then be systematically traced, visualized, and analysed both quantitatively and qualitatively (as detailed in Sections 3 and 4).

Despite its potential, the use of ATLAS.ti for short-form multimodal social media discourse studies is still in its infancy. Most studies have focused on monomodal data, such as the linguistic analysis of blogs (Paulus and Lester 2014), or traditional content analysis methods like theme identification in tweets about parks during COVID-19 (Choe et al. 2022). This highlights the need for a more comprehensive, multimodal analytical framework for social media discourse analysis with ATLAS.ti.

ATLAS.ti (version 24.2.0) has introduced new features such as concept cloud generation, opinion mining, sentiment analysis, and AI-based summaries. While these tools offer valuable insights, especially for large-scale pattern recognition, our pilot study revealed that they may not yet be robust enough to fully replace human interpretation or advanced natural language processing methods. For this reason, the present study does not rely on these automatic features but instead emphasizes the interpretive potential of ATLAS.ti in combination with manual analysis.

This paper aims to demonstrate how ATLAS.ti can be applied to construct and analyse multimodal corpora of social media discourse. The case study focuses on the multimodal representation of #freerange on Twitter over a six-month period (Malory and Sha 2025). The topic has gained prominence amid growing public concern in the UK about animal welfare and ethical consumption (RSPCA 2022), particularly in debates surrounding the ethics and transparency of free-range farming practices (Lekakis 2022).

The case study addresses three research questions.

1. How is #freerange usually represented – as food products or as animals?

2. How are associated social actors and free-range methods appraised?

3. How does appraisal differ across tweeter identities?

We will outline the data collection process and provide a detailed description of the corpus construction in the methodology section. Following this, we will introduce four functionalities for conducting corpus-assisted multimodal discourse analysis (CAMDA) using ATLAS.ti. This paper focuses on demonstrating the application of ATLAS.ti’s functionalities for constructing and analysing multimodal corpora, highlighting their methodological value. For practical details of the software, such as operational steps and interface navigation, readers are encouraged to refer to the comprehensive manual available on the official ATLAS.ti website (ATLAS.ti 2024b).

3.1 Data collection

The multimodal tweet dataset for (Malory and Sha 2025) was collected using Twitter’s Advanced Search function. The focus was on English-language tweets containing the hashtag #freerange within the period from 1 January 2022 to 1 June 2022. Data collection took place in July 2022, with January–June 2022 chosen as the period of focus due to an uptick in public debate about the ethics of free-range farming during the ongoing outbreak of avian flu affecting the UK (Race 2022). Increased discussion of free-range agriculture during this period, including on Twitter, provided valuable multimodal data for trialling the methodological protocol with ATLAS.ti outlined in this paper. Using a Google Chrome extension called Twitter Media Downloader, we extracted both the tweet text and associated multimodal elements. During this process, the text, along with metadata such as usernames and the number of likes, was automatically saved in an Excel file. Other semiotic resources, including images and videos, were automatically downloaded in bulk and saved to a local folder. Following a manual review to filter out tweets irrelevant to the research focus, 1,003 tweets were retained.

As we were interested in comparisons across different tweeter identities, we further refined the dataset by retaining only tweets that could be categorized as commercial (primarily promoting products for financial gain), institutional (focused on animal rights advocacy), or personal accounts (lacking any discernible commercial or ethical affiliations). These classifications were inferred from the users’ profile details and the content of their posts. However, it is important to acknowledge the limitations of this categorization method, especially in the case of personal accounts where profiles may be fictitious or misleading. As a result, additional exclusions of ambiguous cases were made, ultimately resulting in a final corpus of 899 tweets.

3.2 Constructing a multimodal corpus on ATLAS.ti

The initial step in constructing the corpus was uploading and setting up the target multimodal tweets. Importing the Twitter data as “surveys” allows researchers to select and retain metadata for analysis, such as Username, Tweet Content, and URLs. Once uploaded, each tweet is treated as an individual document within the ATLAS.ti project.

In the examined dataset, 94.22 % of the tweets are multimodal, incorporating semiotic resources such as text, emojis, emoticons, memes, videos, external links, and retweets. This high proportion highlights the importance of retaining multimodal features for social media studies.

Of these semiotic resources, text and emojis are directly displayed in the tweet cases. For other semiotic resources like memes, photos, and external links, the network function in ATLAS.ti allows for the linking of these resources within the same tweet. This enables the cohesive presentation of all semiotic resources on a single page, as demonstrated in Figure 1. To facilitate this, these external semiotic resources need to be uploaded into the corpus using the Add Documents function and then linked to the corresponding tweet content (see Figure 1 for the multimodal page view for a tweet case).

Figure 1:

Page view for a tweet case.

Videos present a unique challenge compared to other semiotic resources, primarily due to the labour-intensity associated with their analysis. For this study, we adopted a strategy of capturing screenshots from the initial meaning-making unit of each video. This is the frame most platform users see before interacting with the video, allowing us to analyse the videos as static images. Additionally, we carefully watched each video in full to ensure that no potential meaning conflicts, such as instances of irony, were missed.

3.3 Annotating multimodal corpora on ATLAS.ti

Since analysis and interpretation in ATLAS.ti rely on the “codes” (i.e., annotations) assigned to the data, the second step involves annotating the target corpus. Researchers have two options for this process. The first is the semi-automated “intentional AI coding”, which allows users to assign coding tasks based on their research questions. However, for CAMDA, this method is currently only suitable for simple tasks such as detecting specific words or summarizing general themes.

The second option is manual coding, which we employed in the present study. Manual coding provides researchers with greater flexibility to develop a coding scheme that aligns more with their specific research questions, rather than being restricted by the limitations of the software’s analytical capabilities. The annotation process was conducted by the first author, with the second author reviewing the codes to ensure plausibility.

To address the research question “How does appraisal differ across tweeter identities?”, we categorized user identities based on the criteria outlined in Section 3.1, classifying them into commercial, institutional, and personal accounts.

For the research question “How are associated social actors and free-range methods appraised?”, we coded the tweets according to the appraisal strategies employed, with a particular focus on the Attitude dimension of Martin and White’s (2005) Appraisal Framework (see Figure 2).

Figure 2:

An overview of the attitude dimension in appraisal framework (Martin and White 2005).

The Attitude coding involves not only the appraisal categories following a threefold structure, such as [Appreciation-quality-positive] for example (1), but also identifying the ideation (Inwood and Zappavigna 2024) or subjects of these attitudes (e.g [free-range food] for (1)). This allowed us to also address the first research question, which is to determine whether #freerange is usually represented as food products or as animals in the tweets.

(1)

Today’s Breakfast Special: #freerange mouthwatering chicken served with pan-fried soup buns

Order online at [anonymized]

#freerange #breakfastspecial #dimsumlover

Additionally, we focused on the cross-modal synergy between appraisal strategies and social actor representations. Using a data-driven approach, we coded the visual representations of social actors into five categories [free-range-few], [free-range-flock], [human commercial], [human consumer], and [human raiser]. This addresses the visual mode’s prominent influence on audience perception, as evidenced by eye-tracking experiments (Garcia and Stark 1991; Holmberg 2004). Specifically, such studies demonstrate that visual elements not only capture initial attention but also sustain the viewer’s gaze for longer periods.

4.1 Frequency analysis

In terms of frequency analysis, ATLAS.ti generates automatic statistics following the coding process, which can be accessed via the Quotation Manager.

To illustrate its utility, we summarize the results on the visual representations of #freerange social actors in Figure 3, including free-range food (PA-Food), a few free-range animals (PA-FR few), and flocks of free-range animals (PA-FR flock). A comparison of tweeter identities reveals that #freerange is predominantly depicted as food across all three user groups. Furthermore, Figure 3 shows that commercial accounts are more inclined to use visual representations of free-range food in their posts, compared to institutional and individual accounts.

Figure 3:

Frequency of visual representations of #freerange (by user identity).

4.2 Code co-occurrence analysis and associated multimodal concordances

4.2.1 Code co-occurrences and extended concordances

Code co-occurrence analysis enables the examination of co-occurring patterns of researcher-designed codes, such as themes, social actor representation strategies, and appraisal strategies in the present study, both within and across modes.

In addition to raw numbers of co-occurrences, ATLAS.ti also provides the co-occurrence coefficient, which is represented by the formula c = n12 ÷ (n1 + n2 − n12), where n12 is the number of times two codes co-occur, and n1 and n2 are the occurrences of each individual code. The coefficient c indicates varying degrees of co-occurrence between the two codes. A value closer to 1 suggests a stronger association, with the codes frequently appearing together, while a value closer to 0 indicates a weaker relationship.

One illustration of code co-occurrence analysis can be found in Figure 4, which shows the strengths of co-occurring patterns between visual representations of #freerange social actors and the attitudes expressed in the corresponding tweets.

Figure 4:

Coefficients between visual representations of #freerange social actors and appraised attitudes.

The disparities among the appraisal of the three #freerange social actor categories are evident in the results shown in Figure 4. Firstly, tweets featuring visual representations of free-range food or products predominantly align with positive attitudes. The strongest association is observed in the co-occurrence coefficient of 0.72 between visual representations of free-range food or products and the appraisal of positive quality ([Appreciation-quality-positive]).

The differences become more intricate when considering the visual representations of free-range animals. Here, a critical distinction emerges between tweets depicting a few individual animals ([Free-range animals (few)] in Figure 4) versus those showcasing a flock ([Free-range animals (flock)]). Tweets representing a small number of animals generally convey positive sentiments, as indicated by a co-occurrence coefficient of 0.42 with the positive appraisal of balance and complexity (i.e [Appreciation-balance & complexity-positive]). In contrast, tweets portraying a flock of animals tend to express more negative appraisal, demonstrated by a notable co-occurrence coefficient of 0.30 with the [Appreciation-balance & complexity-negative] code.

Nevertheless, an inherent limitation of the coefficient and similar metrics is their susceptibility to distortion from disparate code frequencies. To guide researchers in identifying significant relationships that might otherwise be overlooked due to imbalanced application frequencies of codes, ATLAS.ti incorporates a visual cue: a yellow dot in the top right corner of the table cell. This marker appears when one code in a pair has been applied more than five times as frequently as its counterpart. This indicator prompts a closer examination of the co-occurrences within that cell (for details, see ATLAS.ti 2024a).

Once (non-)salient co-occurrence patterns have been identified, researchers can click on the corresponding co-occurrence columns, where ATLAS.ti displays the relevant “quotations” (i.e., concordances) of these co-occurrences on the right-hand side (see Figure 5). This structured page view largely resembles the concordance functionality in monomodal linguistic corpus software. It also enables researchers to closely examine the multimodal contexts of target patterns, as well as the associated contextual factors of each concordance, such as engagement metrics and user account details (see Figure 1). The short-form nature of tweets, due to the character limit, makes them suited for multimodal concordance analysis, with each tweet functioning as its own concordance.

Figure 5:

Code co-occurrence analysis and corresponding concordances.

4.2.2 Visualization

One of the strengths of ATLAS.ti in CAMDA is its visualization capabilities. One such visualization tool is the force-directed graph, which enables graphic representation of inter-code relationships. This method employs three elements to convey information: size of node, indicating the frequency of a code within the corpus, and length and thickness of the connecting lines, where shorter, thicker lines indicate a higher coefficient and therefore stronger co-occurrence between codes. The corresponding multimodal concordances for these co-occurrences are also traceable, with a page view similar to that shown in Figure 5.

This visualization reveals the web of co-occurring relationships, uncovering patterns that might otherwise remain hidden. We now employ Figure 6, which shows the co-occurrence patterns across the appraised subjects in the corpus, to demonstrate the utility of force-directed graphs. The prominence of the “Food/product quality” node in Figure 6 highlights the evaluations of free-range food and products being the most frequent within the corpus. In contrast, the smallest “Human commercial lifestyle” node suggests its infrequent appearance in the evaluations. As previously outlined, the length and thickness of the lines in the figure illustrates the strength of co-occurrences between nodes. The shortest and thickest line, linking “Food/product quality” and “Human consumer lifestyle”, indicates that these subjects are frequently evaluated together in the tweets. There are also notable connections between “Animal lifestyle” and “Food/product quality”, as well as between “Animal lifestyle” and “Human consumer lifestyle” in Figure 6. However, some nodes remain disconnected, indicating a lack of co-occurrence among them within the dataset. For instance, the absence of a line between “Policy & implementation” and “Human raiser lifestyle” indicates that the associated evaluations never co-occur in the analysed corpus.

Figure 6:

Co-occurring patterns across appraised subjects.

4.3 The query tool

The third functionality is the query tool, which can be used to filter concordances based on specific code (non-)co-occurrences. This tool is especially useful for further investigating particular (multimodal) discursive patterns within the corpus, or when unexpected patterns emerge that extend beyond the initial research questions, as is common in corpus studies (Gillings et al. 2023). Similar to the code co-occurrence analysis page view (see Figure 5), it also directs researchers to the corresponding concordances in detail and enables in-depth analyses of the patterns.

For instance, the code co-occurrence analysis summarized in Figure 6 reveals frequent pairwise co-occurring patterns across three subjects: animal lifestyle, food/product quality, and human consumer lifestyle. This prompted us to further investigate cases where these appraised subjects co-occur, especially in tweets from commercial accounts, and to explore whether causative links or sequential occurrences are represented.

We set the query tool and qualitatively analysed the 15 cases that met the criteria, and found that 66.67 % suggested a causal relationship; wherein a balanced and harmonious animal lifestyle is portrayed as directly contributing to high-quality food, which, in turn, is implied to support an environment- and health-conscious consumer lifestyle. Example 2 is a self-created instance to demonstrate this pattern (while ensuring compliance with copyright considerations). The example opens by emphasizing the well-being of the sheep, personifying them as “fluffy friends” and ascribing human emotions, “Happy sheep”, in the text. The portrayal of the animals’ lifestyle, via descriptions like “wandering through their snowy paradise”, is positive, whilst the accompanying non-linguistic semiotic resource, a photo which shows sheep walking in a snow-covered landscape, further implies ease and leisure. The focus then shifts to food quality, with the emojis and the hashtag #farmfresh implying a direct link between the sheep’s welfare and production of meat and cheese that is superior. Finally, the hashtags #sustainablefood and #lowfoodmiles tie this narrative to sustainable practices, highlighting reduced environmental impact and connecting animal welfare to the environmentally conscious and health-oriented lives of consumers.

(2)

Happy sheep wandering through their snowy paradise #farmfresh #happysheep #freerange #sustainablefood #lowfoodmiles