An Unsupervised Learning Study on International Media Responses Bias to the War in Ukraine

2.1 Topic Modeling and Social Science Studies

One of the most critical goals of data analytics is determining the characteristics that data points share (Vayansky and Kumar 2020), which is what topic modeling comes into play. Thus, topic modeling has been a popular method in the field of education, sociology, linguistics and so forth. It helps researchers to understand topics in large collections of unstructured texts. Basically, the main data source of topic modeling in social science is social media, including Twitter, news articles, etc. Based on social media, researchers are able to investigate international relations (Mishler et al. 2015), health (Sidana et al. 2018), and sentiment (Biktimirov et al. 2021). Meanwhile, on account of the particularity of texts, researchers proposed plenty of revised models, such as Topic Flow Model (Churchill et al. 2018) which can track temporal change of topics, sentiment topic models (Rao et al. 2014) that are capable of digging into the emotion of texts, visual topic model (Zhou, Liang, and Du 2012) that maps visual data to text topics, and Microblog Sentiment Topic Model (Ahuja, Wei, and Carley 2016) which especially suitable for analyzing short-length texts like tweets.

2.2 Studies in the Russia–Ukraine War

The bulk of articles focus on the huge changes that the war brought. Orhan (2022) comments that this war has given rise to a severe humanitarian catastrophe and endangered the stability of international geopolitical connections for the reason that this war causes a large number of people besieged and displaced. Given Russia and Ukraine are major producers of necessities, such as food, minerals and energy, the disruptions of supply chains and exports rose the costs for the global economy (Liadze et al. 2023) which resulted in a global rise in prices including the oil prize and gas prize (Appiah-Otoo 2023; Mbah and Wasum 2022). The disruption of supply chains as well as resource scarcity led to higher costs for obtaining these resources, which were passed on to consumers in the form of higher prices and became inflation influencing stock prizes (Deng et al. 2022). Additionally, some research notice the food security, such as Behnassi and El Haiba (2022) highlight that armed conflicts incurred prohibitively high cost for food causing adverse impacts for nations reliant on food imports and those with low-income populations.

Some scholars have shown interest in the role of social media in this war. Geissler et al. (2022) found robust support for a Russian propaganda campaign that manipulated beliefs and behaviors according to the propagandists’ interests. Aside from the power of social media, fake news and false information were an important focus by researchers, such as the proposal of Russia–Ukraine conflict hate speech datasets (Thapa et al. 2022), automatic fake information detection system (La Gatta et al. 2023), and so forth. Although the proliferation of discussion about the effects of this war on fake news attests to its significance, previous research ignores the potential influences on the online atmosphere. Heed the gap our study intends to bridge is the discussion of differences and potential guidance in social media and the recognition of biases existing within discourses.

3.1 American

The compilation of the American data was done manually using random sampling. The articles from the American corpus are from the three news publishing sites Politico, Washington Post and CNN. These three publishers were chosen because they are big companies, whose articles are read by a large amount of the US population and thus better reflect the media response of the nation. Politico was selected because they have a neutral reporting style and publish longer articles that cover topics concerning politics. Washington Post was selected to the similarly more objective reporting style, which aids in the process of obtaining less biased data and therefore a more representative result. The final news outlet that was chosen is CNN because according to YouGov (YouGov) it is in the top 10 of the most popular news websites in America. Furthermore, the selection of CNN was influenced by the factor of accessibility since the articles are published for free online and thus accessible to anybody that is able to use the internet, whereas another popular news outlet such as the NY Times requires a paid subscription to access their full articles.

The random sampling was done by filtering for a specific date within the predefined date range and by searching for the key words “Ukraine”, “war”, “invasion” and “Russia”. Afterward, for Politico and Washington Post either all articles were compiled in the corpus if there were few results, or if there were many results two to three articles were chosen randomly and added to the corpus. In contrast, the CNN data for the dates between the 16th and the 20th March was compiled by including all available articles in the corpus that were found, which may result in the overrepresentation of certain words and topics that were primarily discussed at that time, thus potentially skewing the output. The data for the CNN part of the corpus is limited to this short time span due to the amount of manual labor and time constraints. Nevertheless, since the Politico and Washington Post data is more balanced, the effect might not be as substantial. Moreover, since the articles from CNN are towards the end of the predefined range, topics discussed at the beginning of the war in Ukraine might be discussed as well. To ensure, the CNN data did not skew the topic modeling output two versions of the corpus were created; one including the CNN data and one without, and topic modeling was performed on both sets. The corpus containing the CNN data was 277,777 words long, and the corpus without contained 180,042 words.

3.2 Chinese

Chinese data are extracted from Renmin (People’s Daily), which is the “microphone” for the Chinese government and the Communist Party of China as well as a pivotal “window” of international communication. Given there are two channels by which Renmin spreads news, namely the Renmin website and Renmin in WeChat Subscription, we collected the Chinese data manually from these two platforms and eliminated the reduplicative data. Eventually, the size of the Chinese dataset is 351,389 characters from 24th February to 20th March 2022. In order to make it easy to be detected by Google Colab, different from America and Swiss data, Chinese data was input into an Excel file where each row is a piece of news. Then, the Excel file was converted into a CSV file.

3.3 Swiss

This corpus contains articles from various news outlets across Switzerland. These articles were newspaper articles as well as website articles. We use the Swissdox database by LiRI,^[1] which contains 260 sources easily accessible to researchers. Since the content of the articles overlapped occasionally due to the fact that they were either published by different newspapers, which publish the same articles but for different cantons, or they were both printed and released online, duplicate values were removed from the corpus together with the corresponding metadata. After the removal of the duplicate values 77 news publication sites and 7516 articles remained in the final version (Figure 1). This amounted to 3,813,007 words in total. The article sources ranking among the top eight in terms of quantity can be seen in Table 1, with the corresponding codes and full names derived from Swissdox.^[2]

Figure 1:

News sources in Swiss German corpus.

To facilitate the analysis of this corpus by both LDA and Mallet the encoding was converted into UTF-8, which is capable of decoding the German characters. Moreover, since these programs experienced issues while reading in the German umlauts, the umlauts were converted into their corresponding two letter version, for instance ö was changed to oe. Furthermore, all links and formatting tags were removed because it distorted the results since these two elements occurred frequently. Finally, the unneeded metadata was removed and the excel file was converted into a text file because it was less difficult for LDA and Mallet to read in.

We acknowledge that the corpus sizes from the media of Switzerland, America, and China are not balanced, with the corpus of Switzerland significantly exceeding the others. Due to the relatively singular nature and higher homogeneity of Chinese news sources, achieving the same level of diversity as Swiss and American media content is challenging. However, because of this singular source, we boldly hypothesize that, once the quantity of news reaches a certain level, the size of news articles does not significantly affect the diversity of topics and from unbalanced data, distinction in topics between different countries can be extracted. In detail, both the Chinese and English datasets have exceeded 270,000 words, providing us with the grounds to believe that the topics we uncover are meaningful. Of course, this is also a direction for our future exploration – whether the quantity of the corpus affects the differences in news topics.

3.4 Stop Word Lists

Stop words do not contain any clear meaning, which makes labeling a topic more challenging. The removal of stop words is essential because without this step the topics will not be clearly defined since these words occur frequently and thus likely will belong to several topics.

4.1 LDA

LDA, namely Latent Dirichlet allocation, proposed by Blei et al. (2003), became a standard technique in topic modeling by ways of automatically discovering topics hidden in documents (Rohani Shayaa, and Babanejaddehaki 2016).

The mechanism of LDA is as follows. The alpha and beta over there are Dirichlet distribution, and the theta and phi are multinominal distributions. Based on the multinominal distributions, we create a bunch of topics and a large number of words (Figure 2).

Figure 2:

Mechanism of LDA model.

Figure 3:

Comparison of terms: war, invasion, conflict and crisis. These three pie charts are from Swiss data, American data, and Chinese data from the left to the right respectively.

On the left of the calculation, we have the probability that a document will appear because this machine could potentially spit out any document we want with tiny probability. On the right, we have four factors, the first two are Dirichlet distribution and the last two are the multinominal distributions, and each one comes with a probability. When we multiply these probabilities, we get the probability of the generated article. When the article is similar to the real document, we derive the topics and words we need.

4.2 Mallet

MALLET, the Machine Learning for Language Toolkit, is a product of the University of Massachusetts Amherst (Barde and Bainwad 2017). Basically, Mallet is overlapped with LDA and their difference can be seen in speed. Specifically, LDA uses the Variational Bayes sampling method which is fast but the precision is low to some extent. Additionally, Mallet supports English alphabet only thus the Chinese data were processed by LDA.

For this paper Mallet was set to 5000 iterations for the American data and 6000 iterations for the Swiss data due to the larger corpus. Furthermore, for the American data 10 topics were set to be determined to be able to better compare the LDA and Mallet results, whereas for the Swiss data 15 topics were set as the output, to minimize the size of the mega topic that was created during several attempt of running the topic model. The optimization interval was set to 1000 and the optimization burn in was set to 100. These settings produced the best results.

4.3 Comparison of Words Concerning War

For the final method of data analysis, the comparison of key words concerning war were decided upon. More precisely, the word “war” (“战争” in Chinese and “Krieg” in German) and the three synonymous words “invasion” (“入侵” in Chinese and “Invasion” in German), “conflict” (“冲突” in Chinese and “Konflikt” in German) and “crisis” (“危机” in Chinese and “Krise” in German) were designated to be evaluated by investigating their proportions to each other. These words were selected because they have different connotations. To elaborate, “war” and “invasion” are more negatively connoted and imply the existence of an aggressor, whereas “conflict” and “crisis” are terms that are used in a more neutral fashion and contrary to the words “war” and “invasion” there is no aggressor.

To generate a diagram demonstrating the proportions of these terms to each other the word count for each word was calculated using grep in the terminal for each respective nation. Subsequently, these results were entered into an Excel table and a pie chart was generated for the results for each country. The method of comparing pie charts and proportions of words to each other was determined to be the most suitable since the corpora were not the same length and this method allows for the processing of each corpus separately.

5.1 American

These two tables (see Appendinces A and B) show that the topics with and without the CNN data are similar and some topics such as topics (Specific) Attacks on Cities and Refugees even contain some of the same words. One major difference is the disappearance of the topic Russia versus Ukraine, which appears to be split into two topics (Russia, Ukraine) in the topic model that contains the CNN data. Moreover, the topics China and Space are one topic in the topic model including the CNN data, whereas the other topic model created a separate topic for China and combined Space with Nuclear Power.

The topics generated fall into different categories that can be labeled as events in Ukraine, refugees, military support, technology, China, misinformation, and internal affairs. The topic of technology can be explained by the fact that there was fear of the use of nuclear weapons due to the fact that both the US and Russia possess nuclear weapons. Moreover, Russia launched an attack near a nuclear power plant and the topic might therefore concern this specific event. China was discussed by the US media because they are considered an ally of Russia and other nations were curious about how they would respond to the attack. The topic of misinformation occurred due to the fact that the Russian media claimed that they were justified in their attack due to various reasons which many other nations deemed to be invalid claims. Furthermore, social media was used to spread Russian propaganda and thus many social media companies occur in this topic. The topic of internal affairs occurs because the two houses and the president had to discuss their response to the war in Ukraine and thus US politics are mentioned in several articles. Many of these categories only contain one topic. However, this is notable because it shows that the American media covered many different facets of the war, both focusing on internal affairs and their response to the war, as well as the events in the Ukraine, and other countries’ responses.

5.2 Chinese

After fine-tunning, we found that, when the topic number is 8, the results were meaningful, which presented the main topics behind unstructured Chinese data. Topic 0 was “attitude of the Chinese Foreign Ministry spokesperson”, where the attitude of the Chinese government is clear. For instance, “正告” (warn) is an official word internationally and the target who was warned by the spokesperson was America and Japan. Topic 1 was “oil and gas”. It makes sense because the price of oil and gas increased as the war goes on. Topic 2 was “negotiation” such as “戈梅利” (Gomel) which is the place where Russia negotiated with Ukraine. Topic 3 was a story in which a fellow from Taiwan was happy because he was able to get on a private plane arranged by Chinese government and go back to China. Topic 4 was “economy”, such as Tokyo Stock Exchange. Topic 5 was “safety of Chinese people”. Chinese government reminded Chinese people of the danger in Ukraine. Topic 6 was “international involvement”. Many countries and organizations hoped to promote the negotiation between Ukraine and Russia. Topic 7 was “evacuation of Chinese citizens”.

Even the results were significant, some problems are revealed. First of all, the tokenization make some mistakes, such as “米科拉·波沃罗兹尼克 (Mykola Povoroznyk)” is an officer in Kyiv, but “尼克” was recognized as a word. Another example is “架接” in Topic 7, which is not a word in Chinese. Besides, some content words have no meaning, such as “之情 (the feeling of)”.

5.3 Swiss

The results generated by the model can be seen in Appendix D. In close inspection, we found that German topics are not very significant. Only four topics are meaningful. Topic 2 was titled as “Swiss response”, Topic 3 was given “weapon”, Topic 5 was “refugee and aid” and Topic 8 was “military”. Generally, the results are not as clear as the ones from the American data set. This might be due to the larger data set that also contained articles that mention other topics apart from the war in Ukraine.

5.4 Comparison of Words Concerning War

The pie charts (Figure 3) display that there is a difference between the nouns China uses to discuss the war and the words Switzerland and the USA use to talk about the situation. There is a tendency for China to use more neutral words. The frequency for the terms conflict and crisis is higher than the frequency of the terms war and invasion. These results are in opposition to the results of the American and Swiss data, where the more negatively connoted words war and invasion are favored over more neutral words to talk about the events happening in Ukraine. The Swiss and the American pie chart are similar, and there are only minor differences, such as the proportion of invasion in the American articles being slightly higher than the one from the Swiss articles.