The Information Infrastructure of Search Engines: A Content Analysis of Covid-19 Immunization Information Online

After the World Health Organization declared the spread of the novel coronavirus (COVID-19) a global pandemic in March 2020, they cautioned of another outbreak: an “infodemic.” This study examines how online search engines are influencing the global spread of immunization information about COVID-19. It aims to address the various ways in which search technology is shaping users’ perceptions of the pandemic and to measure the credibility of the sources they provide.

iJournal, Vol 6, No. 2, After the World Health Organization (WHO) declared the spread of the novel coronavirus  a global pandemic in March 2020, they cautioned of another outbreak: an "infodemic." Likening the proliferation of misinformation and disinformation to a virus, they warned that it was "highly contagious and grows exponentially" (World Health Organization, 2020). While the WHO was worried about "the trolls and conspiracy theorists" whose misleading news served to "undermine the outbreak response" and erode public trust in government officials, they were also particularly concerned about misinformation on immunization and its spread on social media platforms (World Health Organization, 2020). As early as February 2020, WHO officials arranged to meet at Facebook headquarters to tackle this infodemic head on and have since partnered with a host of other companies, including Google, TikTok, YouTube, and Tinder (World Health Organization, 2020).
Given that people increasingly seek health information online, and that most of them begin their query on a search engine (Kata, 2010;Allam et al., 2014;Abenhaim & Baazeem, 2014), this intervention is crucial. While search engines like Google, Yahoo, and Bing are typically seen as facilitators in the search for information, as several scholars have pointed out, their role is actually one of mediation and interference (Kaplan, 2014;Mager, 2012;Noble, 2013). Their algorithms are steeped in capitalist ideology, and yet most users are unaware of the fact that (or at least the extent to which) search engines are tailoring their online experience. This is due in large part to the invisibility of the information infrastructure of search technology. Most users mistake search engine ranking for a quality ranking and extend their trust for the search engine to the sources they present (Allam et al., 2014;Lewandowski, 2017). Furthermore, since users typically only focus on the first few results provided by the search engine, "this gives the sorting/ranking algorithms of the search engines tremendous influence over the global spread of information and other communication processes" (Allam et al., 2014).
How are search engines influencing the global spread of information on COVID-19? This paper reports the findings of a study conducted across five search engines. It aims to address the various ways in which search technology is shaping users' perceptions of the pandemic and to measure the credibility of the sources they provide.

Literature Review
Several studies have examined the quality of health information returned by search engines. Kata (2010) conducted a search of Google to determine how many of its results were pro-or anti-vaccination. She found that language had a significant impact on the quality of search iJournal, Vol 6, No. 2, results, with 71% of results for "vaccination" being classified as anti-vaccination, whereas 100% of results for "immunization" were pro-vaccination. Madden, Nan, Briones, & Waks (2012) analyzed the tone and nature of sources that people seeking information about the HPV vaccine are exposed to on different search engines. Though they found that most results were either positive or neutral in tone, vaccine-negative sources were also present, and they cautioned health professionals to consider their impact. Allam, Schulz, and Nakamoto (2014) conducted two online experiments with manipulated search engines to determine whether manually sorting and ranking criteria (to lean either more positively or negatively) had an impact on user's beliefs surrounding vaccination. They determined that users are largely unaware of how the quality of their sources affect their opinions. Using different search engines, Ghezzi et al. (2020) searched for "vaccines autism" in four different languages. They found that both alternative and commercial search engines returned anti-vaccination webpages, except for Google, which returned none. They also found that "privacy-preserving engines," such as DuckDuckGo, "give more visibility to webpages promoting vaccine hesitancy or with a clear anti-vaccine position than Google" (p. 4). Brophy & Bawden (2005), for their part, looked at the general quality of search engine results in comparison with academic library resources and found that while library databases "are superior for quality of results," that Google is "clearly superior in both coverage and accessibility" (p. 510).
This paper builds on these previous studies by extending the analysis of search results to the advertisements also present on the search page. Furthermore, no study, as of yet, has looked at information about immunization for COVID-19.

Research Methods
Web searches were conducted on December 2, 2020 using the search term "COVID-19 vaccine" input into Google, Yahoo, Bing, DuckDuckGo, and Ecosia. The focus on immunization was chosen in part due to vaccine misinformation that proliferates the web, but also because news about COVID-19 had turned towards the development and distribution of vaccines in recent weeks. Furthermore, Google saw an exponential rise of searches for the term "vaccine" since the start of the COVID-19 pandemic (see Figure 1). This seems, therefore, like an ideal time to ensure that the sources that result from those searches are reliable.
The searches were completed using a Firefox browser cleared of all cookies and search history to avoid personalization of results. Research shows that online health information seekers only look at the first ten search results (Kata, 2010;Madden et al., 2012). For this reason, the study was limited only to the results that appeared on the first page. These ranged from seven to iJournal, Vol 6, No. 2, ten results.
The URL results from each search engine, excluding those listed as "Top Stories" or "News," were transferred to a spreadsheet. The websites were then visited, and the content and tone of the pages were coded as either Positive, Negative, or Neutral. Websites that outlined the benefits of vaccines or that recommended vaccines by directly addressing potential concerns were coded as Positive. Those that discussed vaccines as a given (i.e. discussed the creation and delivery process, compared the different vaccines currently in development, etc.) without directly addressing vaccine hesitancy were coded as Neutral (though it must be noted that, in presenting vaccines as an established fact, even those neutral sources lean significantly towards the positive side of the spectrum). Those that promoted vaccine hesitancy or cautioned against the risks of vaccines were coded as Negative. Additionally, all sources were classified as either governmental, academic, journalistic, professional, or encyclopedic.

Source Type
Of the 43 results coded, 67% were news sources (see Figure 2). Following this, the two most common sources were academic and encyclopedic (11.6% each). While governmental sources only made up 6.9% of total results, they were only present on Google, therefore making up 33% of Google search results. All websites, except for DuckDuckGo, had sponsored advertisements by credible sources (the World Health Organization and the Government of Canada) that linked directly to public health information about vaccines. All five search engines had an Note: Data retrieved from Google Trends (Google Trends, 2021) additional section for "Top Stories" or "News," which all linked to a variety of news sources.
Since these update frequently, they were not coded. Google alone had a section entitled "Common questions" which presented readers with a variety of potential enquiries ranging from "Are you immune to COVID-19 if you get it once?" to "Is bleach an effective cleaning agent for the coronavirus?" All these questions linked directly to information provided by the Government of Canada or the World Health Organization. Figure 3 shows the ranking of Positive (green), Neutral (yellow), and Negative (red) websites returned on different search engines. Two of the 43 sources could not be analyzed because the articles required a subscription. 58% of the remaining sources were coded as Neutral, 39% as Positive, and 2.4% as Negative.

Overlap
There was also significant overlap between sources across four of the five search engines. Yahoo and Ecosia had the most overlap, with six of Yahoo's (nine) and Ecosia's (eight) results being the same. All the other search engines, apart from Google, had between three and five overlapping results. Google's search results were completely unique.

Discussion
As with other studies that have looked at the quality of sources across different search All of Google's other sources also presented vaccines as fact (Neutral).
The fact that Google had no overlap with other search engines is also noteworthy because it means that while other search engines are likely using similar algorithms, Google's algorithm is unique. It is also public (unlike the others) and is therefore open to scrutiny. Scholars who have scrutinized it understand that "reputation and trustworthiness are key factors included in Google's ranking algorithm" (Ghezzi et al., 2020, p. 4). This results in more credible sources.
Though Google is far from perfect, especially when it comes to protecting its users' privacy (Kaplan, 2014;Mager, 2012), as the dominant search engine on the market, one might argue that it has an ethical responsibility to provide users with results from reputable sources. The fact that it does means that, in this respect at least, Google appears to be the most trustworthy of search engines.
Meanwhile, DuckDuckGo, the only search engine in this study that emphasizes users' privacy, was the least dependable. Since it does not profile users, all sources provided by DuckDuck-Go were based in the United States, despite the fact that the searcher was located in Canada. The fact that they did not display sponsored advertisements also meant, in this instance, that searchers were not linked to the credible sources (WHO.int and Canada.ca) that other search engines displayed as advertisements.
The search on DuckDuckGo also resulted in the only Negative source. The article came from The Western Journal, an American conservative news and political website based in Arizona. Interestingly, the website was blacklisted from Google in 2017 for spreading misinformation (The Western Journal, 2020). The article in question, entitled "Key Ingredient of Some Potential COVID-19 Vaccines Horrifies Conservatives, Christians," was coded as Negative for inducing fear about vaccines by linking it to abortion. The article calls the fetal tissues used in vaccines "unborn baby parts" and argues that vaccines are "diabolical." Though this was the only Negative source, these results support Ghezzi et al.'s (2020) finding that privacy-savvy search engines are more likely to display anti-vaccination sources, because they do not have mechanisms in place to prevent the spread of misinformation.
Overall, most search results across platforms provided reliable information. However, as Madden, Nan, Briones, & Waks (2012) suggest, it is likely that "those in vaccine critical communities have found alternative forums for criticism … such as social networking sites or YouTube" (p. 3745). Though social media platforms fell outside of the bounds of this study, they are often cited as the main culprits in the spread of misinformation online (Cooke, 2017) and should therefore not be discounted.
This research demonstrates that, despite the capitalist ideology that normally drives search engine ranking and advertising, in times of crisis, search engines might be trusted to provide its users with reliable information. However, the invisibility of their interference in our information seeking suggests that users might remain unaware of any potential breach of trust. We must, therefore, continue to question the power we give these corporations rather than simply hope they will adapt responsibly in the face of a global crisis. as opposed to "COVID-19" and "immunization" or "vaccination" instead of "vaccine."

Limitations
The searches were also conducted from a single device in one geolocation. It is likely that conducting the search across various devices and in different locations would have resulted in different information. It was also impossible to control for the possibility of personalized searches beyond clearing the browser history. It would have been interesting to see whether a different iJournal, Vol 6, No. 2, searcher using the same method would have been presented with the same results. Moreover, the fact that the evaluation of sources was conducted by a single person means the analysis is naturally biased. Other researchers might have interpreted the sources differently.
Finally, as Kata (2010) notes, any study of internet searching is limited by "the transient nature of the Internet" and by the fact that the Internet is "borderless," meaning that one source always leads to another (p. 1715). A more comprehensive search would have likely produced different results. Nevertheless, this study provides a snapshot into the kind of information provided by different search engines on the COVID-19 vaccine, and in so doing, invites us to question the information infrastructure of search technology.

Conclusion
This analysis shows that search engines are not free of bias, but that bias can sometimes be beneficial. When search engines intervene to ensure users are provided credible sources (as is the case with Google), they prevent further harm on public health caused by the spread of misinformation and disinformation. Nevertheless, data security should not come at the cost of data privacy.
By tracking how search results varied across different search platforms, this study aimed to shed light on the way search engines tailor the online search experience and how they might have been participating in the development of the public's consciousness on COVID-19 vaccination. In ensuring that they are providing accurate information about vaccines, search engines can join WHO in the fight to end the "infodemic" surrounding COVID-19.