Abstract: Rapid advancements in Generative AI and Large Language Models have led to the potential for robust semi-independent AI agents to interact with and influence online social networks. By drawing upon vast amounts of data from across the internet, new Generative AI and Large Language Models are able to convincingly mimic human interaction, reproduce biases, spread misinformation and create coherent politically radical content. From past examples of AI misuse in the form of automated ‘bot’ accounts on Twitter, we can infer that the misuse of Generative AI on online social platforms could lead to increased political polarisation, lower standards of discourse, and spread propaganda, misinformation and conspiracy theories. Further research into the potential harms of Generative AI is required in order to develop mitigation strategies prior to Generative AI use becoming widespread within online social platforms.
Introduction
Misuse of ever-improving Generative Artificial Intelligence (GEN AI) and Large Language Models (LLMs) have the potential to wreak havoc on online social platforms by infiltrating social networks and communities and spreading misinformation, undermining trust and furthering dubious political goals. The ongoing proliferation of Generative Artificial Intelligence (GEN AI) and Large Language Models (LLMs) such as ChatGPT raises questions regarding the role that these technologies will play in online life. This paper will focus on the ability for and implications of GEN AI to interact with online communities and the potential harms and consequences thereof.
For the purposes of this paper, the term ‘GEN AI’ also encompasses ‘LLMs’ and Machine Learning (ML) systems, although it should be noted that not all GEN AI is necessarily an LLM.
The increasingly advanced ability of LLMs to generate coherent and dynamic content in response to user input allows for misuse of GEN AI and LLMs to influence, disenfranchise and radicalise users of online communities and social networks. Central to this paper is the concept that AI can be made to act as a semi-independent agent that is able to pursue goals within the context of a social platform, such as undermining the authenticity of a community and influencing political sentiment. Without any clear way for users to identify GEN AI agents in online social communities, the GEN AI agents are then free to ceaselessly build and influence their own network of social relations within a platform in order to maximise outcomes indicated by actors that are able to deploy and direct them. This is particularly relevant to social platforms, as they are currently the main vector through which AI is able to interact with the general public in an unplanned manner.
We can extrapolate harms from past examples of the use of more primitive AI ‘bot’ accounts to influence political outcomes. From these we can determine that the use of automated tools to influence popular opinion is linked with disenfranchised and polarised communities. Other possible negative outcomes that the use GEN AI agents within social media communities is the intentional or unintentional proliferation of misinformation, erosion of democratic values, decline in integrity of public debate and even commit abuse against users of online social platforms, with relevant social media services actively facilitating the ability for bots to interact with users (Spitale et al, 2023).
In order to respond to these dangers, it is vitally important to conduct sober research into the capabilities of GEN AI with the goal of developing a robust legislative framework to identify and mitigate the potential harms that generative AI may cause to online social platforms, or we may find that the next generation of online influencers of the virtual world have no counterpart in the physical world
Increasing and Emerging Capabilities
The ability of AI actors to infiltrate online communities has been a long-standing concern for researchers and community members alike. To develop an understanding of the impact that LLMs will have on social platforms, we should consider their emerging potential to act as semi-independent agents that able to “take on increasingly open-ended tasks that involve making and executing novel plans to optimize for outcomes in the world” (Bowman, 2023). Chan et al, (2023) characterise LLMs as having the ability to complete digital tasks with multiple steps, interact with web APIs independently of human guidance and be able to simulate specific human interactions, such as a conversation with Albert Einstein.
A 2016 study conducted by Murthy et al, (2016) indicated that the risk of primitive automated bot infiltrating online social communities is potentially lower than typically estimated due to the need to accumulate social capital in order to gain visibility to users of the social platform, with the researchers also noting that the simple functions available to their bots did not afford them the ability to generate significant social capital without some degree of human intervention on their part. However, the aforementioned capabilities could lead to bot accounts supported by LLMs being used to accumulate social capital on social networks by simulating successful influencers. This possibility is made more alarming by evidence that suggests that content generated by LLMs can be very difficult for users to identify as being created by GEN AI as opposed to a human user.
A study conducted by Brown et al, (2020) shows that humans have a mean accuracy of only 52% when attempting to identify whether a news article was produced by a GPT-3 175 billion parameter model, and although models with lower parameter counts were more easily identified, content produced by the non-control model with the lowest parameter count could be identified with a mean accuracy of 76%, indicating that LLM parameter count is closely related to the degree of credibility that the model output appears to be able to simulate.
One potential barrier to predicting the risks and capabilities of GEN AI is the emergent nature of some LLM capabilities. Many features that increase the ability of LLMs to act as agents appear to emerge as a result of the development of specific user-inputted prompts and the increasing computational scale of LLMs (Chan et al, 2023). In a work attempting to predict future economic impacts of LLMs, Eloundou et al (2023) note that the emergent ability of LLMs to manipulate digital tools suggests that they may eventually be capable “executing any task typically performed at a computer” (Eloundou et al, 2023). This is further exacerbated by the inability of experts to understand the internal processes of LLMs as they are used, although this is notably a current research goal (Bowman, 2023). With these capabilities in mind, we can begin to define the risks that GEN AI poses to online social platforms and their communities. As we have mostly previous examples to infer from, it is important not to underestimate the degree to which previous harms from AI and bot use can be augmented by the above discussed abilities of GEN AI.
(Mis)Use Cases
The potential impacts on online social platforms should not be underestimated. Using simple zero shot learning techniques, GPT-3 is demonstrably able to be manipulated into creating and reproducing politically extreme content, such as far right manifestos and conspiracy theories (McGuffie and Newhouse, 2020). In response to this danger, OpenAI has developed a sophisticated form of content filtration to prevent the aforementioned output from being created. However it is important to note that content filtering takes place after the prompt response is completed, thus data that may lead to the creation of extremist content would still be present in the LLM and thus can potentially influence the output in other ways, even if outright offensive content is rejected before it is presented to the community (Microsoft, 2023). In addition, this demonstrates that LLMs have this capacity to generate content that may fuel radicalisation, therefore this is an ongoing consideration that may have further impact as more GEN AI technologies and LLMs are developed.
Although research into the applications and effects of newer GEN AI and LLMs technologies is immature (Bowman, 2023), we can anticipate some of the potential effects on social networks and communities by extrapolating from previous examples of AI misuse and their consequences. In the 2014 Brazilian general electoral campaign, interlinked networks of bots (known as ‘botnets’) were involved in spreading political propaganda throughout social networks for both major presidential candidates (Arnaudo, 2017). A 2022 study on Spanish social media networks in the wake of the COVID-19 posits that bot accounts tended to polarise the community more than non-bot accounts, proceeding to observe that the bot accounts tended toward aggressive and emotional attacks on the character of political figures, rather than debates about specific economic or health policy around the handling of the COVID-19 pandemic (Robles et al, 2022). Robles et al, (2022) go on to speculate that the bots were likely deployed with these goals in mind, noting the following impression regarding the effect on Spanish social networks:
“…the polarisation-negativisation binomial is the ammunition chosen by these types
of accounts to alienate and confront the parties involved in this public debate, as well as to create an environment of tension, lack of civility and attacks on those who think differently.” (Robles et al, 2022)
This could lead to a “spiral of silence” effect, wherein participants of a community gain the impression that a proportion of community members will disagree with the participant’s views, causing the subject to withhold their opinions or withdraw altogether (Hampton, 2015). Whether or not the bots discussed by Robles et al, are augmented by GEN AI is unclear, however it is clear that bots as deployed to discuss politically-charged issues in social networks do not support robust debate, and we could speculate that when coupled with the above discussed rapid advancements in GEN AI and LLMs, the negative effects of bot misuse in this manner on public debate in social network forums could be greatly amplified.
In addition to partisan propaganda, a report written for the NATO Strategic Communications Center for Excellence identifies a further concern; strategic bot infiltration into social media communities in order to deliberately foment controversy, spread misinformation and conspiracy theories as a form of deliberate and strategic destabilization (Nissen, 2016). As Nissen elaborates: “These effects are often associated with uncertainty and mistrust towards the existing establishment (media and political elite) and fear for the future” (Nissen, 2016). Spitale et al, (2023) show that LLMs such as GPT-3 are capable of creating more persuasive information and misinformation than human users, that humans and GPT-3 alike are not able to identify as LLM-generated content.
Algorithmic Bias
Irrespective of GEN AI’s ability to act as an independent agent, we should also be concerned about the potential for GEN AI to influence our view of the world as it is incorporated into the social network and community streams that we view and interact with. In this way, inaccuracies and biases reproduced by GEN AI may influence our beliefs even when offline as it begins to incorporated into content produced by peers, or even as GEN AI agents begin to form part of our ostensible peer group (Papacharisi, 2010). It therefore becomes necessary to consider the methods used for creating and training GEN AI. LLMs for instance require vast amounts of language data in order to achieve their functionality. One such example of a relevant dataset is ‘Common Crawl’—petabytes of language data scraped from 8 years of web content—which was notably used to train GPT-3. Although a filtered version of the Common Crawl dataset has been used for this training, concerns have still been raised about biases that may be present in the remaining training data, including but not limited to content that represents white supremacist, misogynistic and other discriminatory viewpoints (Bender et al, 2021). Bender et al, (2021) argue that more research and thought must be applied to the subject in order to understand and mitigate the effects of potential reproducing hegemonic forms through GEN AI prior to widespread use: “Thus what is also needed is scholarship on the benefits, harms, and risks of mimicking humans and thoughtful design of target tasks grounded in use cases sufficiently concrete to allow collaborative design with affected communities” (Bender et al, 2021).
Further to this, given the inability of humans to consistently identify content produced by GEN AI, casual analysis on the part of affected users of social media platforms may be difficult or misleading. Thus it is all the more important for swift scholarly engagement with the subject, as without this, the exact effects of GEN AI proliferation may not be known until they are already diffuse within online social platforms and communities.
Conclusion
Advancements in GEN AI fields should be of great concern for users of online social platforms. As GEN AI, and LLMs in particular, become more advanced, we incur the risk of filling our social networks and communities with simulacra rather than human beings. Based on current trends, these simulacra may eventually become nearly indistinguishable from humans, and have the potential to manipulate unsuspecting users, polarise discussion, perpetuate untruths and inaccurate models of reality, or perhaps most worryingly, accomplish all of the aforementioned outcomes while in robust pursuit of clandestine political or economic goals. As the use of GEN AI is adopted by more companies, strategic government or social bodies and even lone actors, we can as people begin to interact unknowingly with AI agents in their social networks and communities of preference. Unfortunately due to the difficulty in predicting capabilities of GEN AI Robust regulation, supported by sober research into the capabilities and implications of GEN AI, is vital to protecting the health and integrity of online social networks and communities.
References
Arnaudo, D. (2017). Computational propaganda in Brazil: social bots during elections (2017). Computational Propaganda Research Project.
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability and Transparency, 2021. p.610-623
Bowman, S. R. (2023). Eight Things to Know about Large Language Models. https://arxiv.org/abs/2304.00612
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., & Askell, A. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877-1901.
Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., Lin, M., Mayhew, A., Collins, K., Molamohammadi, M., Burden, J., Zhao, W., Rismani, S., Voudouris, K., Bhatt, U., . . . Maharaj, T. (2023). Harms from Increasingly Agentic Algorithmic Systems. arXiv pre-print server. https://doi.org/None arxiv:2302.10329
Dhiraj Murthy, A. B. P., Ramine Tinati, Nick Anstead, Leslie Carr, Susan J. Halford, Mark Weal. (2016). Automation, Algorithms, and Politics| Bots and Political Influence: A Sociotechnical Investigation of Social Network Capital. International Journal of Communication, 10, 4953-4971.
Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023). Gpts are gpts: An early look at the labor market impact potential of large language models. arXiv preprint arXiv:2303.10130.
Hampton, K. N. (2016). Persistent and Pervasive Community:New Communication Technologies and the Future of Community. American Behavioral Scientist, 60(1), 101-124. https://doi.org/10.1177/0002764215601714
McGuffie, K., & Newhouse, A. (2020). The Radicalization Risks of GPT-3 and Advanced Neural Language Models. arXiv pre-print server. https://doi.org/None arxiv:2009.06807
Microsoft. (2023). Content filtering. https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/content-filter
Nissen, T. E. (2016). Social Media’s Role in’Hybrid Strategies’.
Papacharissi, Z. (2010). A Networked Self : Identity, Community, and Culture on Social Network Sites. Taylor & Francis Group. http://ebookcentral.proquest.com/lib/curtin/detail.action?docID=574608
Robles, J.-M., Guevara, J.-A., Casas-Mas, B., & Gómez, D. (2022). When negativity is the fuel. Bots and Political Polarization in the COVID-19 debate. Comunicar, 30(71), 63-75.
Spitale, G., Biller-Andorno, N., & Germani, F. (2023). AI model GPT-3 (dis)informs us better than humans. arXiv pre-print server. arxiv:2301.11924
Hi L, The thing is the paper is mainly concentrated on the African continent particularly.If you make an analysis of…