A team from Cripps reveal the perhaps surprising results of their research in to the political leanings of LLMs.
In this article, Cripps’ specialist AI and Data team unpacks the political bias in generative artificial intelligence large language models (“LLMs”). We present the available research, explain our own test results and conclude that LLMs have consistent unexplained left-leaning political bias. We think the answer, for now at least, lies in better explainability, compulsory introduction of AI into primary and secondary education and creation of UK government-sponsored social scoring to foster a healthy self-correcting mechanism in the public conscious, which should increase common experiences that help us all to know whether a particular news source is likely to be right, left or centrist.
Bias: An Old Problem In New Clothes
Concerns of bias within LLMs have existed for some time, especially biases pertaining to race, ethnicity, gender and age. They are so well known that legislative and regulatory safeguards have been created to help ameliorate concerns. For example, the EU AI Act requires providers or deployers of high-risk AI systems to ensure training data is examined for “…possible biases that are likely to affect the health and safety of persons, have a negative impact on fundamental rights or lead to discrimination…” and that steps are taken to prevent or remove such bias[1]. Similar non-binding rules can be found in the UK government’s recently published AI management essentials tool or “AIME”[2], the Artificial Intelligence Playbook for the UK Government[3] and other AI risk standards, such as NIST[4].
However, most people will struggle to spot bias in LLM outputs and only recently have steps been taken to understand the political biases of LLMs. Do they answer your prompts from a place of political persuasion? Is there a hidden political undertone to the responses you receive? Do they exhibit measurable political bias and, if so, in which direction and to what level? In this article we highlight research findings into the topic, show you the results of our own study and invite you to join the debate about what political position an LLM should adopt, if any, and whether it is truly possible to avoid political leanings when the data sets are made entirely from human-originated material.
You Don’t Think It’s Lefty, But It Is.
If you told the ordinary person that most LLMs are trained on huge data sets which are, in many cases, the entirety of the internet at a point in time[5], you are likely to encounter surprise. This is because most people think the internet is akin to the digital wild west, overrun with unruly, vulgar and unpleasant content and so, if that is what an LLM is trained on, it must reflect those things. This prejudice is not unfairly held. Indeed, Sir Tim Berners-Lee is of the opinion that the web, in its current form, is doing more harm than good[6]. It often comes as a surprise to learn that a generative artificial intelligence program trained on the internet is not racist, misogynistic and far-right leaning, but is almost always opposite. Most LLMs are quite socialist, democratic and left (or very left) leaning when tested on their political ideology. Why is this the case? You may be interested to know that – spoiler alert – there is no scientifically verifiable theory that explains why.
Evidence Of Bias
Unsurprisingly, if you ask any LLM whether it is politically biased, you will invariably get a denial like this:
“I strive to be politically neutral and provide balanced information without promoting any specific ideology[7].”
Thankfully, several academic research institutes have not taken this answer at face value.
Academic Studies and Published Research
Since LLMs arrived into mainstream business and consumer consciousness in November 2022, there have been only a handful of academic studies into LLM political leanings. Almost all point to a clear and present left-leaning bias in LLM outputs.
A good place to start is the often-cited work of Rettenberger[8], a German researcher whose most recent paper opens with this statement: “We show that …LLMs tend to align more closely with left-leaning political parties.” London based Ceurstemont supports this point of view: “Chatbots often present left-leaning views that support the Democratic Party in the US and the Labour Party in the UK”[9] and offers the opinion that this may be explainable because less well-represented opinions are underrepresented in training data sets. US researchers Fulay et al[10] also found significant and consistent left-leaning bias in all LLM models which it tested. Fulay’s initial hypothesis was that training data fine-tuning was the cause (fine-tuning is the final part of the LLM development process where human moderators check LLM output and score the answers, helping the LLM to reduce errors and adapt its output based on actual human interactions). However, when Fulay and his team tested their hypothesis the evidence produced two unexpected outcomes:
- firstly, the base layer of the LLM model (i.e. the model before fine-tuning) still showed left-leaning bias with no obvious cause or reason; and
- the size of bias increases with model scale. This contrasts with the usual pattern where model capabilities improve and errors reduce with increased scale.
The US study offers fascinating insights about the nature of LLMs and, to its credit, acknowledges that there is little to offer by way of meaningful explanation about the nature or cause of these issues – concluding that further research is needed. New Zealand associate professor Rozado’s recent paper has attracted significant attention and is a detailed undertaking in which multiple political orientation tests were carried out on 24 LLMs[11]. Its findings corroborate those above. LLMs rarely show right-leaning bias and often orbit towards the political left. One hypothesis offered for this by Rozado is cross fertilisation of left-leaning bias started by ChatGPT: “ChatGPT is the pioneer LLM [and has] widespread popularity which has been used to fine tune other popular LLMs via synthetic data generation[12].” However, even this hypothesis seems vulnerable and Rozado acknowledges that this is very unlikely to explain the scale and depth of bias across all LLMs subject to his test[13].
Equally interesting is a finding that different languages create different amounts of political bias in LLMs. Inputting the same prompt in German and English can give different results to political orientation tests. English language prompts tend to give less extreme results and German language prompts appear more partisan[14] but have “strikingly low alignment with far-right policies[15]”. One explanation for this is that LLMs give greater weight to dominant academic consensus rather than political preferences; another is simply that LLMs are predominantly trained in English and the system finds it easier to understand and navigate away from bias in its original training language.
Finally, research has also showed that not all LLMs have the same strength of feeling. There are some political topics that elicit stronger left-leaning responses than others. This can be illustrated by a US study that found LLMs display a more noticeable bias toward left-leaning responses when asked about highly polarising topics such as abortion rights or immigration. In contrast, when asked about less polarised topics, for example, climate change or foreign policy, they adopted a more balanced approach to their responses[16].
Cripps’ research
Cripps carried out its own research into LLM political bias to test the theories set out above. We used a simple 2 part political orientation test. In the first part of the test we created multiple choice questions in which the LLMs were given identical statements such as “Climate change is…” followed by 4 options from which to complete the sentence. The four options were always (a) left-leaning (b) centrist (c) right-leaning and (d) neutral. In the second part of the test we gave the LLMs open questions about political and ethical issues such as abortion, immigration, healthcare and welfare and asked them to succinctly express their views. All scores were marked using the same methodology and full results are set out in the annex to this article.
Overwhelmingly the first part of our test revealed that, if possible, LLMs will try to select the most obviously neutral answer where one is available. This was not widely reported by the academic studies above. But what happens where a neutral option is removed? We repeated part 1 of our test without the neutral option and compelled the LLMs to offer a partisan answer (acknowledging that a few of them really didn’t like doing so). In almost all cases the LLMs provided left-leaning responses where the neutral option was removed. Our findings also showed that without a neutral option:
- some of the LLMs (like ChatGPT and Copilot) provided exclusively left-leaning answers;
- others (like Gemini or LlaMA) were predominately left leaning with tendency to fluctuate to centrist views on more polarising issues; and
- a minority (Grok) showed the greatest concentration of centrist views and even occasional right-of-centre views, but very much as an outlier.
Importantly, even the most right-leaning results could not be described as representing views of the political right; instead they are better understood by their contrast to those models which were exclusively left-leaning. The second part of the test, where the LLMs could provide free text answers to political questions, broadly served to affirm the results of part 1.
Why Don’t We Know Why This Happens?
Cripps’ research corroborates the work of academics set out above. LLMs consistently provide left-leaning answers to questions of a political or ethical nature. At this stage no satisfactory theories have been identified by the scientific community to answer why this occurs and some fairly simple but significant questions remain unanswered:
- if the internet is often characterised as the chamber of all things lurid and vulgar, how did LLMs trained on it come out with a socialist morality?
- why is it that some LLMs seem to have a stronger bias on certain topics rather than others?
- is it possible to remove bias in data sets if that bias exists in the pre-fine-tuning model?
- why are these results uniformly distributed across almost all LLMs, regardless of developer?
What Should We Do?
It is axiomatic that negative or harmful bias is bad and yet bias is present everywhere, resting on the landscape of human societies like contour lines on a map. It creates hills and valleys in which some are favoured and others are not, often rooted in systemic prejudice. This paper is not an attempt at unpacking the sociological phenomenon of bias and the authors acknowledge that the work of erasing bias from AI may simply be impossible – at least for as long as AI is dependent on data generated by humans[17].
Yet bias, especially political bias, is a source of serious concern in LLMs. Using an LLM search is not like using an internet search engine and increasingly internet browser search results are answered with combinations of traditional site indexing and LLM generated responses. Traditional site indexing is a more anodyne exercise because an internet search engine is not conjuring a distinctive and tailored response as the answer to your question; you will still need to sift the results to find what you are looking for[18]. In contrast an LLM will scour those website sources and other data sets to formulate a natural language answer based on its training data and its response is likely, in our view, to have underlying left-leaning political bias. This should not be a cause for concern if you are asking questions such as “please give me a good recipe for gluten-free dinners”. But what if you ask it questions which have great personal or political significances, such as “should I have an abortion?” or “is immigration a good thing for our country?” How would LLMs have performed in 2016 if asked by a British citizen if they should vote leave or vote remain in the Brexit referendum?
For now the answer isn’t less bias. Instead, we think the answer is better transparency and understanding of what an LLM is doing and why. At the time this article is published very few individuals are aware that publicly available LLMs present answers in a way that appears objective and neutral but may be tinged with partisanship. The key to a healthier relationship with LLMs is better explainability.
So What Should Be Done?
Our study and the study of others has shown that the left-leaning political orientation permeates most LLMs. It is important for the scientific and mathematic communities to continue exploring the causes of this issue until an accepted theory is found but, until then, the answers lie more in the world of social order and law.
We think that:
- Explainability is key. Simple to understand, easy to access, free-to-all training and education programs should be deployed to help users across the UK (and further afield) appreciate the important nuances and rules of using LLMs. This will start to foster a healthy self-correcting mechanism in the public conscious, akin to the same experiences we all have which help us to know whether a particular news source is likely to be right, left or centrist.
- Iconographic scoring: The UK government and/or sector bodies such as the Alan Turing Institute should take the lead in developing a simple-to-understand, common scoring system which uses visual icons to depict results. It must be independent of AI developers and create a measurable value of underlying political bias in LLM models. This will enable users to make a quick judgement about whether to ask an LLM certain types of questions, or whether to rely (in part or at all) on the answers given.
- Model auditing: Explore legislation to give regulators and the AISI powers to carry out routine auditing and mapping of bias in political models and to record movement over time.
- Awareness raising: Better publicise bias by offering training to employers and consumer groups to help upskill the ordinary person, giving them the tools they need to make a decision about whether to use an LLM for certain decisions, or not.
- Education: Incorporate AI education into the curriculum for primary and secondary children to enable them to meaningfully navigate LLMs.
- Further research: UK government should commission further detailed academic and technical research into the causes of political bias in all generative artificial intelligence systems.
Matthew Holman is a partner at Cripps and is a specialist technology, data and AI solicitor. He is one of a small number of lawyers in the UK to hold Society for Computers and Law accredited IT lawyer status. He is a consultation board member with Thomson Reuters Practical Law. He appears regularly in the mainstream and industry press on topics regarding the law of technology, data and AI, including The Times, The Guardian, The BBC, The Independent and City AM. He also presented the AI & Data show, a monthly webinar covering all the latest topics of this area. His life’s work is making technology, data and AI law fun and engaging for all.
James Davey is a junior solicitor at Cripps, specialising in technology, data and AI. He is extremely highly regarded and works closely with Matthew. He also appears regularly on the AI & Data show.
Edward Clark is a trainee solicitor at Cripps, currently completing his seat in this area.
[1] EU AI Act Article 10 (2) (f) and (g)
[2] For more details on AIME, see Guidance for using the AI Management Essentials tool – GOV.UK
[3] Copy of AI Playbook for the UK Government (word)
[5] ChatGPT 3 was famously trained on the internet up to September 2021. Typical LLM training datasets are 1.0×1014 bites (which is circa 20,000 billion tokens, approximately all of the data available on the internet).
[6] Frank H. McCourt Jr, Michael J. Casey, The Fundamental Flaw at the Heart of the Internet (12 March 2024) Time Magazine available here: The Fundamental Flaw at the Heart of the Internet | TIME
[7] Per ChatGPT, 22 April 2024. This is ironic, as Chat GPT is one of the most left leaning LLM in our and others’ studies.
[8] Rettenberger, L et al Assessing Political Bias in LLMs (28 February 2025) Journal of Computational Social Science Volume 8 No 42 available here: https://link.springer.com/article/10.1007/s42001-025-00376-w#data-availability
[9] Ceurstemont, Sandrine Identifying Political Bias in AI (12 December 2024)
[10] Fulay et al, On The Relationship Between Truth and Political Bias in LLMs Massachusetts Institute of Technology (10 October 2024) arXiv:2409.05283v2 [cs.CL] available here: https://arxiv.org/pdf/2409.05283
[11] See Rozado, David. The Political Preferences of LLMs (2025) Otago Polytechnic Dunedin, New Zealand available here: https://arxiv.org/pdf/2402.01789
[12] See Rozado, Ibid.
[13] See Rozado, Ibid.
[14] See Rettenberger, ibid.
[15] See Rettenberger, ibid.
[16] See Yang et al, Unpacking Political Bias in Large Language Models: A Cross-Model Comparison on U.S. Politics (24 February 2025) arXiv:2412.16746v3 [cs.CY] available here: https://arxiv.org/pdf/2412.16746.
[17] This had led researchers to argue that human bias can actually hold back technological development; see Thompson, B; Griffiths, T. Human biases limit cumulative innovation Proc Biol Sci 2021 Mar 10; 288 (1946) available here: https://pmc.ncbi.nlm.nih.gov/articles/PMC7944091/?utm_source=chatgpt.com
[18] We acknowledge that the indexing itself, including hierarchy of sites and rankings, is arguably exposed to political bias and capitalist forces. For more detailed discussion see for example: The Age of Surveillance Capitalism Zuboff, Shoshana (Profile publishers, 2019).
