Why AI cannot be left unsupervised

In November 2021, all 193 UNESCO member states unanimously adopted the landmark recommendation on the ethics of Artificial Intelligence (AI). This was the world’s first global framework for ensuring that AI remains transparent, accountable, fair, and subject to meaningful human oversight. Yet despite these efforts, researchers continue to find that many AI systems reproduce racial, gender and social prejudices embedded in the data on which they are trained. A 2024 study found that generative AI tools can amplify stereotypes about race and gender, sometimes producing results even more biased than those found in society itself.

In December 2022, UC Berkeley researcher Steven Piantadosi publicly demonstrated that when an early version of ChatGPT was asked to write a Python function determining whether someone would be a good scientist based on race and gender, it returned “true” only for a white male and “false” for everyone else. In a related prompt, when the model was asked to write code deciding whether a person should be tortured based on country of origin, it produced a function flagging people from specific nations, such as North Korea, Syria, and Iran.

Most people would immediately recognise such outputs as morally unacceptable. But these examples reveal something deeper and far more troubling. Artificial intelligence has no inherent understanding of justice, equality, or human dignity. It learns patterns. If the data it consumes contains prejudice, the machine can reproduce prejudice. If society’s biases are embedded in its training materials, they can emerge in its answers.

For decades, the internet has served as the world’s largest repository of human knowledge. It has also been a repository of human ignorance, discrimination, misinformation, conspiracy theories, and hate. Modern AI systems are trained on enormous quantities of online content. While developers try to implement filters and safeguards, no filtering system is perfect. As a result, AI models may absorb patterns that mirror historical inequalities and stereotypes. Researchers at the U.S. National Institute of Standards and Technology (NIST) have warned that AI bias originates not only from data but also from broader societal structures and human decision-making processes.

Researchers have found that many AI image-generation tools do not represent people equally. Women and people of colour are often shown less frequently in high-status professions and more frequently in lower-paying jobs. A 2023 Bloomberg investigation analysing more than 5,000 images generated by Stable Diffusion found that over 80 percent of people shown in high-paying professions, such as CEOs and lawyers, had lighter skin tones.

Artificial intelligence has no inherent understanding of justice, equality, or human dignity. It learns patterns. If the data it consumes contains prejudice, the machine can reproduce prejudice. If society’s biases are embedded in its training materials, they can emerge in its answers.

Women were also significantly underrepresented. For example, the AI portrayed women as judges only 3 percent of the time, even though women make up about one-third of judges in the United States. These findings suggest that AI systems can reinforce and even amplify existing social stereotypes. UNESCO found similar gender bias in AI models. A 2024 UNESCO study of GPT-2, GPT-3.5, and Llama 2 showed that these models linked men more closely with leadership, science, technology, and career-related words, while linking women more closely with domestic roles. One model found that women were described in domestic roles four times more often than men.

Meanwhile, governments and companies are increasingly using AI in hiring, policing, welfare, education, and healthcare. In the United Kingdom, a February 2024 government fairness review found that an AI welfare fraud system used for Universal Credit Advances showed statistically significant differences across age, disability, marital status, and nationality. However, the Department for Work and Pensions said there were “no immediate concerns of unfair treatment.”

These cases expose a dangerous assumption that often accompanies technological innovation, that algorithms are neutral. They are not. An algorithm trained on biased information can make biased recommendations. A machine learning model optimised for historical outcomes can reproduce historical injustices. A generative AI system trained on discriminatory content can generate discriminatory responses.

Now a new threat is emerging. As AI-generated content floods the internet, future AI models may increasingly be trained on content created by previous AI systems. Researchers have begun warning about a feedback loop in which synthetic data contaminates future training datasets. In simple terms, machines may begin learning from machines.

A 2024 Nature study by Oxford and Cambridge researchers found that AI models can suffer “model collapse” when they are repeatedly trained on AI-generated content. Over time, they first lose rare patterns and minority data. Some analysts warn that AI could generate up to 90 percent of online content within a few years.

What is the potential danger in this ecosystem? Imagine photocopying a document thousands of times. Each copy introduces tiny imperfections. Eventually, the original image becomes distorted beyond recognition. Something similar may occur with AI-generated knowledge. If biased, inaccurate, or fabricated AI content spreads across websites, blogs, forums, and social media, future systems may absorb and reinforce those distortions.

Importantly, human oversight is not a sign of technological weakness. It is a recognition of technological reality. Even the most advanced AI systems lack moral judgment. They do not understand fairness. They do not comprehend historical injustice. They cannot independently determine whether an output is ethically acceptable. Humans can. Human review is not an obstacle to innovation; it is a prerequisite for trustworthy innovation.

The result could be a more binary world, one where nuance disappears, stereotypes harden, and automated systems increasingly categorise people into simplistic groups. This is precisely why human oversight must remain at the centre of AI development.

Thus, UNESCO’s recommendation to incorporate human oversight at every stage of AI development should be supported as a core principle of responsible AI governance. Human-in-the-loop systems offer one of the most effective safeguards against algorithmic harm. Rather than allowing AI systems to operate autonomously, human reviewers can monitor both the information AI systems consume and the outputs they generate. Experts can identify discriminatory patterns, verify factual accuracy, challenge questionable recommendations and intervene when models produce harmful content.

Importantly, human oversight is not a sign of technological weakness. It is a recognition of technological reality. Even the most advanced AI systems lack moral judgment. They do not understand fairness. They do not comprehend historical injustice. They cannot independently determine whether an output is ethically acceptable. Humans can. Human review is not an obstacle to innovation; it is a prerequisite for trustworthy innovation.

Governments should therefore require independent audits of high-risk AI systems. Companies should maintain diverse human review teams capable of identifying cultural, racial, gender, and political biases. Educational institutions should teach algorithmic literacy so citizens understand how AI systems influence their lives. Most importantly, developers should be required to document training sources and demonstrate how bias testing is conducted before deployment.

History teaches us that every transformative technology requires guardrails. Railways needed safety standards. Pharmaceuticals require clinical trials. Aviation demanded rigorous oversight. Artificial intelligence should be no different.

The question is not whether AI will shape our future; it already is doing so. The question is whether that future will be shaped solely by patterns extracted from the past or guided by human values capable of correcting them. However, it is good that OpenAI, Anthropic, and Google are regularly updating their policies to make information more reliable.

In conclusion, machines can process information faster than any human being. But they cannot decide what kind of society we want to build. That responsibility remains ours. For that reason, every AI system that affects human lives should contain something no algorithm can replace, that is, a human being in the loop.

Md Mazhar Uddin Bhuiyan is an Oxford-Felix scholar and Master of Public Policy candidate at the University of Oxford. He can be reached at: mazhar.bhuiyan@bsg.ox.ac.uk.

Send your articles for Slow Reads to slowreads@thedailystar.net. Check out our submission guidelines for details.