In AI and Information Retrieval, feedback loops and model contamination in large language models (LLMs) have been an ongoing issue since the inception of these systems, and it has only worsened over time. These powerful models are at risk of perpetuating and amplifying errors, biases, and false information. If this issue continues, some AI tools might become as useless as a chocolate teapot.
The Cycle of Misinformation: The Snake Eats Its Tail
LLMs learn by studying vast amounts of online content, which helps them model and generate human-like text. However, as more AI-generated content appears online, it mixes with the content that future models will learn from, creating a feedback loop where the mistakes of one model become part of the training data for the next. This is a major concern because AI-generated text can be very convincing and hard to distinguish from human-written text. It has been found that text that contains false and inaccurate information is often generated (“hallucinations” in LLMs).
EXAMPLE OF INACCURATE INFORMATION GENERATION
We did a test where we asked a famous chatbot and virtual assistant to write a news article about the launch of WebXray according to the official information provided.
RESPONSE
In a significant development in the search engine landscape, former Google engineer Lars Hansen has launched WebXray, a new search engine that promises to disrupt the status quo.
We don’t need to include all the generated text to find the problem. As we can see from the first lines there is already one piece of incorrect information: the creator of WebXray is Tim Libert, not “Lars Hansen”.
Fake News and AI-Generated Text
During the COVID-19 pandemic, numerous pieces of misinformation and conspiracy theories were spread online. Some AI-generated text, designed to mimic human writing, contributed to the proliferation of false information. For example, AI-generated articles and social media posts about treatments and vaccines were circulated, becoming part of the training data for other models and perpetuating inaccuracies.
When AI Gets It Wrong
The feedback loop and model contamination problem could make AI tools unreliable in many areas. If contamination continues, the trustworthiness of AI insights will decrease, potentially leading to poor decisions in important fields like healthcare, finance, and law. The problem extends beyond false information to biases in training data, which can be amplified by feedback loops, leading to AI systems spreading and strengthening unfair practices, such as biased hiring algorithms or discriminatory profiling in law enforcement.
Biased Hiring Algorithms
AI systems used for resume screening have shown biases, particularly against women and minorities. For example, a prominent tech company scrapped its AI hiring tool because it was found to be biased against female candidates. The tool was trained on resumes submitted to the company over a decade, which included biases present in human hiring practices. As the tool was used and generated biased recommendations, these biases were further entrenched.
AI in Law Enforcement
Predictive policing tools, such as those used in some U.S. cities, have been criticized for reinforcing racial biases. These systems often rely on historical crime data, which reflects existing biases in policing. As these tools make predictions and guide law enforcement actions, they can exacerbate existing biases, leading to a feedback loop where marginalized communities are disproportionately targeted.
Broader Implications
The feedback loop and model contamination issue applies to other AI systems beyond text-generating LLMs, including recommendation systems, search engines, translation models, sentiment analysis, and speech recognition systems.
Recommendation systems
If a movie recommendation system learns from a biased dataset that over-represents certain genres or demographics, it may start pushing content that reinforces these biases, leading to a feedback loop where the system only promotes a narrow range of content and ignores diverse options.
Search Engines
A search engine trained on biased information may prioritise certain types of content over others, leading users to find information that reinforces existing biases. This can lead to a feedback loop where the search engine continues to learn from and promote biased content.
Translation Models
If a translation model is trained on data that contains biased translations, it may consistently produce translations that reflect those biases, thereby spreading and reinforcing them across languages and cultures.
Sentiment Analysis and Other Predictive Models
A sentiment analysis model trained on biased data may consistently misinterpret the sentiment of certain types of content, leading organisations to make poor decisions based on inaccurate insights.
Speech Recognition Systems
A speech recognition system trained primarily on data from speakers with a particular accent may struggle to accurately transcribe speech from speakers with different accents, creating a feedback loop where the system continues to perform poorly for those users.
What are the solutions to the problem?
To address the problem of feedback loops and model contamination across various types of LLMs, the following strategies can be applied:
- Better filtering and validation methods: develop better filtering and validation methods to ensure training data is accurate, including separating human-generated content from AI-generated content.
- Algorithmic Fairness: Developing and implementing algorithms that can detect and mitigate biases during both the training and inference stages.
- Human Oversight: Incorporating human oversight in the training process to identify and correct biases and errors in the data.
- Data Diversity and Quality: Ensuring that training datasets are diverse and representative of all relevant demographics and contexts. This helps in minimizing biases and errors from the outset.
- Transparency and Accountability: Maintaining transparency about data sources and training methodologies, and holding developers accountable for the impacts of their models.
- Continuous Monitoring and Updating: Regularly monitoring the performance of models and updating them with new, unbiased data to prevent the accumulation of errors and biases over time.
Conclusion
The feedback loop and model contamination problem is a significant threat to the future of AI and information retrieval. Without proactive measures, AI systems, once seen as revolutionary tools, risk becoming unreliable sources of information. We must work on the integrity of LLMs, ensuring they continue to help human knowledge grow rather than undermine it. By taking these steps, we can safeguard the reliability and utility of all types of large language models, maintaining their role as valuable assets in various domains. Let’s not allow our AI systems to become the proverbial snake that eats its own tail.
