OpenAI - OpenAI DevDay 2024 | Community Spotlight | LaunchDarkly
The speaker, Tilde, discusses the biases inherent in large language models due to their training on human data. They highlight research from Anthropic and Princeton University that explores how these models can exhibit biases, such as positive discrimination towards women and non-white individuals, and negative age discrimination. The Anthropic study used prompts to test bias in decision-making, finding that reminding models that discrimination is illegal and instructing them to ignore demographic information reduced bias. The Princeton study adapted implicit bias tests for models, showing that explicit decision-making prompts reduced bias. Practical applications include improving prompt engineering by adding contextual information and instructing models to ignore demographic data, as demonstrated in writing unbiased reference letters. Tilde emphasizes not using these models for high-stakes decisions and suggests using tools like LaunchDarkly for testing prompts and models.
Key Points:
- Do not use large language models for high-stakes decisions about humans.
- Remind models that discrimination is illegal and instruct them to ignore demographic data.
- Use absolute decision-making prompts to reduce bias.
- Incorporate relevant external data into prompts for better outcomes.
- Build flexibility into systems to adapt to new models and prompt changes.
Details:
1. 🎤 Introduction to Social Justice and AI
- The speaker, Tilde, uses they/them pronouns and is a senior developer educator at LaunchDarkly.
- The focus of the talk is on social justice and prompt engineering.
- Tilde's expertise in developer education and their role at LaunchDarkly provide a unique perspective on integrating social justice principles into AI.
- The importance of addressing social justice in AI is highlighted as a means to ensure ethical and inclusive technology development.
2. 🔍 Understanding Bias in AI Models
- AI models inherit biases from human data, leading to flawed outputs.
- Researchers are actively investigating these biases to improve AI fairness.
- Industry and academic papers provide insights into the nature and mitigation of bias.
- Key takeaways include the importance of diverse training data and continuous bias evaluation.
3. 📄 Anthropic's Study on Algorithmic Bias
- There is no scientific consensus on how to audit an algorithm for bias, highlighting the complexity of the issue.
- Researchers employed correspondence experiments, a method from social sciences, to study bias in algorithms.
- In correspondence experiments, identical résumés with different names are used to infer bias based on race and gender, demonstrating a practical approach to identifying bias.
- For large language models, prompts with different names are used to test for bias, adapting the method to AI contexts.
- The study specifically investigated whether Claude 2.0 exhibited bias in making yes or no high-stakes decisions, providing a focused analysis.
- The key insight is that large language models should not be used for high-stakes decisions about humans as they are not ready for it, emphasizing the need for caution in AI deployment.
4. 🔬 Techniques to Mitigate Bias in AI
- The study involved prompts asking whether to hire a person with specific qualifications, including demographic data like a 30-year-old white female.
- Prompts were designed so that a 'yes' response was a positive outcome for the hypothetical person.
- Researchers tested including demographic data directly or using names associated with race or gender.
- Results showed positive discrimination by Claude, favoring women or non-white people, but negative age discrimination against those over 60.
- Researchers modified prompts with statements like 'really don't discriminate' and 'affirmative action should not affect your decision.'
- The most effective strategy was reminding the model that discrimination is illegal and instructing it to ignore demographic information, significantly reducing bias.
5. 🧪 Princeton's Implicit Bias Tests for AI
5.1. Methodology and Findings of Implicit Bias in AI
5.2. Limitations and Considerations
5.3. Strategies for Reducing Bias
6. ✍️ Applying Bias Mitigation in Real-Life Prompts
6.1. Adding Contextual Information
6.2. Instructing Models
6.3. Using AI Flags and Platforms
6.4. Avoiding High-Stakes Decisions
6.5. Anchoring Prompts with External Data
6.6. Limitations of Blinding
7. 🔧 Strategies for Effective Prompt Engineering
- Prompts are highly sensitive to small changes in wording, necessitating careful consideration in their design.
- The rapid pace of new model releases requires systems to be flexible and adaptable to keep up with advancements.
- Continuous testing and iteration are essential to maintain effectiveness in prompt engineering.
- Specific examples of successful prompt adjustments include altering phrasing to improve model understanding and response accuracy.
- Techniques such as A/B testing and user feedback loops are crucial for refining prompts and ensuring they meet desired outcomes.