Bias and fairness
Where model bias comes from, the workplace situations where it bites, and practical countermeasures that fit in a prompt.
Models learned from human writing, and human writing carries human bias — about gender, age, ethnicity, accents, schools, job titles. The model didn't decide to be biased; it absorbed the averages of the internet. At work, those averages can quietly shape real decisions.
Where it actually bites at work
- Hiring: ask for 'a job ad for a rockstar engineer' and the defaults skew young and male-coded ('dominant', 'crush it'). Ask it to screen résumés and it may echo historical patterns about names, schools, and career gaps.
- Performance language: studies of human reviews show women described as 'supportive' and men as 'strategic' for the same behavior — the model learned from exactly such text and will reproduce the pattern in drafts.
- Customer-facing tone: drafts may unconsciously adjust formality or warmth based on names in the thread.
- 'Typical customer' brainstorms: ask it to describe your typical buyer and it returns a stereotype composite — fine as one input, dangerous as 'data'.
Countermeasures that fit in a prompt
- 1Strip identity signals when they're irrelevant. Summarizing candidates? Remove names, photos, graduation years first; evaluate the anonymized version.
- 2Run the swap test. Suspicious about a draft? Swap the name or gender and regenerate. If tone or substance shifts, you've found bias — fix the process, not just the draft.
- 3Ask for the audit: "Review this job ad for language that could discourage any group from applying — gendered coding, age signals, unnecessary requirements. List issues and rewrite."
- 4Never let the model rank people. Extract facts per person if you must (green zone), but ordering humans is a human decision with legal weight in most jurisdictions.
Review the following draft for biased or exclusionary language — gender-coded words, age signals, culture-bound idioms, and assumptions about the reader. For each issue: quote it, explain who it could exclude and why, and offer a neutral rewrite. Then give the full corrected draft. Text: [paste]
The model is genuinely good at detecting bias when explicitly asked — the same training data that taught it the biases also taught it the critiques of them. The failure mode is only when nobody asks.
Hiring, lending, housing, promotion, discipline: in many jurisdictions, using automated systems in these decisions triggers specific legal duties (e.g., NYC Local Law 144 requires bias audits of hiring tools). If AI touches these at your company, legal review isn't optional. That's a big piece of the AI Governance, Risk & Compliance course.