# AI bias

> When an AI system treats some groups of people unfairly, usually learned from its data.

- Level: Basic
- Topics: AI
- URL: https://tidbit.legios.com.ar/ai-bias/
- In Spanish: [Sesgo en la IA](https://tidbit.legios.com.ar/es/ai-bias/)

**AI bias** is when an AI system produces results that are systematically unfair to certain groups of people: by gender, skin color, age, accent, disability, neighborhood. Usually nobody programmed it on purpose. A model learns from examples, and if the examples reflect an unequal past, the model learns that inequality as if it were a rule.

Real cases made this concrete. In 2018 Reuters reported that Amazon had dropped an experimental hiring tool after finding it downgraded résumés that included the word 'women's', because it had learned from a decade of mostly male hires. The same year, the Gender Shades study found that commercial face-analysis systems made far more errors on darker-skinned women than on lighter-skinned men, because their training photos were skewed. Language models can do it too, for instance by assuming a doctor is a man and a nurse is a woman when translating.

Bias can enter at many points: data that underrepresents some people, labels that carry human prejudice, a goal that measures the wrong thing, or a system used on a population it was never tested on. It matters most where AI touches decisions about people: hiring, loans, insurance, healthcare, policing. Reducing it takes deliberate work: more representative data, testing results separately for each group, independent audits, and keeping a human able to question and override the system. Laws such as the EU AI Act now require this kind of checking for high-risk uses.

## An analogy

Like a new employee trained only by reading the company's old decisions. If those decisions favored one kind of person, the new employee will copy the pattern, sincerely believing it is just how the job is done.

## Keep in mind

- Usually learned from data that reflects past unfairness.
- It can hide in data, labels, goals or where the system is used.
- It matters most in hiring, credit, health and policing.
- Testing results per group and human oversight help reduce it.

## Go deeper

- [Algorithmic bias on Wikipedia](https://en.wikipedia.org/wiki/Algorithmic_bias)
- [What is AI bias? (IBM)](https://www.ibm.com/think/topics/ai-bias)
- [Identifying and managing bias in AI (NIST)](https://www.nist.gov/publications/towards-standard-identifying-and-managing-bias-artificial-intelligence)

## Related terms

- [Artificial intelligence](https://tidbit.legios.com.ar/artificial-intelligence/): Getting computers to do things that, until recently, only people could do.
- [Amazon](https://tidbit.legios.com.ar/amazon/): An online store with its own delivery network, and the company that runs AWS.
- [Training and inference](https://tidbit.legios.com.ar/training-and-inference/): Training builds the model once; inference is using it, millions of times a day.