# Data mining

> Digging through large amounts of data to find patterns nobody wrote down.

- Level: Intermediate
- Topics: Data
- URL: https://tidbit.legios.com.ar/data-mining/
- In Spanish: [Minería de datos](https://tidbit.legios.com.ar/es/data-mining/)

**Data mining** means searching large datasets for patterns that are useful but not obvious: which products are bought together, which customers behave alike, which transactions look strange. Nobody asks a precise question up front, like "how much did we sell in May"; the point is to let the data show something you did not know to ask.

A few kinds of results come up again and again. Association rules find things that happen together, like a supermarket noticing that people who buy pasta often buy grated cheese in the same trip. Clustering groups similar customers or patients without being told the groups in advance. Classification learns to sort new cases, for example a bank flagging card payments that look like past fraud. Anomaly detection spots records that do not fit any pattern.

The term became popular in the 1990s, and today most of the techniques live under machine learning and data science. The cautions are the same as ever. If you search enough data for enough patterns, some will appear by pure chance, so findings need to be checked on fresh data. And mining personal data raises privacy questions that laws like GDPR try to answer.

## An analogy

Like panning for gold in a river: you shovel through huge amounts of sand and gravel, and most of it is worthless, but the few shiny bits you keep are what you came for.

## Keep in mind

- Finds patterns in large datasets without a precise question up front.
- Common results: things that go together, groups, predictions, anomalies.
- Overlaps heavily with machine learning and data science today.
- Patterns found by chance must be checked on new data.

## Go deeper

- [Data mining on Wikipedia](https://en.wikipedia.org/wiki/Data_mining)
- [What is data mining? (IBM)](https://www.ibm.com/think/topics/data-mining)

## Related terms

- [Machine learning](https://tidbit.legios.com.ar/machine-learning/): Teaching computers by showing them examples instead of writing every rule.
- [Data science](https://tidbit.legios.com.ar/data-science/): Using statistics, code and business sense to answer questions with data.
- [Personal data and GDPR](https://tidbit.legios.com.ar/personal-data-and-gdpr/): Information about you, and the laws that say who can use it and how.