Data mining
Digging through large amounts of data to find patterns nobody wrote down.
Data mining means searching large datasets for patterns that are useful but not obvious: which products are bought together, which customers behave alike, which transactions look strange. Nobody asks a precise question up front, like "how much did we sell in May"; the point is to let the data show something you did not know to ask.
A few kinds of results come up again and again. Association rules find things that happen together, like a supermarket noticing that people who buy pasta often buy grated cheese in the same trip. Clustering groups similar customers or patients without being told the groups in advance. Classification learns to sort new cases, for example a bank flagging card payments that look like past fraud. Anomaly detection spots records that do not fit any pattern.
The term became popular in the 1990s, and today most of the techniques live under machine learning and data science. The cautions are the same as ever. If you search enough data for enough patterns, some will appear by pure chance, so findings need to be checked on fresh data. And mining personal data raises privacy questions that laws like GDPR try to answer.
Keep in mind
- Finds patterns in large datasets without a precise question up front.
- Common results: things that go together, groups, predictions, anomalies.
- Overlaps heavily with machine learning and data science today.
- Patterns found by chance must be checked on new data.
