Natural language processing
The field of AI that works with human language: reading, writing, translating.
Natural language processing (NLP) is the branch of AI that deals with human language, the 'natural' kind we speak and write, as opposed to programming languages. Its goal is to get computers to do useful things with text and speech: translate it, summarize it, answer questions about it, sort it, or turn it into a spoken reply.
Language is hard for machines because meaning depends on context. 'The movie was not bad' is mildly positive, even though it contains the word 'bad'. 'Bank' can be a river or a place for money. For decades, NLP relied on grammar rules written by linguists, then on statistics over large collections of text. Since the late 2010s it has been dominated by neural networks, especially the transformer, and today's large language models are its most visible product.
Classic NLP tasks are still everywhere, often hidden: spam filters, autocomplete on your phone, machine translation, voice assistants that turn speech into text, search engines that understand what you meant, systems that read thousands of customer reviews and tell a company which ones are angry. Many of these tasks that used to need a separate specialized model can now be done by one general language model with the right prompt.
Keep in mind
- Covers translation, summarizing, classifying, search and speech.
- Hard because the same words change meaning with context.
- Moved from hand-written rules to statistics to neural networks.
- Large language models are the current, most general form of NLP.
