Amazon SQS
A waiting line for messages between programs, so none of them gets lost.
Amazon SQS (Simple Queue Service) is a managed message queue. One program (the producer) drops messages into the queue; another (the consumer) picks them up when it is ready. The two never talk directly and do not even need to be running at the same time.
It solves a common problem: one part of a system is faster than another. An online shop takes 5,000 orders in a minute during a sale, but the service that prints shipping labels handles 50 per minute. With a queue in the middle, orders wait in line instead of failing, and the label printer works through them at its own pace.
SQS keeps each message until a consumer deletes it. If the consumer crashes halfway, the message becomes visible again after a timeout and another consumer retries it. Messages that fail too many times are moved to a dead-letter queue for a human to inspect. It pairs naturally with AWS Lambda: a function can be triggered for every batch of messages that arrives.
There are two kinds: standard queues (very high throughput, order mostly preserved, a message may rarely be delivered twice) and FIFO queues (strict order, no duplicates, lower throughput).
Keep in mind
- Producer and consumer are decoupled: neither needs to know about the other or be online at the same time.
- A message stays in the queue until it is explicitly deleted, so a crashed consumer does not lose it.
- Standard queues favor throughput; FIFO queues guarantee order and no duplicates.
- Messages can wait in a queue for up to 14 days (4 by default).
