Amazon Rekognition

Amazon Rekognition is computer vision as an API. You send it an image or a video stored in S3, and it answers with what it found: objects and scenes ("dog", "beach", "car"), text written in the picture, faces and their position, and whether the content looks unsafe. Each answer comes with a confidence score, like "Person, 99%", and often a box marking where in the image it is.

Common uses are practical and unglamorous: tagging a photo library so it can be searched, flagging inappropriate images users upload before anyone sees them, reading text from signs or documents in photos, or checking that a selfie matches the photo on an ID during sign-up. If the built-in labels are not enough, Custom Labels lets you train it with your own examples, like telling apart your company's products on a shelf.

Face analysis is the sensitive part. Studies found that facial recognition systems, Rekognition among them, made more mistakes with some groups of people than others, and in 2020 Amazon paused police use of its face comparison feature. Using faces in any product raises questions of consent, privacy laws and bias, so the confidence score is something to think about, not just a number to accept.