# AWS DataSync

> An AWS service that copies large amounts of files over the network, and checks they arrived intact.

- Level: Intermediate
- Topics: AWS, Cloud
- URL: https://tidbit.legios.com.ar/aws-datasync/
- In Spanish: [AWS DataSync](https://tidbit.legios.com.ar/es/aws-datasync/)

**AWS DataSync** moves files and objects from one storage system to another over the network. The typical case is a company with years of documents, images or research data on a file server in its own office, which it wants in Amazon S3, Amazon EFS or Amazon FSx. It also works between AWS services and from other clouds, like Azure Blob Storage or Google Cloud Storage.

For on-premises storage you install a small virtual machine, the **DataSync agent**, near your file server. It reads from the usual protocols, such as NFS, SMB, HDFS or object storage, and sends the data to AWS. You define a task with a source, a destination and options, then run it once or on a schedule. Each run only copies what changed, encrypts data in transit and verifies that what landed matches what left.

You could copy files yourself with scripts, but at millions of files the hard parts are speed, retries, permissions and proving nothing got corrupted. DataSync uses many parallel streams to make the most of your link, and it handles those details. When the link is too slow for the amount of data, a physical transfer is the alternative.

## An analogy

A moving company that packs your office box by box, drives back and forth as many times as needed, and checks each box against the inventory list when it reaches the new place.

## Keep in mind

- Copies files and objects into, out of and between AWS storage services.
- Reads NFS, SMB, HDFS and object storage, plus some other clouds.
- Copies only changes on repeat runs and verifies data integrity.
- Uses an agent installed near your on-premises storage.

## Go deeper

- [What is AWS DataSync? (official docs)](https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html)
- [AWS DataSync product page](https://aws.amazon.com/datasync/)

## Related terms

- [Server](https://tidbit.legios.com.ar/server/): A computer whose job is to answer requests from other computers, all day, every day.
- [Amazon S3](https://tidbit.legios.com.ar/amazon-s3/): A giant online drawer for files, rented by the gigabyte.
- [Amazon EFS](https://tidbit.legios.com.ar/amazon-efs/): A shared network folder in AWS that many servers can use at the same time.
- [Amazon](https://tidbit.legios.com.ar/amazon/): An online store with its own delivery network, and the company that runs AWS.
- [Microsoft](https://tidbit.legios.com.ar/microsoft/): The company of Windows and Office, now just as much a cloud company with Azure.
- [Object storage](https://tidbit.legios.com.ar/object-storage/): Files stored as whole named objects, reached over the web, almost without limit.
- [Google Cloud](https://tidbit.legios.com.ar/google-cloud/): Google's cloud business: computing, storage, data and AI services rented by the hour.
- [Hybrid and multi-cloud](https://tidbit.legios.com.ar/hybrid-and-multi-cloud/): Running systems across your own servers and the cloud, or across several cloud providers.