Upload an Entire s3 Bucket/Local Directory to Another s3 Bucket From Node.js

I have come across several requests wherein a team needs to upload an entire folder/source s3 bucket (that may contain subfolders) to another s3 bucket (destination).

We are all aware of the nice little AWS CLI command [s3 sync] to complete the above task.

Bucket to bucket sync:
 s3 sync s3://srcBucket s3://destBucket 
Local folder to s3Bucket sync:
 s3 sync . s3://destBucket 

The problem is that when we would like to achieve the same result from a script rather than the command line, we find that none of the SDKs provided by AWS helps us achieve this.

Some suggest using AWS's data pipeline though I believe this is unnecessary when there is no transformation requirement.

So How do we go about this?

There is a node package — aws-cli-js.  You can read more about it here.

Following code helps us to make use s3 sync cli command from scripts.

var awsCli = require('aws-cli-js');
var Options = awsCli.Options;
var Aws = awsCli.Aws;
var options = new Options(
/* accessKey */ CRED.accesskey,
/* secretKey */ CRED.secretkey,
);
var aws = new Aws(options);
aws.command(' s3 sync s3://srcBkt s3://destBkt, function (err, data) {
console.log('data = ', JSON.stringify(data.raw));
});

Note:  Make sure to pass on the right credentials. We can use the callback method and process the results too.

I hope this helps. Let me know your thoughts and anything that can be improved!