Running Asynchronous JavaScript Code in Sequence with Async Waterfall

Published Sep 13, 2016Last updated Jan 18, 2017

This is originally posted by the author on this blog. This version has been edited for clarity and some parts may appear different from the original post.

Introduction

Async is a JavaScript library that allows you to control the flow of asynchronous JavaScript code. In this tutorial we are going to explore the Async.waterfall method to run asynchronous functions "in-order".

Project Files

Click here to download the project files.
Make sure to run npm install before running node main.js

Setup

First, let's set up the project:

mkdir -p ~/Desktop/async-example && cd $_ && touch main.js && npm init

Once you are prompted with the options, just accept the defaults. When the package.json file is created, install async:

npm i async -S

Then open the main.js file of your project and add the following to the file:

var async = require('async');

async.waterfall([
  function (done) {
    done(null, 'Value 1');
  },
  function (value1, done) {
    console.log(value1);
    done(null, 'Value 2');
  }, function (value2, done) {
    console.log(value2);
    done(null, 'done');
  }
], function (err) {
  if (err) throw new Error(err);
});

You can run this with node main.js. Once you run it, you should get the following output:

Value 1
Value 2

Example

Let's go over this simple code example:

The waterfall method takes two parameters:

async.waterfall([], function (err) {});

The first argument is an array and the second argument is a function.
Using the first argument (i.e. the array), you can specify what you want to run in order.
Using the second argument (i.e. the function), you can catch any errors that happens in any of the steps.

Now let's explore the first argument in more detail. To define the steps that you need to run, you need to create a function for each step. For example, if you need two steps, you need to create two functions and put them in the array:

async.waterfall([
  function firstStep() {},
  function secondStep() {}
],
function (err) {});

As you can see, we have defined two functions (they can be anonymous or named, it doesn't matter). The next important thing to know is the arguments passed to these "step" functions:

async.waterfall([
  function firstStep(done) {},
  function secondStep(previousResult, done) {}
],
function (err) {});

Every step function takes two arguments, except the first one. The first step function only takes one argument. Using the arguments, you can access the result of the previous step and also invoke the next step.

async.waterfall([
  function firstStep(done) {
    done(null, 'Value from step 1');
  },
  function secondStep(previousResult, done) {
    console.log(previousResult);
    done(null);
  }
],
function (err) {});

If you notice, we are calling the done function with two arguments: the first argument is any error that we want to pass to the next step, and the second argument is the actual result or value that we want to pass to the next step. As you can see, for now we have set error values to null, because for now, we don't really care about the errors. Hopefully, it should make more sense why the first step function takes one parameter. It's because nothing has been executed before the first function, so there are no results to be passed onto the first function.

Note: In the code snippet above, both async.waterfall and promise have the same speed. The point of the code above is that the sencondStep depends on previousResult

To complete the example, let's add another step function and print the result from the step two function:

var async =  require('async');
async.waterfall([
  function firstStep(done) {
    console.log('start!');

    done(null, 'Value from step 1'); // <- set value to passed to step 2
  },
  function secondStep(step1Result, done) {
    console.log(step1Result);

    done(null, 'Value from step 2'); // <- set value to passed to step 3
  },
  function thirdStep (step2Result, done) {
    console.log(step2Result);

    done(null); // <- no value set for the next step.
  }
],
function (err) {
  if (err) {
    throw new Error(err);
  } else {
    console.log('No error happened in any steps, operation done!');
  }
});

Once you run this (node main.js) you should get the following output:

start!
Value from step 1
Value from step 2
No error happened in any steps, operation done!

Wrapping up

In the case of async and promise, both can deal with the async nature of JavaScript (which might not be easy to do). However, in some cases, we'll have callbacks which depend on the previous one (which is the case mentioned in this article). In such cases, we have to invoke the second callback after the first one is finished. So either async.waterfall or promise can not make it faster as our second callback is based on the result of the first one.

So here, essentially, the async.waterfall is conceptually similar to

promise.then(...)

They have the same dependency nature but different code styles

That said, I hope you found this tutorial useful!