How to schedule tasks in Node.js/Express using Agenda

Published Mar 05, 2018Last updated May 26, 2018
How to schedule tasks in Node.js/Express using Agenda

If you've ever wanted to schedule tasks in your Node.js application and couldn't find a straightforwarded way to accomplish this, then you will find this article very useful.

I came across Agenda.js while working on a personal project (a rideshare app). I was looking to automatically update rides when the departure date arrives, send reminders to passengers a few hours before their ride, prompt them to leave a review after the ride, and much more.

I didn't want to use the setInterval function because I wanted to be able to update (or delete) these tasks (jobs) should, for example, a user cancel a booking. Maybe there is a way with setInterval but I really liked how this package solve this problem.

My project structure:

bin/
  - www
controllers/
  ride/
    	- postRideController.js
jobs/
  - agenda.js
    jobs/
    	- archive-ride.js
        - welcome-email.js
routes/
  - ride.js
- app.js
- .env

To start, you need to download the agenda.js library.

npm install --save agenda

I keep my environment variables in a dotfile at the root of my project called .env. You will need something like that in production, so download it now.

npm install --save dotenv

You need to import it early in your app.js to make sure you can access those environmment variable everywhere in your app, like this.

app.js

require('dotenv').config();

Now you need to set up agenda.js, located in the jobs/directory.

jobs/agenda.js

const Agenda = require('agenda');

const mongoConnectionString = 'mongodb://localhost:27017/yourdatabase';

// or override the default collection name:
let agenda = new Agenda({db: {address: mongoConnectionString, collection: 'jobs'}});

let jobTypes = process.env.JOB_TYPES ? process.env.JOB_TYPES.split(',') : [];

jobTypes.forEach(function(type) {
  require('./jobs/' + type)(agenda);
});

if(jobTypes.length) {
  agenda.on('ready', function() {
    agenda.start();
  });
}

function graceful() {
    agenda.stop(function() {
      process.exit(0);
    });
}
  
process.on('SIGTERM', graceful);
process.on('SIGINT' , graceful);

module.exports = agenda;

Here, I point Agenda to my database so it can save tasks in a collection called 'jobs'. The variable jobTypes hold an array of all of the jobs you define, which are saved in your .env file, like so:

JOB_TYPES='welcome-email,archive-ride'

Basically, process.env accesses your .env file without any imports and looks for JOB_TYPES, thus process.env.JOB_TYPES.

Make sure to point to right folder here.

jobTypes.forEach(function(type) {
  require('./jobs/' + type)(agenda);
});

./jobs/ here is a second jobs folder inside the root jobs folder. The rest is basically copy paste from the agenda documentation.

Now, let's define our jobs.

In agenda, you define jobs. Then you can schedule or process them however you want throughout your app.

Let's look at the archive-ride job.

jobs/jobs/archive-ride.js


const MongoClient = require('mongodb').MongoClient;

module.exports = function(agenda) {
    agenda.define('archive ride', function(job, done) {

        // Connect to the db
        MongoClient.connect("mongodb://localhost:27017/yourdatabase", function(err, db) {
            if(!err) {
                db.collection('rides').findOneAndUpdate({_id: job.attrs.data.rideId},
                    { $set: { status: 'expired' }}, function (err) {
                    if(!err) {
                        done();
                    }
                })
            }
            if(err) {
                console.log(err);
                done();
            }
        });
    });
};

Here, I use the Node.js Driver for MongoDB (not Mongoose the ORM). If you are using Mongoose, then you simply need to import your models in this file and do your CRUD operations.

So first, we define our task 'archive ride'.

agenda.define('archive ride', function(job, done) {
  // your code goes here
  })

Next, I create a connection to my database.

MongoClient.connect("mongodb://localhost:27017/yourdatabase", function(err, db) {
    if(!err) {
        // If no error then I query and update my database
    }
})

Again, if you are using an ORM (object relational mapper) like Mongoose, then you don't need to create this connection.

db.collection('rides').findOneAndUpdate({_id: job.attrs.data.rideId},
                    { $set: { status: 'expired' }}, function (err) {
                    if(!err) {
                        done();
                    }
                })

That's it for defining the job.

Now we need to schedule our job when a user creates a ride and this is done in our controller.

controllers/ride/postRideController.js

let agenda = require('../../jobs/agenda'); // Not the agenda library itself but our configuration file jobs/agenda.js

module.exports = {
  post_ride: async (req, res) => {
    	try {
        	let myRide = {
            	departureDate: new Date(req.body.departureDate),
                rideData: req.body.myData
            }  // submitted throug a form or AJAX
            
            let db = req.db // I created a middleware in app.js to make my database connection accessible in all my routes
            
            await db.collection('rides').insertOne({rideData: myRideData})
            .then( (ride) => {
            	agenda.schedule(new Date(req.body.departureDate), // accepts Date or string
                	'archive ride', // the name of the task as defined in archive-ride.js
                    { rideId: ride.insertedId }, // passing in additional data accessible with: job.attrs.data.rideId
                    ); // You can pass an obtional callback to execute after the 
                    //job is saved on the database
            } )
            
        } catch (err) {
        	// Do something when an error occurs
        }
    }
}

There are many way to schedule and excedute jobs with agenda. Here I use agenda.schedule(); to schedule a job to be executed in the future date I specified. Alternatively you can just use agenda.now(); to excecute the job immediately. Go the documentation for other methods.

If you are curious about the database connection middleware, I simply added this npm package express-mongo-db. It's actually just one small file. You can open it and understand what's going on there easily.

Here is how it's set up in my app.js:

app.js

// mongoDB Connection URL
const url = 'mongodb://localhost:27017/yourdatabase';

// MongoDB connection middleware
const expressMongoDb = require('express-mongo-db');
app.use(expressMongoDb(url));

Now, in my route file, I export my function 'post_ride' into my route like this:
routes/ride.js

const express = require('express');
const router = express.Router();
const rideController = require('../controllers/ride/postRideController.js

router.post('/post-ride', rideController.post_ride)

module.exports = router;

To conclude, make sure you have your jobs well defined in your .env file and you should be good to go. If you have questions or suggestions, please leave a comment below.

Discover and read more posts from Miguel Kouam
get started