Speed Up Your MongoDB Tests in Node

You've got a couple of choices when it comes to backend tests that interact with mongo: mock the calls or query a live database. The former has the serious disadvantage of potential inaccuracy (e.g. there could be typos in your selectors). The latter can be impractically slow when running a large number of tests.

The best solution I've found is programmatically spin up an in-memory real mongodb instance for each group of tests, using the mongodb-memory-server package.

npm i -D mongodb-memory-server

Alternatively, if you want to share the same mongodb binary for all of your projects (this is what I prefer) you can run:

npm i -D mongodb-memory-server-global

If you read through the docs, you'll that see it works in numerous testing environments, and with exiting ODMs like mongoose. Additionally, it will download the necessary mongodb binary for you, so there's no external setup required.

In the remainder of this article, I'll explain how I use mongodb-memory-server along with the native mongodb driver. Finally, we'll wrap up with a simple Jest example.

Here's my implementation of a class called DBManager which encapsulates all of the DB setup and teardown functionality:

const { MongoClient } = require('mongodb');
const { MongoMemoryServer } = require('mongodb-memory-server-global');

// Extend the default timeout so MongoDB binaries can download
jest.setTimeout(60000);

// List all of your collection names here - I'll add some examples
const COLLECTIONS = ['users', 'groups', 'comments'];

class DBManager {
  constructor() {
    this.db = null;
    this.server = new MongoMemoryServer();
    this.connection = null;
  }

  // Spin up a new in-memory mongo instance
  async start() {
    const url = await this.server.getUri();
    this.connection = await MongoClient.connect(url, {
      useNewUrlParser: true,
      useUnifiedTopology: true,
    });
    this.db = this.connection.db(await this.server.getDbName());
  }

  // Close the connection and halt the mongo instance
  stop() {
    this.connection.close();
    return this.server.stop();
  }

  // Remove all documents from the entire database - useful between tests
  cleanup() {
    return Promise.all(COLLECTIONS.map((c) => this.db.collection(c).deleteMany({})));
  }
}

module.exports = DBManager;

And here's a simple Jest test suite using a DBManager instance:

const DBManager = require('./DBManager');

const GROUP = { _id: 'g1', createdAt: new Date() };

describe('Group.getGroupById', () => {
  const dbman = new DBManager();

  afterAll(() => dbman.stop());
  beforeAll(() => dbman.start());
  afterEach(() => dbman.cleanup());
  beforeEach(() => dbman.models.Group.collection.insertOne(GROUP));

  it('should return null with an invalid id', async () => {
    expect.assertions(1);
    const result = await dbman.models.Group.getGroupById('x');
    expect(result).toBeNull();
  });

  it('should return a group with a valid id', async () => {
    expect.assertions(1);
    const result = await dbman.models.Group.getGroupById(GROUP._id);
    expect(result).toEqual(GROUP);
  });
});

The details of the tests aren't that important, but note how we spin up a mongodb instance at the start of the test block with beforeAll, and then stop it at the end of the test block with afterAll. For each test in the block, we keep the same database running, but remove all of its contents with dbman.cleanup(). This ensures each test runs in a clean environment.

Note the above implementation requires that all available collections are listed in the COLLECTIONS array. A more sophisticated approach would be to query the database for all collections at the time of removal. There are packages that do precisely this.

At the time of this writing, I have 712 backend tests (98% of which require the database), and the total execution time is 64.7 seconds on my aging i5-2400.

Enjoy the speed increase!

Photo by Veri Ivanova on Unsplash