[JS] Beginner’s Guide: securing your GitHub App webhook endpoint

A beginner-friendly tutorial on webhooks, HMAC signatures, and middleware. Learn how to secure your server by writing Express middleware that verifies the authenticity of the event payloads GitHub sends.

GitHub Apps let you set up a webhook to listen to all sorts of events in organizations and repos. These can range from new or updated issues to PR comments and deployments. Using the GitHub API your app becomes capable of reacting to these events. The possibilities are really endless for anyone with a love for automation.

But with great power comes great responsibility! You wouldn’t want to automate actions from untrusted or malicious payloads.

Luckily GitHub provides an optional ( but seriously recommended ) webhook secret field on your app’s settings page. Using this secret and a little wizardry we can verify that the payloads are legitimate and direct from GitHub before taking any action.

What is a webhook?

A webhook is an endpoint that one server uses to relay information to another server. In simple terms it’s a way of letting third parties be notified of an event or action. Services can control their own internal reactions to events while also letting third parties receive and process the same events at their discretion.

Say we have two servers — the event server and the listening server. In the context of GitHub Apps the event server is GitHub and the listening server is your app’s server.

Listening servers can register with event servers to hook into events that occur. They configure what events they want to be notified for and what endpoint they want the notifications (request) to be sent to. This endpoint is known as the listening server’s webhook endpoint.

A common way of securing webhook endpoints is to use a webhook secret. This secret combined with some cryptography provides a mechanism of trust between the event and listening servers.

How the webhook secret works

When you provide a webhook secret to GitHub they use it to create an HMACsignature. HMAC signatures are a way to sign a payload using a secret that only the event and listening servers know.

This signature is attached as a header with the event requests sent to your app’s server. GitHub in particular uses HMAC-SHA1 but the hashing algorithm is independent of how HMACs work.

An HMAC signature does not encrypt the payload. It is a verifiable signature hash made from a secret [our webhook secret] and plain-text data [GitHub event payload]. You can encrypt the data as well but it is an optional step.

The signature is used to verify the legitimacy of both the source and data itself. When the request is received by the listening server it uses its stored secret and the event payload to generate its own HMAC-SHA1 signature to compare with the header signature.

When the secret and the payload are the same on both sides then the HMAC signatures will match. This match proves the authenticity of the request and data.

What about data tampering?

Because the signature is created by GitHub using our app’s secret and their event payload we can trust that both are authentic when they leave their end in the request.

But what about over the wire if the message is intercepted and changed? How does an HMAC signature help us here?

Remember that both GitHub and our own app are generating signatures from the same secret and payload. As long as the secret is never leaked then the signature generated can only change if the payload changes.

The attacker would only have the signature from the header and the original payload. They would have no [reasonable] way of deconstructing our secret from the two —  as long as you use a strong secret.

Without the secret they’d have no way of falsifying a new signature to pair with the altered data. Any attempted tampering of the payload would result in a different generated signature which would fail the comparison against the header signature.

People are so damn smart. On to the build!

Setup

Go to your GitHub app’s page and scroll down to the webhooks section.

You can find your app page under [user or org] settings → GitHub Apps

The webhook url

If you are still in development you can use a tool like ngrok to expose the local port your app is running on. It lets the outer web connect to your local server through a secure tunnel. We will use this to test out the endpoints for our app.

• npm i ngrok --save-dev
• start your app server on your chosen port
• ngrok http <Your App's Local Port>
• then copy the https link that it generates

Now just add that ngrok url (or live url if you are deployed already) followed by your webhook path to your app’s settings.

Make sure to turn off ngrok when you are done using it. Don’t leave an exposed port even if its exposed through a randomly generated url

Shhhhh

Next let’s generate a secret. I chose to generate a random hash but feel free to use your own approach. Just be sure to use something that is actually secure — no plain text dog names!

Because the GitHub docs are in ruby and it’s all new information to me I went down a standard issue black hole of research. Luckily, unlike most of my 50+ wiki tabs and wtf even am I anymore results, this time I struck gold.

User663183 explainswhy using randomBytes() is more secure than using a timestamp or random number generator. Along with some other neat information — worth a read.

1. open up your terminal and the node shell
2. const crypto = require('crypto');
3. crypto.randomBytes(32).toString(‘hex’);
4. copy the output string

Now you will want to save this string in your app environment or some other safe location. Then paste it again as the secret on your GitHub app settings.

Middleware

Finally on to some real code! We’ll write two functions — a utility for generating a comparison signature and our webhook middleware function.

What is middleware?

If you aren’t familiar with middleware don’t worry! It is actually really simple and logical. Middleware refers to behavior that you want to execute on an incoming request. You can apply middleware at the server level (affects all incoming requests) or for specific routes.

Without middleware your http request-response process looks like this:

request → route handler processing → response

With middleware your process looks like:

request → middleware processing → route handler processing → response

So middleware just gets in the middle of the request before it hits your route handler. And you can combine or stack middleware to have multiple intermediary steps before your route handler sees the request.

Middleware can be used to intercept the request object and perform verification, logging, transformation, or injection to it. Then the middleware passes the request down the line, optionally through other middleware, to the request handler it was originally destined for.

You can do things like check for logged in users (from a cookie or auth header), exchange for a user, and inject it into the request object as req.user. Or something convenient like body-parser which just cleans up the incoming request data and puts it in a more friendly form on req.body.

The possibilities are endless. And our case is a perfect candidate for middleware. We’ll check the validity of all the event payloads before any processing is done by our app. Any invalid payloads we can reject early and prevent unwanted or dangerous behavior.

The code

Payload verifying middleware

[middleware] verifyGithubPayload()

This function presumes you are using body-parser middleware at the app (affecting all incoming requests) level. If you aren’t then you can install and configure it using their docs.

Our middleware function will do the following:

1. extract the headers and body [payload] from the request
2. extract the x-hub-signature header which GitHub uses to send their HMAC signature with the request
3. create our own internal signature for comparison using createComparisonSignature()
4. compares the signatures through compareSignatures()
5. return a 401 (unauthorized) status code if the signatures don’t match
6. inject an event_type property from the header into the request object. The available types can be found here
7. inject action and payload properties into the request to make things easier downstream (using the “separating the yolk” object destructuring and spread syntax)
8. call next() which passes the request to the next in line for the request (our route handler for/events)

[utility] createComparisonSignature()

createComparisonSignature() is a utility function that:

1. creates a SHA1 HMAC signature base using our secret (stored in a .env file)
2. calls update() to add the payload (body) to the signature base to form the complete signature
3. calls digest('hex') on the complete signature to turn it into a hex string for comparison
4. return sha1=SIGNATURE to match the shape GitHub uses in their header signature

[utility] compareSignatures()

At first glance this function may seem out of place. If the two signatures are both just strings why can’t we compare them through signature !== comparison_signature?

The reason is that string comparisons will return false as soon as a character mismatch occurs. This means that the time it takes to return a failed response is dependent on how many characters match before a mismatched one causes the expression to return false.

Attackers can use this information to perform a timing attack. They can change the first letter, then the second, and so on determining how close they are based on the response time.

Using the crypto.timingSafeEqual() method protects us against this attack vector. The method will run in constant time regardless of where the character mismatch occurs. Because of this the response time will always be consistent.

Applying the middleware

Back in app.js we add the middleware to the /event route

app.use(‘/events’, verifyGithubPayload, eventsHandler);

Here we are saying: for every request that goes to /events first pass the request through verifyGithubPayload then [next()] send it to the eventsHandler route handler.

const eventsHandler = (req, res) => {
  console.log(req.event_type, req.action, req.payload);
  return res.send('got authentic event data through my new middleware!');
};

module.exports = eventsHandler;

Notes

• great article on HMAC

Event types

I wrote a script to scrape the GitHub docs for .json and .txt files of the event types. You can use these to create a white list of accepted events. The JSON file has a description and url field if you need it for documentation.

You can find the scraper and outputs here.

Debugging

You can view your app’s event requests and responses under

settings → advanced → recent deliveries.

This page is useful for debugging and getting sample payloads for tests.

GitHub App event payload monitoring

— Vamp