How We Used IoT to Improve Our E‑Commerce Experience
I admit it, the title is kind of clickbait-y.
I’m going to tell you how we used AWS IoT service to improve our e-commerce experience.
Before, I wrote about a nasty bug we had to hunt down before releasing our new service.
Our storefront with Christmas decorations
We’re currently working on a new e-commerce solution for Grano. We aim to provide a better customer experience by modernizing our platform and improving automation and integration to our production services. Our tech stack is reasonably cutting edge, considering that we’re working with an enterprise-level product and environment: Node.js, Angular, Redux, and AWS accompanied by high code quality and modern development standards.
A little bit of background
Grano does mainly print, and the webstore is no exception. We sell anything from business cards to big roll-ups for exhibitions and trade shows. The one thing in common with almost all print products is that you need something to print.
In our store, the customer uploads a PDF file as the print material. A simple file upload to S3, you might think. We want to do more, though. In order to reduce manual work in the order flow, we want to check the PDF for compatibility with the selected product before allowing the customer to place the order.
Waiting for the PDF report…
This checking or “preflight” step is done using third party software called Enfocus PitStop Server ©. It runs on a Windows instance — that’s as good a reason as any to also run a virus check on the uploaded PDF files before allowing further file processing and downloading. This is achieved by using an AWS Lambda function to check the file and move it to the right S3 bucket.
How it looks if you uploaded a file with the wrong size. (Localization improvement is on the backlog.)
What does IoT have to do with all this?
All of that checking and processing takes time. PitStop is known to take minutes if checking a complicated PDF. We needed a way to tell the customer when the process has been completed and the checkout can be finished without telling them to refresh or use polling. I mean, come on, it’s 2017.
Our first idea for the problem used a nifty little library called Gun.js. It’s basically a Firebase real-time database, but you can integrate it easily to your existing Node.js application and host it yourself. It worked during our tests.
However, we had a couple of problems with it.
First of all, we obviously expect our project to be a great success and, therefore, we want to design it with high availability, performance, and fault tolerance. To achieve this, we use the AWS Application Load Balancer to start up API instances when the load increases.
This is a problem because Gun.js keeps the database in-memory using an optional storage adapter, such as S3, but to keep the real-time part of it working, we would need to make the API instances peer dynamically when instances are created or terminated.
Secondly, we didn’t find a smart way to limit read/write access to the Gun.js database. Anyone could modify PDF reports and the changes would be updated to any connected clients. On top of that, we took a look at the source code and — albeit well tested — it made us feel uncomfortable.
The solution
We set out to find a better solution that we could easily deploy to AWS and that wouldn’t increase maintenance work. With some searching online, we ended up reading about AWS IoT.
AWS IoT is a managed cloud platform that lets connected devices easily and securely interact with cloud applications and other devices. AWS IoT can support billions of devices and trillions of messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely. With AWS IoT, your applications can keep track of and communicate with all your devices, all the time, even when they aren’t connected.
We found out that AWS IoT provides a nice JavaScript SDK that supports real-time pub-sub messaging via WebSockets. The cost is $5 per a million messages — not bad for a hosted solution. For our needs, we only need to send one message per file upload.
The required code changes to the backend were quite minimal.
We use the generated file key, which consists of a random uuid, plus the filename, to scope the report messages. That should be sufficiently secure. In any case, the reports don’t include any personally identifiable information.
The changes to the front-end code were a bit more involved, since we now had to fetch the report status from the API and subscribe to the IoT messages. This is because the customer could close/reload the tab and later return to finish the checkout, only to have missed the IoT message.
We made a nifty service that takes a file key and returns an observable that simply gives you the initial status and any updates from the IoT service that we link to our Redux store to update the view.
The biggest problem we had was with authorization in AWS. We had to somehow link AWS Cognito and our API with our production and development AWS accounts so that our API instances from both environments could publish messages — and our clients from both environments could subscribe to those messages. But after some policy wrangling, we got it working and the messages were flowing.
The final file upload flow
The Takeaway
The lesson of the story is that there is always a better solution — you just have to find it. You also have to avoid getting attached to your code, always thinking about how to improve it. To summarize:
- Don’t be afraid to throw away code, even if it works!
- Prototype ALL THE THINGS
- The solution might seem strange at first — IoT, seriously?
Ville works as a Lead Developer at Grano, the most versatile content service provider in Finland and the leading graphic industry company in the Nordics, which is currently hiring. Currently, he is working on Grano’s E-Commerce solutions.
