How and why I built Smoke detection library

About me

I'm a Ph. D. candidate who has worked on computer vision problems since 2010

The problem I wanted to solve

As a participant in a government project, I had to create an algorithm to detect smoke captured by a surveillance camera. The algorithm should have been implemented using C++ and embedded into a solution with GUI build by another software developer.
One of the limitations was the limited processing power of a single CPU core while the goal was achieving at least five frames per second performance.

What is Smoke detection library?

The program I built could receive a sequence of frames from a camera and detect if smoke appeared on them.

Tech stack

I've used Microsoft Visual Studio to write and compile the C++ code.
OpenCV library was utilized to simplify I/O and standardize the data format to cooperate with other developers.

The process of building Smoke detection library

In the very beginning, I started by reading state-of-the-art research articles that explained different approaches to solve my problem.
I chose the algorithm that I considered fast enough for me.

I started making an algorithm and implementing it in Matlab. It worked very slowly, but the detection rate was acceptable. I also added my own filters, such as edge density, since smoke usually has different density, causing multiple edges inside itself.

The next step was reimplementing the algorithm in C++ and OpenCV. All of the data was stored in Mat type and vector<T> data structure.
After testing the algorithm, I could achieve five fps, but it started failing on new test sets. My handcrafted features and decision rules failed. I then added more data using Haar-like features and fed everything into AdaBoost hoping it can generalize the input well. AdaBoost achieved 90%+ accuracy, which was enough for the project.

Challenges I faced

The difficult part was communication with other developers who haven't used OOP before and preferred functional programming. I had to develop the whole structure of the solution with all of the classes given by dynamic libraries.

Key learnings

In the project, we haven't used a version control tool such as Git. All new versions of source code were stored in different folders, making the final collection of changes a nightmare.
I also improved my knowledge about STL.

Tips and advice

Use version control.
Write tests (TDD helps, especially when boundary conditions are checked).
Try to test code on multiple sets to avoid overfitting of your approach.

Final thoughts and next steps

That project was helpful in the understanding of teamwork and implementation of different algorithms.
I think I had enough of handcrafted features. Deep learning opened opportunities to improve overall accuracy when it is possible to get large datasets. It is my passion now.