Keras training with EarlyStopping callback to avoid overfitting

Hello, my name is Alex.

I'm learning machine learning.
And when I finish some task, I like to publish it.
So that people could find easy samples to get started with.

You can run google colab or use your computer.

Overview

In this tutorial we will cover

1. How to split data into test and train
2. How to compile neural network with keras
3. How to draw live chart of accuracy of neural network

1. Download data from kaggle.

There will be 2 files

• train.csv.zip
• test.csv.zip
  I've no idea why, but test file doesn't make any sense,
  since there are no lables there.

https://www.kaggle.com/oddrationale/mnist-in-csv

2. If you using google colab

Drag'n'Drop train.csv.zip file to files

3. Unzip file

!unzip train.csv.zip

3. Import other stuff we need

!pip install livelossplot

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
plt.style.use('ggplot')
import keras
from keras.callbacks import EarlyStopping


from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from livelossplot import PlotLossesKeras

4. Cook the data

train_df = pd.read_csv('/content/train.csv') #might be in other place
train_labels = train_df['label'] #We need Y values - labels
train_labels = train_labels.to_numpy() # nothing smart just convert to numpy array
del train_df['label'] # remove label from original dataframe to use it as X
train_data = train_df.to_numpy()


# we can't use values 1,2,3,4,4,5 for Y
# instead we should use smth like [1,0,0,0,0,0], [0,1,0,0,0,0], ...
y = LabelBinarizer().fit_transform(train_labels) 


#Split train and test data
X_train, X_test, y_train, y_test = train_test_split(train_data, y, test_size=0.1)


# Ok, ready to go

5. Compile neural network

# Define sequential model

model = keras.Sequential()

# Define the first layer
model.add(keras.layers.Dense(128, activation="relu", input_shape=(784,)))
model.add(keras.layers.Dense(128, activation="relu", input_shape=(128,)))
model.add(keras.layers.Dense(128, activation="relu", input_shape=(128,)))
model.add(keras.layers.Dense(128, activation="relu", input_shape=(128,)))

# Add activation function to classifier
model.add(keras.layers.Dense(10, activation='softmax'))

# Finish the modecl compilation
model.compile('adam', loss='categorical_crossentropy', metrics=['accuracy'])

# patience - how many epochs to wait before stop training
# if there is no further improvement 
monitor_val_acc = EarlyStopping(monitor = 'val_loss', patience = 5)

# Complete the model fit operation

# callbacks=[PlotLossesKeras()] - this is a single magic line of code which draw #live chart

#Also we set up big epochs size, just to test that easly stopping is working
model.fit(train_data, y, epochs=10000000, validation_data=(X_test, y_test), callbacks=[PlotLossesKeras(), monitor_val_acc], verbose=0)

Awesome, you did it!

Now you can train neural network to detect pokemons!

Follow me in twitter
@alexpolymath