How to re-initialize Keras model weights

Published Dec 18, 2017
How to re-initialize Keras model weights

There are multiple ways one can re-initialize keras weights, and which solution one chooses purely depends on the use case. I will be listing two such methods:

Saving weights to a file:

model.save_weights('my_model_weights.h5')
model.load_weights('my_model_weights.h5')

Code from: Keras FAQs page

This method works well when one needs to keep the starting state of the model the same, though this comes up with an overhead of maintaining the saved weights file.

Retriggering the initializer

def reset_weights(model):
    session = K.get_session()
    for layer in model.layers: 
        if hasattr(layer, 'kernel_initializer'):
            layer.kernel.initializer.run(session=session)

Note: this has been tested only for Convolutional and Dense layers

This method is useful when one just needs re-initialize the model weights, which could lead to a different starting point, but removes the overhead of maintaining a file to save model weights.

I was working on one of my projects and was dealing with many models and many minor iterations. In that situation, maintaining a file to save weights seemed taxing, and I was willing to have the trade-off of different starting states of the model.

I was unable to find a direct solution for this situation and had to go through a little documentation to come up with this.

Hope this helps.

Discover and read more posts from Nitin Surya
get started