How to re-initialize Keras model weights
There are multiple ways one can re-initialize keras weights, and which solution one chooses purely depends on the use case. I will be listing two such methods:
Saving weights to a file:
model.save_weights('my_model_weights.h5')
model.load_weights('my_model_weights.h5')
Code from: Keras FAQs page
This method works well when one needs to keep the starting state of the model the same, though this comes up with an overhead of maintaining the saved weights file.
Retriggering the initializer
def reset_weights(model):
session = K.get_session()
for layer in model.layers:
if hasattr(layer, 'kernel_initializer'):
layer.kernel.initializer.run(session=session)
Note: this has been tested only for Convolutional and Dense layers
This method is useful when one just needs re-initialize the model weights, which could lead to a different starting point, but removes the overhead of maintaining a file to save model weights.
I was working on one of my projects and was dealing with many models and many minor iterations. In that situation, maintaining a file to save weights seemed taxing, and I was willing to have the trade-off of different starting states of the model.
I was unable to find a direct solution for this situation and had to go through a little documentation to come up with this.
Hope this helps.