How I learned Pytorch
About me
I am a Deep Learning Engineer and Researcher who has primarily worked with Tensorflow and/or Keras for most of my projects until a few months ago. I have also been using Python extensively for a few years.
Why I wanted to learn Pytorch
I had always tried to improve my knowledge of Tensorflow when I had to implement tricky papers rather try out other frameworks. But recently I noticed that more and more papers had open sourced implementations available in Pytorch but not Tensorflow especially directly from the authors so I decided to try learning Pytorch. Also I had heard how much more Pythonic Pytorch is compared to Tensorflow.
How I approached learning Pytorch
My approach to learning Pytorch was the same as most people with any Deep Learning framework. That is, try to implement something very straightforward like an MNIST Classifier. But at the same time while implementing this I was trying to learn things that would come in handy for harder problems. Let us consider the steps to training an MNIST Classifier. First would be to get your data. The most straightforward approach would be to just load all your data as numpy arrays. But this doesn't teach us much. Especially because harder problems will have a lot more data which can't all fit into RAM. Also usually we would like to augment the data. So as I tried to figure ways to do that I learnt about datasets in Pytorch and how they can have multiple workers and can be made very efficient. Next was to define the model. This was actually the most intuitive part. Pytorch requires models to be a class inherited from torch.nn.Module and to have a forward function which does the forward pass. The backward pass is taken care of by Pytorch. Next is the training. Playing around with training was a big problem for me especially in Keras. Keras requires you to use callbacks to do custom operations during training. But Pytorch gives you all the control. Every training step is done manually but at the same time using very little code. It gives you total control to play around with things such as the loss, learning rate etc.
Challenges I faced
The main challenge in my opinion with learning Pytorch is the poor documentation. Luckily the Pytorch forum helps a lot. Pytorch has a lot of functionality for almost any problem you have to solve but finding the functions means digging through the documentation which feels unorganised. Otherwise it is just a matter for finding the right classes to inherit or directly import and you're good to go.
Key takeaways
Take your time and go through the documentation any time you're stuck. Also due to the popularity of Pytorch, if you are ever stuck implementing something you will probably find it on Github already implemented.
Tips and advice
If you are familiar with Python and like structuring your code into classes and using objects then you'll love Pytorch. It comes with a lot of functionality while also being easy to extend. Also if in doubts head to the Pytorch forums.
Final thoughts and next steps
That's about it. Next step for you would be to find a paper you want to implement and try it out on Pytorch. Your time might be spent more on going through documentation than writing actual code but you'll remember the things you read for your next implementation.
