× {{alert.msg}} Never ask again
Receive New Tutorials

6 Useful Python Libraries Recommended by #1 Stack Overflow Answerer

– {{showDate(postTime)}}
Martijn Pieters, Stack Overflow Python Legend

Codementor Python expert and Stack Overflow legend Martijn Pieters sat down with us during Office Hours and shared some of the less-well known libraries that he likes to use for web development or daily tasks.

The text below is a summary done by the Codementor team and may vary from the original video and if you see any issues, please let us know!


Requests: HTTP for Humans

I am a huge fan of the requests library. If you have to do anything with http requests to other servers, this library goes straight past your urllib that comes with Python, and install requests. It is a fantastic tool and really has a much more intuitive API, which makes handling web requests much easier.

Beautiful Soup

Another great tool for web scraping is Beautiful Soup. When you’re web scraping and looking at web pages to extract information from, you should check this tool out. Beautiful Soup is a great html parsing library that makes and sends off the html “soup”.


I recently came across a new library that marries the aforementioned two tools—Robobrowser. It is based on the requests library and beautiful soup to replace an older tool called mechanize, which does similar things. Namely, the Robobrowser lets you simulate a browser that goes out to the web, fetches html pages, lets you fill in forms, and submits those. Therefore, if you have to crawl a web page that requires forms or follow links to get to the data you want to get to and extract, take a look at Robobrowser, which can be also found on GitHub. Though it’s a relatively young project, I like the way the API works and the tool it uses.


On the server side of web development, Autobahn has piqued my interest. This tool makes programming web sockets easy, which is very helpful to me as a web developer. It’s also interesting as it makes use of Python 3.4’s new library.

Flake 8 Lint

For daily development, I like to use flake8 lint, which is based on Flake8. This Sublime Text plugin combines Pep8, which is for the Python style guide testing, together with Pyflakes, which detects common errors in your code (e.g. imports you forgot to add or imports you have too many of). Flake8 makes writing code much easier for me, and it integrates quite nicely with modern IDEs. I use sublime text and I have a plugin that puts flake8 error messages right at my fingertips. This is not a new project, but it is certainly something I use every day.


I haven’t actually come across a project where I use Scrapy, but to my knowledge Scrapy is excellent at extracting patterns for specific data. For example, if you want to scrape the latest football scores of Britain’s major leagues, you’d have to go to multiple sites and pages to extract the same information. Scrapy is good for such a task, as it can automate searching and walk through a site to extract very structured data you can specify through expressions. I’d love to have a project for me to use Scrapy that but I haven’t had the chance yet.

Other posts in this series with Martijn Pieters:

Martijn Pieters

Need Martijn’s help? Book a 1-on-1 session!

or join us as an expert mentor!

Martijn Pieters
Martijn Pieters
I'm no longer available for mentoring, sorry!
Hire the Author

Questions about this tutorial?  Get Live 1:1 help from Python experts!
Jessamyn Smith
Jessamyn Smith
Experienced Full-Stack Web Developer
I am an experienced full-stack software developer with over a decade in the field, primarily in back-end development for web applications. My...
Hire this Expert
Ray Phan
Ray Phan
PhD, former university instructor and software engineer with 20 years of software development experience in MATLAB, Python, Java, C, and C++. Image Processing and Machine Learning are part of my all-balanced diet. Students get a 33% discount!
NOTE 1: Please note that if your language doesn't appear in my tags in my headline above, I probably don't know it! NOTE 2: I do not accept...
Hire this Expert

Or Become a Codementor!

Live 1:1 help from expert developers

Codementor is your live 1:1 expert mentor helping you in real time.

comments powered by Disqus
Codementor is your live 1:1 expert helping you in real time