Python Requests Tutorial | Python Tutorial

Python requests bag almost 400,000 downloads everyday. This number is evident enough to understand about the popularity of this python library. In recent years, python programming language has become the most desired programming language for many developers. The concepts and libraries like requests is one of the many reasons for developers to transition from other programming languages to python. In this blog, we will go through the following topics:

Python requests was written by Kenneth Reitz and licensed under apache 2.0. It is a human friendly HTTP library as it is mentioned on the official documentation page. It is easy to use and basically used for making all sorts of HTTP requests. Following are a few advanced features that requests comes with:

1. Keep alive and connection pooling
2. International domains and urls
3. Sessions with cookie persistence
4. Browser-style SSL verification
5. Automatic content decoding
6. Basic/digest authentication
7. Elegant key/value cookies
8. Automatic decompression
9. Unicode response bodies
10. HTTPs proxy support
11. Multipart file uploads
12. Streaming Downloads
13. Connection timeouts
14. Chunked requests

These are all the advanced features of python requests library, lets try to understand why do we use python requests in the first place.

Why Use Python Requests?

When it comes to why do we use python requests? the reason is pretty simple. While using python requests, you don’t have to manually add the queries to your urls and form-encode post data. It makes our job easier when making http requests of any kind.

Now that we are familiar with python requests and why we use them in python, lets try to understand how are we going to install requests on our project or system.

How To Install Python Requests?

The installation part is very easy as well. If you have the pipenv setup installed on your system, you can simply run the following command in the terminal.

$ pip install requests

This will install the requests library on your system. There is one more approach to install requests. If you are using pycharm, you can add requests

on the project interpreter in the settings. It serves the same purpose as the terminal in case of installing the library on our project.

Now that we are through with the installation, lets try to understand how we will make get and post requests in python.

How To Make Get & Post Requests?

Get request is basically used to request the data from the server. Following is the syntax to make a get request.

import requests

res = requests.get('url')

#res is the response object here.

Post request is used to submit the data to be processed to the server. Following is the syntax to make a post request.

import requests

payload = {'key1': 'value1'}

res = requests.post('url' , data= payload)

Now that we know how we can make get and post requests, lets take a look at how we can pass parameters to the url using the get request.

Passing parameters in a url is as simple as making a get request. Following is an example to pass parameters to url.

import requests

payload = {'key1': 'value1' , 'key2': 'value2'}

res = requests.get('url', params= payload)

print(res.url)

#this will print the url with the parameters passed through the get request.

We can check the status code as well, following is the code to check the status code:

import requests

res = requests.get('url')

print(res.status_code())

If the code returns 200, it means there is no error and the request is all well. If we make a bad request, the code will return code like 404 or 505 which will raise an http error.

Response Content

We can also read the contents of the server’s response. The library will automatically decode the content from the server.

import requests

res = requests.get('url')

print(res.content)

Requests also has a builtin json decoder.

import requests

res = requests.get('url')

print(res.json())

#this will get the response in a json format

Multi-Part File Upload

It is very easy to upload multi-part files using requests.

import requests

files = {'file' : open('filename', 'rb')}

res = requests.post('url' , files= files)

print(res.text)

For sending multiple files we will specify multiple files in the files parameter.

We can view the server’s response headers and cookies using the response object. Following is the code to view the server’s headers.

import requests

res = requests.get('url')

print(res.headers)

We can pass custom headers to the url as well. Lets take a look at the code.

import requests

headers = {'key1' : 'value1'}

res = requests.get('url', headers= headers)

print(res.headers)

Requests does not change its behavior based on custom headers. They are simply passed onto the final request. cookies  can also be viewed using the response object.

import requests

#to pass our own cookies we can use the cookies parameter

cookies = dict(cookies= 'working')

res = requests.get('url' , cookies= cookies)

print(res.text)

Cookies are returned in a RequestCookieJar, which acts like a dictionary but also offers more complete interface, suitable for use over multiple domains or paths.

Session Object

The session object allows you to persist certain parameters across the requests.

• Persists cookies across all requests made from the session instance
• Use urllib3 connection pooling
• Significant performance increse
• A session object has all the methods of the main requests API

Following is the code to persist some cookies across requests.

s = requests.session()

s.get('url')

res = s.get('url')

print(res.text)

Errors And Exceptions

Following are the errors and exceptions that are raised in a python request.

• In the event of the network problem, requests will raise a ConnectionError exception.
• Response.raise_for_status() will raise an HTTP error when there is an unsuccessful status code.
• If there is a timeout, it will raise a Timeout exception
• TooManyRedirects exception is raised if the request exceeds the configured number of maximum number of redirects.

In this blog we have discussed the python requests module in which we have various advanced features. We discussed installation and making a get and post request with the response content and other concepts in requests library in python. Python requests module is one of the many extraordinary out of the box features of python programming language.

Have any queries? mention them in the comments section, we will get back to you.