Requests for fun and profit

Introduction

The purpose of this tutorial is to give you an overview of how to make HTTP requests in python with the requests library. Let yourself be charmed by concision and purity!

Setting up requests library

First of all, you'll need to install requests library like this:

  $ pip install requests

GET request

Making GET request is very straightforward. Watch by yourself!

  import requests
  r = requests.get("https://jsonplaceholder.typicode.com/posts")
  print(r.status_code)
  print(r.json())
  print(r.headers)

In just three lines you made a get request, retrieve answer code, json response, and headers, the three main types of information you need on a daily basis. Fan-tas-tic!

GET request with parameters

You can pass a dictionary as parameters to a get request by using params

  import requests
  payload = {'name': 'robert', 'age': '15'}
  r = requests.get('http://httpbin.org/get', params=payload)
  print(r.json()['args'])

POST request

Requests provides us json parameter to handle a post request.

  import requests
  payload = {'title': 'hello requests', 'body': 'do it and go home'}
  r = requests.post('https://jsonplaceholder.typicode.com/posts', json=payload)
  print(r.json())

PUT request

No comment!

  import requests
  payload = {'title': 'hello requests again', 'body': 'do it and go home now'}
  r = requests.put('https://jsonplaceholder.typicode.com/posts/1', json=payload)
  print(r.json())

DELETE request

No comment!

  import requests
  r = requests.delete('https://jsonplaceholder.typicode.com/posts/1')
  print(r.status_code)

Adding Headers

Another parameter here is headers. As the previous, it takes a dictionnary of headers.

  import requests
  payload = {'title': 'hello requests', 'body': 'do it and go home'}
  headers = {"Accept": "application/json", "Content-Type": "application/json"}
  r = requests.post('https://jsonplaceholder.typicode.com/posts', json=payload, headers=headers)
  print(r.json())

Upload file

Because you will need to send or download a file. Replace post by get in the second case

  import requests
  url = 'http://httpbin.org/post'
  files = {'file': open('the_old_man.jpg', 'rb')}
  r = requests.post(url, files=files)
  print(r.json())

Basic Authentication

Let's authenticate

  imports requests
  requests.get('https://somesecuredwebsite/user', auth=('username', 'password'))

Streaming

In order to handle large data efficiently, just streaming it!

  import json
  import requests
  streaming_request = requests.get('http://httpbin.org/stream/100', stream=True)
  for entry in streaming_request.iter_lines():
      if entry:
          print(json.loads(entry) )

Going deeper

That's it! It was a glance. Sufficient for everyday queries.
Do not hesitate to go further by taking a look at the official requests website !