Codementor Events

Data import from URL, local CSV and JSON files into DB- Django app

Published Oct 27, 2017
Data import  from URL, local CSV and JSON files into DB- Django app

Steps to import data into your Django app
So you have a Django app and want to load data from a URL, a CSV file or a JSON file, I got you 🙂
First, create your models, make migrations, migrate and all that setup you have to do, but I’m sure you got all that covered 😉
See it on Github here: https://github.com/NEbere/data-import/tree/master

Import from URL:
This code gets data from a URL, be sure to have requests installed or any other HTTP library of choice.

"""
Import json data from URL to Datababse
"""
import requests
import json
from import_data.models import Movie #Import your model here
from django.core.management.base import BaseCommand
from datetime import datetime
 
IMPORT_URL = 'https://jsonplaceholder.typicode.com/photos' # URL to import from
 
class Command(BaseCommand):
  def import_movie(self, data):
  title = data.get('title', None)
  url = data.get('url', None);
  release_year = datetime.now()
  try: #try and catch for saving the objects
    movie, created = Movie.objects.get_or_create(
    title=title,
    url=url,
    release_year=release_year
    )
    if created:
      movie.save()
     display_format = "\nMovie, {}, has been saved.
     print(display_format.format(movie))
  exceptExceptionas ex:
    print(str(ex))
    msg = "\n\nSomething went wrong saving this movie: {}\n{}".format(title, str(ex))
    print(msg)
  def handle(self, *args, **options):
  """
  Makes a GET request to the API.
  """
  headers = {'Content-Type': 'application/json'}
  response = requests.get(
  url=IMPORT_URL,
  headers=headers,
  )
  response.raise_for_status()
  data = response.json()
  for data_object in data:
  self.import_movie(data_object)

to run this command, do:
manage.py <the_file_name> (without the .py extension
so if file was importer.py, i'd run the command as:

manage.py importer

NB: mind the code indentation on your IDE
continue reading:
http://happinessnwosu.com/2017/10/17/import-data-into-database-django-app/

Again, here is the github repo with all the import scripts and sample files:
https://github.com/NEbere/data-import/tree/master
Happy coding 🙂

Discover and read more posts from Happiness Nwosu C.
get started
post commentsBe the first to share your opinion
Muhammad Arslan
6 years ago

what if i have no sql?

Stew
6 years ago

Hello thank you! i have not so clear idea about import by CSV

Happiness Nwosu C.
6 years ago

Hi, what are the challenges you’re having?

Stew
6 years ago

no one, but reading post i don’t understand how import and show data from CSV in django

Happiness Nwosu C.
6 years ago

Alright… It’s basically about importing data into a Django app … JSON data from a URL or from a file… or from a CSV file. You can clone the git repo, try to run the command after setting up the project, perhaps then it’d make more sense. :smile:

Stew
6 years ago

Oh thank you, you right! Just to clarify, i have a csv file with ecommerce products (price, name, asin, title, description, ecc ecc) and i want popolate with it a django site, your script allow do that? Thanks

Show more replies