Implementing a file upload to Cloudinary endpoint with Python + DRF
Inspiration for this post
I recently implemented an image upload endpoint in Cloudinary, using python and Django Rest Framework. The whole process was pretty straight forward but things got a little interesting when I tried to test the endpoint.
After solving this problem, I thought it wise to write a blog post that kinda explains all that I did from the setup to the testing.
Hope you have a fun read.
Setup Virtual environment and install dependencies
Let's start by creating a working directory which we would call django-app. In your terminal run:
mkdir django-app
cd django-app
The above commands would create a directory called django-app and enter that directory from your terminal
Next thing is to setup a virtual environment using pipenv as follows:
sudo pip install pipenv #installs pipenv
pipenv shell # creates virtual environment using pipenv
Then we install all our dependencies.
pipenv install django djangorestframework cloudinary
Setting up our django project
NB: If you have worked with django and django-rest-framework before you should be able to skip this section
Once our dependencies have been installed, we then create our Django project by running:
django-admin startproject upload_project .
This would create a project and our working directory would look like this:
.
|-- Pipfile
|-- Pipfile.lock
|-- manage.py
`-- upload_project
|-- __init__.py
|-- settings.py
|-- urls.py
`-- wsgi.py
Now we create a new django app, upload_app by running:
python manage.py startapp upload_app
The folder structure should look like this:
.
|-- Pipfile
|-- Pipfile.lock
|-- manage.py
|-- upload_app
| |-- __init__.py
| |-- admin.py
| |-- apps.py
| |-- migrations
| | `-- __init__.py
| |-- models.py
| |-- tests.py
| `-- views.py
`-- upload_project
|-- __init__.py
|-- settings.py
|-- urls.py
`-- wsgi.py
Next thing is to add Django Rest Framework and upload_app in the list of installed apps by modifying the INSTALLED_APPS array in upload_project/settings.py.
INSTALLED_APPS=[
...,
'rest_framework',
'upload_app',
...,
];
If everything works well you should be able to start the server by running:
python manage.py runserver
Setup Cloudinary
Before we can create start to upload images to cloudinary, we must first create a cloudinary account at the Cloudinary Website.
Once this is done you would be able to obtain an API_KEY, API_SECRET and CLOUD_NAME.
In order to configure Cloudinary, go to settings.py and add the following block of code at the top
...
import cloudinary
cloudinary.config(cloud_name='<cloud-name-here>',
api_key='<api_key_here>',
api_secret='<api_secret>')
...
Note that ideally, you would want to put config data like these in an environmental variable as you don't want these sensitive info to be exposed to the public
Implement the endpoint
You can implement the endpoint by adding the view in upload_app.views.py to
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework.parsers import MultiPartParser, JSONParser
import cloudinary.uploader
class UploadView(APIView):
parser_classes = (
MultiPartParser,
JSONParser,
)
@staticmethod
def post(request):
file = request.data.get('picture')
upload_data = cloudinary.uploader.upload(file)
return Response({
'status': 'success',
'data': upload_data,
}, status=201)
The first few lines of the file import the necessary modules and creates UploadView class that is an APIView.
The UploadView class specifies a set of parsers. The MultiPartParser allows the View to be able to recognise images when they are sent to the application.
Then, the post method tells tries to get a file in picture attribute of the request body, uploads the image to cloudinary using cloudinary.uploader.upload and finally sends the response from cloudinary back to the user.
Now it is time to add the upload url to our app. We do this by updating urls.py as follows
from upload_app.views import UploadView
urlpatterns = [
path('api/upload-image', UploadView.as_view()),
]
Test app using postman
We can test our code using postman to see that it working.
The screenshots below show a sample of how the endpoint works:
In the above screenshots we see that we are using form-data to select images from our local machine. Then, we send the request to our API and get the expected result.
NB: The request might take a few seconds to run depending on your internet connection
Writing automated tests for the endpoint
Setting up the test and installing pytest-django
Now we are going to do the final part of this which is writing automated tests for the endpoint. We are going to be using django-pytest to test the endpoint, thus we would want to install it as follows:
pipenv install pytest-django
While that is installing your can create a pytest.ini file that sets up pytest-django. This file should look like this:
[pytest]
DJANGO_SETTINGS_MODULE = upload_project.settings
python_files = tests.py test_*.py
The file option DJANGO_SETTINGS_MODULE points to our settings.py file while the second configuration tells pytest what patterns it should match.
Writing the actual test
This endpoint has 3 main features that make writing unittest for it a special case:
- It interacts with an external service and tries to make an Ajax request to Cloudinary
- It involves uploading an image. This image would ideally need to be the same. It should also have the same size anytime we hit the endpoint.
Let's discuss the implications of these features a bit
Note on testing external services
While testing parts of a codebase that interacts with external services, you usually do not want the test to actually try to hit those services. In these cases, you create a mock response from the external service.
In our own case, we would be mocking the response returned from Cloudinary and using it as a basis of our test. We are going to do this mocking using unnitest.mock.Mock. With this module, we can mock the response that is returned by cloudinary rather than actually making the request.
Note on sending images
The first thing we have to do is to add an image to the project. I have put an image named mock-image.png in upload_app such that the new file system now looks like this
.
|-- Pipfile
|-- Pipfile.lock
|-- db.sqlite3
|-- manage.py
|-- upload_app
| |-- __init__.py
| |-- admin.py
| |-- apps.py
| |-- migrations
| | `-- __init__.py
| |-- mock-image.png
| |-- models.py
| |-- tests.py
| `-- views.py
`-- upload_project
|-- __init__.py
|-- settings.py
|-- urls.py
`-- wsgi.py
When testing the file upload, we must ensure that the code that we end an object to our test that mimics what we want to actually do.
In order to achieve this, we would create a tempfile.TemporaryFile object that contains the content of our image file. Then we pass this TemporaryFile object to our test.
Writing tests
We would be writing the test in upload_app.tests.py. Here is how it looks
import pytest
from unittest.mock import Mock
from rest_framework.test import APIClient
import cloudinary.uploader
import os
from tempfile import TemporaryFile
django_client = APIClient()
class TestUploadImage():
def test_upload_image_to_cloudinary_succeeds(
self):
cloudinary_mock_response = {
'public_id': 'public-id',
'secure_url': 'http://hello.com/here',
}
cloudinary.uploader.upload = Mock(
side_effect=lambda *args: cloudinary_mock_response)
with TemporaryFile() as temp_image_obj:
for line in open(os.path.dirname(__file__) + '/mock-image.png', 'rb'):
temp_image_obj.write(line)
response = django_client.post(
'/api/upload-image',
{'picture': temp_image_obj},
format="multipart",
)
response_data = response.data
assert response.status_code == 201
assert response_data['status'] == 'success'
assert response_data['data'] ==cloudinary_mock_response
assert cloudinary.uploader.upload.called
The first few lines import the necessary modules to the project. Then, we define a test class named TestUploadImage that contains only one test case test_upload_image_to_cloudinary_succeeds.
This method creates a variable cloudinary_mock_response which a dictionary which we would use to mock the response from Cloudinary.
Then using unittest.mock.Mock, we mock cloudinary.uploader.upload function so that it returns our mock dictionary rather than make an API call to Cloudinary.
The next line uses a context manager to create a TemporaryFile. TemporaryFiles are not actual files but allow us to create file-like objects that we can use in our code. In our example, we use it to temporary store info of our image file, just before we send the post request to our app. You can read the docs for more info on TemporaryFiles
The for-loop in the next line reads our image and stores it in the TemporaryFile. Afterwards, the API is hit with the django_client created previously followed by a set of assertions that ensure that it returns what was expected.
Note that the last assertion, cloudinary.uploader.upload.called, checks if the cloudinary.uploader.upload method was called.
Run the test
You can run the test via
pytest
The only one test should fail
Concluding notes
I would like to end by telling some good practices that was skipped in this tutorial.
-
Usually, you would want to limit the size of the images that you upload to cloudinary in order to ensure that you don't use up your space very quickly.
-
You would want to validate that the file that the user is trying to upload is an image. This you would achieve by modifying the post method in
views.py. -
The last thing is that because image upload to cloudinary takes a lot of time. You usually would not want to make the user wait for it to complete before you give control to the user. Therefore, you would probably want to do such heavy task in a seperate worker using
Celery -
Finally, when the user wants to upload an image to celery. You would probably want to delete the previous image of the user in your db. You do this by calling
cloudinary.uploader.destroy.
I hope this article was helpful. You can access the code in my Github Repo
