Blog

Ideas and insights from our team

How to configure Sass and Bower with django-compressor - part 2 (deployment to Heroku and S3)


This is the part 2 of the guide on how to configure a Django 1.8 project with Sass (django-libsass), Bower (django-bower) and django-compressor. You can read the part 1 by clicking here. In this part 2, we'll describe how to deploy this setup to Heroku and AWS S3.

Example code

Keep in mind that we are working over the code we've described in the part 1 of this guide. If you want to see right now the complete example of this setup, click here to check the GitHub repository at the branch with-s3.

Why S3?

Django wasn't made to serve static files, it was made to run application code and respond to complex requests. S3 can host and serve static files more efficiently, since it's tailored for hosting static files. That said, S3 is not a CDN. While you should host your static (CSS, images and scripts for running your app) and media (user files uploaded through your app) files in S3, you might also need a CDN to function as a cache if your app has a high traffic. In AWS, the CDN is complementary to S3 and it is called CloudFront. But the first step certainly is to save your Heroku server resources by serving static files with S3. So lets do it.

Get S3 Bucket Access Keys

If you've already configured S3 for a Django project, you can skip this section. If not, please follow the steps below:

  • Create a new bucket, and write down its name.
  • Create a new IAM user to get access keys: go to AWS IAM, click on Users section, then on Create New Users, then input a name like s3-your-bucket-name and keep Generate an access key for each user checked.
  • Now the user is created, click on Show User Security Credentials to get its access keys. Write them down, we will need them for the deployment step. After that, click on Download Credentials for backup purposes.
  • After that, we need to give this new user access to the bucket we've just created. Click on this user in the list to see its details, then scroll to Permissions, click on Inline Policies, then on To create one, click here. Choose Custom Policy and Select. Add a Policy Name like s3-your-bucket-name-policy and on Policy Document fill as below (don't forget to replace with your bucket-name where necessary!):
{
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::replace-with-your-bucket-name",
                "arn:aws:s3:::replace-with-your-bucket-name/*"
            ]
        }
    ]
}
  • Finally, configure the CORS for your bucket (if you don't know what CORS is, read here). Access the S3 bucket list, right click on the bucket, then Properties, after Permissions and on Add CORS Configuration fill as below:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>Authorization</AllowedHeader>
    </CORSRule>
</CORSConfiguration>
  • Now we can move on to edit your Django project settings.py, but first we need to install some dependencies.

Install the dependencies with pip

These dependencies are additional to the ones we've seen in part 1 and they're necessary for sending static assets to S3 and deploying app code to Heroku.

pip install django-storages-redux python-decouple django-toolbelt

Edit Django settings

We will use django-storages-redux for hosting your static files in S3. django-storages-redux is a fork of the original django-storages, which is abandoned. Follow the steps below for configuring S3 in Django settings.py:

  • Add storages to INSTALLED_APPS:
INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'storages', # add this

    'djangobower',
    'compressor',

    'sample_app',
)
  • Add settings for AWS, staticfiles and django-compress. Here we are using the python-decouple lib, which provides the config function. config gets the setting from environment variables.
from decouple import config, Csv

if not DEBUG:
    ALLOWED_HOSTS = config('ALLOWED_HOSTS', cast=Csv())
    SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

    AWS_ACCESS_KEY_ID = config('AWS_ACCESS_KEY_ID')
    AWS_SECRET_ACCESS_KEY = config('AWS_SECRET_ACCESS_KEY')
    AWS_STORAGE_BUCKET_NAME = config('AWS_STORAGE_BUCKET_NAME')
    AWS_S3_CUSTOM_DOMAIN = '{0}.s3.amazonaws.com'.format(AWS_STORAGE_BUCKET_NAME)

    STATICFILES_STORAGE = 'sample_app.custom_storages.StaticCachedS3BotoStorage'
    DEFAULT_FILE_STORAGE = 'sample_app.custom_storages.MediaS3BotoStorage'

    COMPRESS_STORAGE = STATICFILES_STORAGE

    STATIC_URL = 'https://{0}/static/'.format(AWS_S3_CUSTOM_DOMAIN)
    MEDIA_URL = 'https://{0}/media/'.format(AWS_S3_CUSTOM_DOMAIN)

    COMPRESS_URL = STATIC_URL

Note that sample_app.custom_storages.StaticCachedS3BotoStorage and sample_app.custom_storages.MediaS3BotoStorage are custom classes available at custom_storages.py. Copy them to your project. The StaticCachedS3BotoStorage class has some adjusments necessary to make django-storages work with S3, based on django-storages documentation. It also sets the static directory to /static/ inside the bucket, storing static files separate from media files, which prevents overwrites. Meanwhile, the MediaS3BotoStorage class sets the directory to /media/ and keeps the media files private with querystring auth, only accesible for some duration after the request. This is usually the desired setting, since media files are generally user uploaded private content.

After settings are done, we only need to add some files required by Heroku.

Add Heroku required files

Since this project uses Bower (which is node based) and Django (Python), we need multiple buildpacks. Buildpacks are the scripts which setup the stack necessary to run apps. Please follow the steps below to properly configure Heroku for running your app:

  • Create a new Heroku dyno or set the Heroku remote
  • Set BUILDPACK_URL to heroku-buildpack-multi, since this project uses multiple buildbacks. You can fork the Vinta heroku-buildpack-multi repository if you prefer:
heroku config:set BUILDPACK_URL=https://github.com/vintasoftware/heroku-buildpack-multi.git
  • Create a file named .buildpacks at the root of the project and fill it with the contents below. This file has the buildpacks necessary to run your Django and Bower setup. You can fork these repositories if you prefer:
https://github.com/heroku/heroku-buildpack-nodejs.git
https://github.com/vintasoftware/heroku-buildpack-python-with-django-bower.git
  • Create a package.json file in the project root with the content below. This hints node that we need Bower:
{"private": true,"dependencies": {"bower": "*"}}
  • Create a Procfile file in the project root with the following content. Replace the example.wsgi with the name of your wsgi file. Your Django project should have one:
web: gunicorn example.wsgi --log-file -
  • Now we can set Heroku environment variables with the command below. Fill the AWS variables with values according to the bucket and IAM user you've created:
heroku config:set DEBUG=False \
                  AWS_ACCESS_KEY_ID=fill-here \
                  AWS_SECRET_ACCESS_KEY=fill-here \
                  AWS_STORAGE_BUCKET_NAME=fill-here \
                  ALLOWED_HOSTS=fill-here-with-a-*-or-a-value-or-a-comma-separated-list \
                  SECRET_KEY=fill-here-with-a-django-secret-key

(ALLOWED_HOSTS and SECRET_KEY aren't related to S3, but it is a good practice to set them as environment variables. ALLOWED_HOSTS can be a * at first, but when all is working, fill it with your domain. And you can generate a new SECRET_KEY with a simple Python script)

  • Deploy!
git push heroku with-s3:master # in our case, in your case may be only: git push heroku master

Please refer the working example project if you have any trouble. Note that the correct branch for this part 2 is called with-s3. If you need additional help, please leave a comment below.

What is next?

Now you have an almost perfect Django frontend setup! The only thing we are missing is using a CDN, which is recommended for web apps with global reach. We will address that in a future post. Follow us on Twitter to keep updated: @vintasoftware. Thanks!

About Flávio Juvenal

Controversial software developer who questions everything: "Are we really going forward?". Python enthusiast, but is afraid JavaScript will conquer the world. Enjoys working with Django and now wants to write system checks for everything on it.

Comments