OverIQ.com

  1. Home
  2. Django 1.11 Tutorial
  3. Settings for Multiple Environments in Django

Settings for Multiple Environments in Django

Last updated on July 27, 2020

Multiple Settings File #

So far, a single settings.py file has served us well. Now we are moving to the production environment, as a result, some of the settings in settings.py file needs to be changed. Most notably, we will change DEBUG to False and ALLOWED_HOSTS to the production server IP or domain.

To efficiently run our Django project in different environments, we will split our single settings.py into multiple files, each one representing settings for a particular environment.

Let's start by renaming settings.py to old.settings.py and creating a directory named settings in the Django configurations directory (djangobin/django_project/django_project). Inside the settings directory create following four files:

  1. __init__.py
  2. base.py
  3. dev.py
  4. prod.py.

The __init__.py file tells Python that the settings directory is a package. The base.py contains the settings common to development and production environment. The dev.py and prod.py contains the settings specific to the development and production respectively.

At this point, Django configurations directory should look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
django_project/
├── celery.py
├── __init__.py
├── old.settings.py
├── settings
│   ├── __init__.py
│   ├── base.py
│   ├── dev.py
│   └── prod.py
├── urls.py
└── wsgi.py

1 directory, 9 files

The complete code of base.py is as follows (changes are highlighted):

djangobin/django_project/django_project/settings/base.py

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
"""
Django settings for django_project project.

Generated by 'django-admin startproject' using Django 1.11.

For more information on this file, see
https://docs.djangoproject.com/en/1.11/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/1.11/ref/settings/
"""

import os, json
from django.core.exceptions import ImproperlyConfigured

with open(os.path.abspath("djangobin-secrets.json")) as f:
    secrets = json.loads(f.read())


def get_secret_setting(setting, secrets=secrets):
    try:
        return secrets[setting]
    except KeyError:
        raise ImproperlyConfigured("Set the {} setting".format(setting))

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.dirname(__file__)))


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/1.11/howto/deployment/checklist/


# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.humanize',
    'django.contrib.flatpages',
    'django.contrib.sites',
    'django.contrib.sitemaps',
    'djangobin',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'django.middleware.common.BrokenLinkEmailsMiddleware',
]

ROOT_URLCONF = 'django_project.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
            os.path.join(BASE_DIR, 'templates'),
        ],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'djangobin.context_processors.recent_snippets',
            ],
        },
    },
]

WSGI_APPLICATION = 'django_project.wsgi.application'


# Password validation
# https://docs.djangoproject.com/en/1.11/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/1.11/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.11/howto/static-files/

STATIC_URL = '/static/'

STATIC_ROOT = os.path.join(BASE_DIR, 'staticfies')

MEDIA_ROOT  = os.path.join(BASE_DIR, 'media')

MEDIA_URL = '/media/'

LOGIN_REDIRECT_URL = 'djangobin:index'

LOGIN_URL = 'djangobin:login'

SITE_ID = 1

Here are few things to notice:

  • For security reasons, you must never hardcode sensitive configurations like SECRET_KEY, database credentials, or API keys in your code. Furthermore, these configurations are also subject to change across deploys. If you put these configurations in the code then you would have to constantly update the code everytime you move to a new environment.

    In our case, we have stored all our sensitive configurations in a JSON file named djangobin-secrets.json, which resides in project root directory (djangobin/django_project). The contents of djangobin-secrets.json looks like this:

    djangobin/django_project/djangobin-secrets.json

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    {
  "SECRET_KEY": "rj3vhyKiDRNmth75sxJemvOuTy1Hy0ogeKgS9JP8Gp7dGDctfSMUuOt5QbSpsS9xAlvBMTXW3Z6VTODvvFcV3TmtrZUbGkHBcs8I",
  "DATABASE_NAME": "djangobin",
  "DATABASE_USER": "postgres",
  "DATABASE_PASSWORD": "pass",
  "DATABASE_HOST": "127.0.0.1",
  "DATABASE_PORT": "5432",
  "EMAIL_HOST_USER": "apikey",
  "EMAIL_HOST": "smtp.sendgrid.net",
  "EMAIL_HOST_PASSWORD": "IK.qQecgqph1Sa9TkOOljo8pA.5Xrj1oyJKuOGBbHnWFmdDe32G8XXojH45W1loxIsktqY3Nc",
  "EMAIL_PORT": "587",
  "EMAIL_USE_TLS": "True"
}

    This file contains the SECRET_KEY, database credentials and email credentials. If you are using a version control system (which you should) add this file to .gitignore.

    To generate the secret key we the get_random_string() function of django.utils.crypto module:

    1
2
3
4
5
6
    >>>
>>> from django.utils.crypto import get_random_string
>>>
>>> get_random_string(80)
'ZEjwrGbwF9fjFAfTXTvz7LxXnCGUPOJyfPszrk2WHtbrbH3mgBeag2NWUueGYYiA7fTw36F50T2R3F5L'
>>>

    In production, we will use PostgreSQL as our database and SendGrid to send emails.

    If you already know how to install and configure PostgreSQL, go ahead and populate the database credentials, otherwise, wait until the next lesson where we will look at how to install and configure PostgreSQL.

    To obtain the email credentials sign up for a free account on SendGrid. As of this writing, SendGrid free plan allows you to send 100 email daily.

    Back to base.py file.

    In lines 16-17, we read the contents of djangobin-secrets.json file and store it in the secrets variable as a dictionary.

    If you try to access a configuration which doesn't exist in the secrets directory, you will get a KeyError exception. Sadly, this is not very helpful. To make debugging easier we have defined get_secret_setting() function. This method returns the value of the setting it is called with or an ImproperlyConfigured exception if the setting is not found in the secrets directory.

  • Our settings file is now one level deep inside the Django configurations directory. In other words, the BASE_DIR setting no longer points to project root directory (djangobin/django_project/) instead it points to the Django configurations directory (djangobin/django_project/django_project). This effectively breaks the path to templates, static files and media files. To account for this, in line 27, we have we added an additional call to os.path.dirname(). This will make sure that the BASE_DIR setting points to the correct base directory.

  • In line 23, we have added 'django.middleware.common.BrokenLinkEmailsMiddleware' to the MIDDLEWARE list. This will email the MANAGERS whenever an HTTP 404 error occurs.

The code for dev.py is as follows:

djangobin/django_project/django_project/settings/dev.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
from .base import *

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = '_5=#=+cl&lp@&ayps6ia0viff)^v$_wvutyyxca!xu0w6d2z3$'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = []

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}


EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'

SERVER_EMAIL = 'infooveriq@gmail.com'
DEFAULT_FROM_EMAIL = SERVER_EMAIL

ADMINS = (
    ('OverIQ', 'admin@overiq.com'),
)

MANAGERS = (
    ('OverIQ', 'manager@overiq.com'),
)

Nothing fancy here, the dev.py just contains development specific settings. To import the common settings from base.py we use from .base import * statement (line 1). Note that we have deliberately hardcoded some of the sensitive configurations because it makes the development process easy. However, this is not the case with prod.py file.

djangobin/django_project/django_project/settings/prod.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
import os
from .base import *

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = get_secret_setting('SECRET_KEY')

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = False

ALLOWED_HOSTS = ["*"]

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': get_secret_setting('DATABASE_NAME'),
        'USER': get_secret_setting('DATABASE_USER'),
        'PASSWORD': get_secret_setting('DATABASE_PASSWORD'),
        'HOST': get_secret_setting('DATABASE_HOST'),
        'PORT': get_secret_setting('DATABASE_PORT'),
    }
}

STATIC_ROOT = 'static'

EMAIL_HOST_USER = get_secret_setting('EMAIL_HOST_USER')
EMAIL_HOST = get_secret_setting('EMAIL_HOST')
EMAIL_HOST_PASSWORD = get_secret_setting('EMAIL_HOST_PASSWORD')
EMAIL_PORT = get_secret_setting('EMAIL_PORT')
EMAIL_USE_TLS = True
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'

DEFAULT_FROM_EMAIL = 'support@overiq.com'
SERVER_EMAIL = 'no-reply@overiq.com'

ADMINS = (
    ('OverIQ', 'infooveriq@gmail.com'),
)

MANAGERS = (
    ('OverIQ', 'manager@overiq.com'),
)

The prod.py is almost similar to dev.py but it defines settings specific to the production environment.

Here are few things to notice:

  1. Unlike dev.py, in prod.py we are loading our sensitive configurations via get_secret_setting() function defined in the base.py file.

  2. In line 10, we set ALLOWED_HOSTS to [*]. ALLOWED_HOSTS setting is a security feature which is used to validate whether the request is coming from allowed domains or not. It specifies a list of a string representing host/domain names this Django project can serve. By default, it is set to an empty list. Once you move to production you required to set it otherwise you will get 500 Internal Server Error.

    If you own example.com and want to allow requests from example.com or www.example.com then you would need to set ALLOWED_HOSTS to:

    ALLOWED_HOSTS =  ['example.com', 'www.example.com']

    If you want to allow requests from example.com and all its subdomains then use the period as a subdomain wildcard. For example:

    ALLOWED_HOSTS =  ['.example.com']

    To allow Django to accept requests from any domain set ALLOWED_HOSTS to "*".

    ALLOWED_HOSTS =  ['*']

    However, in a real deploy, you should limit this setting only to the host/domain you want to allow.

  3. In lines 15-19, we define configurations to connect to PostgreSQL database.

  4. In lines 25-28, we define settings required to send emails via SendGrid.

Running Project #

We have refactored our code quite a lot. Now let's take a look at how we can interact with our Django project with this new setup.

In the terminal execute ./manage.py file and you will get the error as follows:

1
2
3
$ ./manage.py
...
Note that only Django core commands are listed as settings are not properly configured (error: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.).

You will get the same error (just different wording) if you try to run the runserver or shell command.

1
2
3
$ ./manage.py runserver
...  
django.core.exceptions.ImproperlyConfigured: Requested setting DEBUG, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

The problem is that the Django doesn't know where our settings file is located. We can specify the location of settings file from the command line using the --setting option.

1
2
3
4
5
6
7
8
$  ./manage.py runserver --settings=django_project.settings.dev
Performing system checks...

System check identified no issues (0 silenced).
June 04, 2018 - 06:37:16
Django version 1.11, using settings 'django_project.settings.dev'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Note that you will need to specify --settings option everytime you execute the manage.py script. For example:

$ ./manage.py shell --settings=django_project.settings.dev

$ ./manage.py makemigrations --settings=django_project.settings.dev

Specifying the --settings option everytime you execute ./manage.py file can quickly become tedious. Alternatively, you can set the DJANGO_SETTINGS_MODULE environment variable to the desired settings file as follows:

$ export DJANGO_SETTINGS_MODULE=django_project.settings.dev

You can now execute the ./manage.py as usual without specifying the path to the settings file.

$ ./manage.py runserver

The DJANGO_SETTINGS_MODULE variable will remain in existence until the shell session is active. Unfortunately, If you start a new shell you will have to set DJANGO_SETTINGS_MODULE again.

A much better approach would be to modify the virtualenv's activate script and set DJANGO_SETTINGS_MODULE environment variable when activating the virtualenv and unset it when deactivating the virtualenv.

Open activate script and modify it as follows:

djangobin/env/bin/activate

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly

deactivate () {
    unset -f pydoc >/dev/null 2>&1

    # reset old environment variables
    # ! [ -z ${VAR+_} ] returns true if VAR is declared at all
    if ! [ -z "${_OLD_VIRTUAL_PATH+_}" ] ; then
        PATH="$_OLD_VIRTUAL_PATH"
        export PATH
        unset _OLD_VIRTUAL_PATH
    fi
    if ! [ -z "${_OLD_VIRTUAL_PYTHONHOME+_}" ] ; then
        PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
        export PYTHONHOME
        unset _OLD_VIRTUAL_PYTHONHOME
    fi

    # This should detect bash and zsh, which have a hash command that must
    # be called to get it to forget past commands.  Without forgetting
    # past commands the $PATH changes we made may not be respected
    if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
        hash -r 2>/dev/null
    fi

    if ! [ -z "${_OLD_VIRTUAL_PS1+_}" ] ; then
        PS1="$_OLD_VIRTUAL_PS1"
        export PS1
        unset _OLD_VIRTUAL_PS1
    fi

    unset VIRTUAL_ENV
    if [ ! "${1-}" = "nondestructive" ] ; then
    # Self destruct!
        unset -f deactivate
    fi

    unset DJANGO_SETTINGS_MODULE
}

# unset irrelevant variables
deactivate nondestructive

VIRTUAL_ENV="/home/pp/django-1.11/djangobin/env"
export VIRTUAL_ENV

_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH

# unset PYTHONHOME if set
if ! [ -z "${PYTHONHOME+_}" ] ; then
    _OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
    unset PYTHONHOME
fi

if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
    _OLD_VIRTUAL_PS1="$PS1"
    if [ "x" != x ] ; then
        PS1="$PS1"
    else
        PS1="(`basename \"$VIRTUAL_ENV\"`) $PS1"
    fi
    export PS1
fi

# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc

pydoc () {
    python -m pydoc "$@"
}

# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands.  Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
    hash -r 2>/dev/null
fi


export DJANGO_SETTINGS_MODULE=django_project.settings.dev

Start a new shell, activate the virtual environment and check the existence of DJANGO_SETTINGS_MODULE environment variable using the echo command:

1
2
$ echo $DJANGO_SETTINGS_MODULE 
django_project.settings.dev

As expected, DJANGO_SETTINGS_MODULE environment variable is available.

Now you can run ./manage.py file without setting any environment variable or specifying the --settings option

Start the Django development server to make sure everything is working as expected.

$ ./manage.py runserver

The output should look like this:

1
2
3
4
5
6
7
Performing system checks...

System check identified no issues (0 silenced).
May 17, 2018 - 14:51:37
Django version 1.11, using settings 'django_project.settings.dev'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

The DJANGO_SETTINGS_MODULE variable will be automatically removed upon deactivating the virtual environment.

Creating Requirements file #

A requirements file is a simple text file containing the project dependencies. We use the requirements file to install the project dependencies.

To create the requirements file execute the following command:

$ pip freeze > requirements.txt

Our DjangoBin project is now complete. In the next lesson, we will learn how to deploy it to the DigitalOcean server.

Django 1.11 Tutorial

Recent Posts