Dotenv

I've got used to python-dotenv library to customize some project's settings on per-deployment basis. But there is one thing which I can't say I like.

Problem

The problem is the way this library makes values from .env available to project - by using os.environ.setdefault which actually pollutes project's environment.

By occasion this can lead to some sensitive information like api keys to leak. Let's fix it.

Solution

My first attempt was to check the library's API and find a way to load values without polluting os.environ. This is possible and here is the code:

# -*- coding: utf-8 -*-

import os

from dotenv.main import dotenv_values

BASE_DIR = os.path.dirname(os.path.abspath(__file__))
DOTFILE = os.environ.get('DOTFILE', '.env')

dotenv = {}
dotenv_path = os.path.join(BASE_DIR, DOTFILE)
if os.path.isfile(dotenv_path):
    dotenv = dotenv_values(dotenv_path)

def env(key, default=None):
    return os.environ.get(key, dotenv.get(key, default))

API_KEY = env('API_KEY', 'test-key')

This solution works and looks pretty, well, reasonable. At the same time dotenv_values function is not documented and there is a chance API can be changed.

Besides python-dotenv has features I don't need, like command-line interface or automatic .env file location up the directory tree (I always know where .env file should be).

So, I've rolled my own solution which has fewer lines of code and is very easy to reason about:

# -*- coding: utf-8 -*-

from __future__ import absolute_import

import os
import codecs
import logging


logger = logging.getLogger(__name__)


def load_dotenv(filename, encoding='utf-8'):

    decoder = codecs.getdecoder('unicode_escape')

    def decode(v):
        if v and v[0] == v[-1] and v[0] in ('"', '\''):
            v = decoder(v[1:-1])[0]
        return v

    with codecs.open(filename, 'rb', encoding=encoding) as f:
        lines = (x.strip() for x in f if x.strip())
        lines = (x for x in lines if not x.startswith('#') and '=' in x)
        lines = (x.split('=', 1) for x in lines)
        lines = ((x.strip(), y.strip()) for x, y in lines)
        lines = ((x.upper(), decode(y)) for x, y in lines)
        for line in lines:
            yield line


class dotenv(object):

    def __init__(self, filename='.env', encoding='utf-8'):
        fn = filename
        if not os.path.isabs(filename):
            filename = os.path.abspath(os.path.join(os.curdir, filename))
        if not os.path.exists(filename):
            logger.warning('dotenv file "%s" not found', fn)
            filename = None
        self.filename = filename
        self.encoding = encoding
        self._data = None

    @property
    def data(self):
        if self._data is None and self.filename is not None:
            self._data = dict(
                load_dotenv(self.filename, encoding=self.encoding))
        return self._data

    def get(self, key, default=None):
        data = self.data or {}
        return os.environ.get(key, data.get(key, default))

It can be used like this:

# -*- coding: utf-8 -*-

import os

from .dotenv import dotenv

BASE_DIR = os.path.dirname(os.path.abspath(__file__))
DOTFILE = os.environ.get('DOTFILE', '.env')

env = dotenv(filename=os.path.join(BASE_DIR, DOTFILE))

API_KEY = env.get('API_KEY', 'test-key')

It is very easy to extend dotenv class with methods like get_bool, get_int, get_list or whatever you need and provide some logic to verify settings values as needed.

Value coming from os.environ takes precedence over .env value when specified (this is useful while running jenkins jobs).

By implementing self-made solution we can simplify our project's dependency tree and be sure everything works as we need.

---

That's it for dotenv. Stay tuned.