Logging into multiple files in Django

27 Feb 2015


In a project I am currently working on we had this simple problem, but stinging problem. We needed logs for both errors and the inner workings of the API we had created from within Django. The problem was the size of the logs. There was so much info logs that they were drowning out the errors and leaving us in the blind. Thankfully, Django allows you a nifty way to handle this problem. Well, it really is a feature in Python, but Django gives you something more to work on

Python’s logger

import logging
std_logger = logging.getLogger(__name__)

The standard Python logger allows you to log several levels of logs. The __name__ simply creates a generic logger that we can use. It is also possible, to create one or more named loggers like this:

debug_logger = logging.getLogger('debug_logger')
error_logger = logging.getLogger('error_logger')

This creates two named loggers called debug_logger and error_logger which we will now be using as loggers dedicated for debug messages and error messages

Django Unchanged

Now we work Django’s magic. We add some nice little settings to Django which will do a few things for us. It will let us decide what, where, when and how to log the logs that are created by Django. These are the setting that we will be using:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    # filters will define when a logger should run
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse',
        },
        'require_debug_true': {
            '()': 'django.utils.log.RequireDebugTrue',
        },
    },
    # format in which logs will be written
    'formatters': {
        'simple': {
            'format': '[%(asctime)s] %(levelname)s %(message)s',
	    'datefmt': '%Y-%m-%d %H:%M:%S'
        },
        'verbose': {
            'format': '[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s',
	    'datefmt': '%Y-%m-%d %H:%M:%S'
        },
    },
    # handlers define the file to be written, which level to write in that file,
    # which format to use and which filter applies to that logger
    'handlers': {
        'debug_logfile': {
            'level': 'DEBUG',
            'filters': ['require_debug_true'], # do not run debug logger in production
            'class': 'logging.FileHandler',
            'filename': os.path.join(BASE_DIR, 'debug.log'),
            'formatter': 'verbose'
        },
        'error_logfile': {
            'level': 'ERROR',
            'filters': ['require_debug_false'], # run logger in production
            'class': 'logging.FileHandler',
            'filename': os.path.join(BASE_DIR, 'error.log'),
            'formatter': 'verbose'
        },
    },
    # here the handlers for the loggers and the level of each logger is defined
    'loggers': {
        'error_logger': {
            'handlers': ['error_logfile'],
            'level': 'ERROR'
         },
        'debug_logger': {
            'handlers': ['debug_logfile'],
            'level': 'DEBUG'
        },
    }
}

Now all you have to do is call the logger in your code with this:

debug_logger.debug(debug_message)
error_logger.error(error_message)
Previous Post Next Post