Manage error logs in python

Posted on 25-May-2020

In every programming language at some point, you'll need to use the logger to debug your running application.

Here is a table with errors logged according to the selected level:

python errors

If you want to test the above list, create a file with the following content and execute it:

#!/usr/bin/env python

import logging

logging.basicConfig(filename='example.log', level=logging.ERROR)

logging.info('info message')
logging.warning('warning message')
logging.error('error message')
logging.debug('debug')
logging.critical('critical message')

To see the messages added in log file, run:

cat example.log

The availble error levels are the following:

logging.CRITICAL = 50
logging.FATAL = CRITICAL
logging.ERROR = 40
logging.WARNING = 30
logging.WARN = WARNING
logging.INFO = 20
logging.DEBUG = 10
logging.NOTSET = 0

If you want more control over the log messages, use the following script as starting point:

You will need to run pip install jsonlogformatter before running the script

#!/usr/bin/env python

import os
import logging
import json_log_formatter


class CustomJsonFormatter(json_log_formatter.JSONFormatter):
    def json_record(self, message: str, extra: dict, record: logging.LogRecord):
        return {
            'message': record.msg,
            'level': record.levelname,
            'time': record.created,
            'pathname': record.pathname,
            'lineno': record.lineno,
            'args': record.args,
            # 'record': record,
        }

class ApplicationLogger:
    application_name: str
    environment: str

    LOGFILE_FORMAT = '{0}-{1}.log'

    ENV_LOG_LEVEL = {
        'prod': logging.WARNING,
        'dev': logging.DEBUG,
        'stg': logging.DEBUG,
    }

    def _set_format(self):
        return CustomJsonFormatter()
        # return logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')

    def get_logger(self):
        return self.logger

    def __init__(self, application_name, environment='dev'):
        self.application_name = application_name
        self.environment = environment

        self.logger = logging.getLogger(self.application_name)
        self.logger.setLevel(logging.DEBUG)

        formatter = self._set_format()

        handler = logging.FileHandler(self.LOGFILE_FORMAT.format(self.application_name, self.environment))
        handler.setLevel(self.ENV_LOG_LEVEL.get(self.environment, logging.DEBUG))
        handler.setFormatter(formatter)
        self.logger.addHandler(handler)

        if bool(os.getenv('DEBUG')) is True:
            stream_handler = logging.StreamHandler()
            stream_handler.setLevel(logging.DEBUG)
            stream_handler.setFormatter(formatter)
            self.logger.addHandler(stream_handler)


class MyApplication:
    def __init__(self, log):
        self.log = log

    def run(self):
        self.log.info('Start my application')
        self._custom_logic()

    def _custom_logic(self):
        self.log.info('Apply custom logic')

if __name__ == '__main__':
    logger = ApplicationLogger('my-app', 'dev').get_logger()

    myapp = MyApplication(logger)
    myapp.run()

If you create a file with the above code and execute it, then you'll see in the terminal two messages in json format and the same messages will be saved in the log file my-app-dev.log.

DEBUG=1 python myfile.py

If you don't append DEBUG env variable, then you'll see the messages only in the my-app-dev.log file.

If you want the messages to be in plain text, you can change that in method setformat from ApplicationLogger class.

Furthermore, you can use a different formatter for the CLI output if you want. Feel free to change the behaviour as you want

Tutorials

Helpers

Manage error logs in python