Installation and configuration

Installation

At the command line:

$ pip install django-cid

Configuration

You need to add cid.apps.CidAppConfig to your list of installed apps.

INSTALLED_APPS = (
    # some apps
    'cid.apps.CidAppConfig',
    # some other apps
)

Generation of the correlation id

The correlation id may be generated by django-cid itself or come from upstream through an incoming HTTP header.

To let django-cid generate an id, set CID_GENERATE to true in the settings:

CID_GENERATE = True

By default, django-cid uses str(uuid.uuid4()) to generate the correlation id but you can customize this generation to suit your needs in the settings:

CID_GENERATOR = lambda: f'{time.time()}-{random.random()}'

Letting django-cid generate a new correlation id is perfectly acceptable but does suffer one drawback. If you host your Django application behind another web server such as nginx, then nginx logs won’t contain the correlation id. django-cid can handle this by extracting a correlation id created in nginx and passed through as a header in the HTTP request. For this to work, you must enable a middleware in the settings, like this:

MIDDLEWARE = (
    'cid.middleware.CidMiddleware',
    # other middlewares
)

The middleware takes care of getting the correlation from the HTTP request header. By default it looks for a header named X_CORRELATION_ID, but you can change this with the CID_HEADER setting:

CID_HEADER = 'X_MY_CID_HEADER'

Note

Most WSGI implementations sanitize HTTP headers by appending an HTTP_ prefix and replacing - by _. For example, an incoming X-Correlation-Id header would be available as HTTP_X_CORRELATION_ID in Django. When using such a WSGI server in front of Django, the latter, sanitized value should be used in the settings.

If a correlation id is provided upstream (e.g. “1234”), it is possible to concatenate it with a newly generated one. The cid will then look like 1234, 1aa38e4e-89c6-4655-9b8e-38ca349da017. To do so, use the following settings:

CID_GENERATE = True
CID_CONCATENATE_IDS = True

This is useful when you use a service-oriented architecture and want to be able to follow a request amongst all systems (by looking at logs that have the first correlation id that was set upstream), and also on a particular system (by looking at logs that have the id added by the system itself).

Inclusion of the correlation id in the response

By default django-cid sets an HTTP header in the HTTP response with the same name as configured in CID_HEADER. You may customize it with CID_RESPONSE_HEADER in the settings:

CID_RESPONSE_HEADER = 'X-Something-Completely-Different'

Note

As indicated in the note above, if Django is behind a WSGI server that sanitizes HTTP headers, you need to customize CID_RESPONSE_HEADER if you want the same header name in the response as in the request.

# Nginx sets ``X-Correlation-Id`` but it is sanitized by the WSGI server.
CID_HEADER = 'HTTP_X_CORRELATION_ID'
# Don't use the default value (equal to CID_HEADER) for the response header.
CID_RESPONSE_HEADER = 'X-Correlation-Id'

If you don’t want the header to appear in the HTTP response, you must explicitly set CID_RESPONSE_HEADER to None.

# Don't include the header in the HTTP response.
CID_RESPONSE_HEADER = None

Inclusion of the correlation id in logs

The most useful feature of django-cid is to include the correlation id in logs. For this you need to add the cid.log.CidContextFilter log filter in your log settings, apply it to each logger, and customize the formatter(s) to include the cid variable.

Here is what it looks like on the the default logging configuration provided by Django’s startproject. Changed lines are highlighted.

LOGGING = {
    'version': 1,
    'formatters': {
        'verbose': {
            'format': '[cid: %(cid)s] %(levelname)s %(asctime)s %(module)s %(message)s'
        },
        'simple': {
            'format': '[cid: %(cid)s] %(levelname)s %(message)s'
        },
    },
    'handlers': {
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
            'formatter': 'verbose',
            'filters': ['correlation'],
        },
    },
    'filters': {
        'correlation': {
            '()': 'cid.log.CidContextFilter'
        },
    },
    'loggers': {
        'testapp': {
            'handlers': ['console'],
            'filters': ['correlation'],
            'propagate': True,
        },
    },
}

You can then use your loggers as you normally do, safe in the knowledge that you can tie them all back to the correlation id.

If you want to include the correlation id in all logs, you need to tweak the “root” key like this:

LOGGING = {
    # ...
    'root': {
        'level': 'INFO',
        'handlers': ['console'],
        'filters': ['correlation'],
    },
    # ...
}

Inclusion of the correlation id in SQL queries

django-cid can add the correlation id as a comment before the SQL query so that the correlation id appears in your database logs like this:

/* cid: 1234567-68e8-45fc-85c1-e025e5dffd1e */
SELECT col FROM table

For this you need to change your database backend to one that is provided by django-cid. For example, for sqlite3 you need to use the following:

DATABASES = {
    'default': {
        'ENGINE': 'cid.backends.sqlite3',
        'NAME': location('db.sqlite3'),
    }
}

django-cid has a wrapper for all backends that are currently supported by Django. Here is the full list:

mysql: cid.backends.mysql
oracle: cid.backends.oracle
postgis: cid.backends.postgis
postgresql: cid.backends.postgresql
sqlite3: cid.backends.sqlite3

By default, the correlation id appears as shown in the example above. You may change that by defining a CID_SQL_COMMENT_TEMPLATE that is a string with a cid format parameter:

CID_SQL_COMMENT_TEMPLATE = 'correlation={cid}'

Also, you may change the position of the correlation id injected in the statement by defining a CID_SQL_STATEMENT_TEMPLATE that is a string with a cid and a sql format parameter:

CID_SQL_STATEMENT_TEMPLATE = '/* {cid} */\n{sql}'

Inclusion of the correlation id in templates

django-cid provides a template context processor that adds the correlation id to the template context if it is available. To enable it, you need to add it in the list of TEMPLATE_CONTEXT_PROCESSORS in the settings:

TEMPLATE_CONTEXT_PROCESSORS = (
    # other template processors
    'cid.context_processors.cid_context_processor',
)

It will add a context variable correlation_id if a correlation id is available. You may include it in your template with the follwing snippet:

{% if correlation_id %}
    <meta name="correlation_id" content="{{ correlation_id }}">
{% endif %}