Skip to main content

Secrets Manager

The Secrets Manager package allows developers to transparently manage their Django secret key and database credentials using AWS Secrets Manager.

More specifically, developers can store their Django secret key and database credentials in AWS Secrets Manager, and configure the package such that it transparently retrieves those secrets for use in the Django application. The package will also periodically refresh the secrets to prevent staleness, and gracefully handle situations where a secret value changes.

AWS Secrets Manager

AWS Secrets Manager is a secrets management service offered by AWS that comes with many useful features such as encryption, automated secrets rotation, and native integrations with other AWS services. For more information on the service, refer to the official AWS documentation.

Storing and retrieving secrets using a dedicated secrets management service provides the following (non-exhaustive) benefits:

  1. Strong encryption of secrets both at rest and in transit.
  2. Avoids storing secrets in potentially unsafe locations such as environment variables, CI/CD systems, or even directly in the source code repository.
  3. Centralisation of secrets to prevent duplication when multiple applications need to use them.
  4. Leverage on centralised authentication and authorization mechanisms during secrets retrieval.

On top of the general benefits, this package is implemented in a way that allows for secrets rotation without having to restart the application. This makes the process of rotation transparent to the end user, which may be critical for high traffic applications.

Installation​

The Secrets Manager package can be installed using the command below:

pip install govtech-csg-xcg-secretsmanager

Source code for the package can be found at https://github.com/GovTech-CSG/govtech-csg-xcg-secretsmanager.

Installing optional packages

The Secrets Manager package comes with support for either MySQL or PostgreSQL as the database backend. As each of these require their own additional dependencies, you will have to specify your database backend when installing the package.

You can do so by appending the backend name to the Secrets Manager package name, for example: govtech-csg-xcg-secretsmanager[mysql] for MySQL support, govtech-csg-xcg-secretsmanager[postgresql] for PostgreSQL support, or govtech-csg-xcg-secretsmanager[mysql,postgresql] for both.

Setup​

The Secrets Manager package provides two main pieces of functionality - management of database credentials, and management of the Django secret key. The sections below will run through the setup required for each of these.

Management of database credentials​

This is achieved through the use of customized database backends for MySQL and PostgreSQL. These are:

  1. govtech_csg_xcg.secretsmanager.db.mysql
  2. govtech_csg_xcg.secretsmanager.db.postgresql

The backends deliver the following functionality:

  • Transparent retrieval of MySQL/PostgreSQL connection parameters from AWS Secrets Manager.
  • Transparent refreshing of connection parameters from Secrets Manager at specified intervals.
  • If the DB password changes and causes a failed connection attempt, perform ad-hoc refreshing of the secret and retry the connection attempt (useful for zero-downtime secret rotation).

Basics​

The example below provides the name of the AWS Secrets Manager secret that holds the DB credentials. No secret refresh interval is provided, so the default value of 3600 seconds (or 1 hour) takes effect transparently. AWS region and credentials are determined based on environment variables and other locations (see later section for more detail).

To use either backend, add it as the default engine to the DATABASES dictionary in your project's settings.py file. You also minimally have to provide a key AWS_SECRETSMANAGER_CONFIG with the subkey secret_arn pointing to the name of your secret in AWS Secrets Manager (this can be the full ARN or just the human-readable name).

./<PROJECT_NAME>/settings.py
DATABASES = {
    'default': {
        'ENGINE': 'govtech_csg_xcg.secretsmanager.db.mysql', # Or postgresql
        'AWS_SECRETSMANAGER_CONFIG': {
            'secret_arn': 'mysecret/name', # Here we use the human-readable secret name
        },
        # Other options specific to the Django MySQL backend can be passed in as usual.
        'OPTIONS': {
            "init_command": "SET sql_mode='STRICT_TRANS_TABLES'",
        }
    }
}

The AWS Secrets Manager secret containing your database credentials must conform with the standard structure documented for RDS MySQL and RDS PostgreSQL. This should already be the case if you created your secret via the AWS Secrets Manager UI and selected the correct database engine.

Additional configuration for database backends​

For further customization, there are multiple optional configuration parameters which are identical across both backends:

  1. region_name

    • If provided, specifies the AWS region to which the DB secret belongs.
    • If not provided, value can be supplied externally in order of precedence below:
      1. Environment variable AWS_DEFAULT_REGION.
      2. region in the AWS config file (see boto3 docs for more information).

  2. aws_access_key_id, aws_secret_access_key, and aws_session_token

    • If provided, specifies the credentials to use when calling the AWS Secrets Manager API.
    • aws_session_token is only required if using credentials from a temporary AWS session.
    • If not provided, values can be supplied externally in order of precedence below:
      1. Environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN.
      2. Through a shared credentials file (see boto3 docs).
      3. Through the AWS config file (see boto3 docs).
      4. Via the EC2 instance metadata service if none of the above yield results and application is running in an EC2 instance with an instance profile attached.

  3. profile_name

    • If provided, specifies the named profile to use when calling the AWS Secrets Manager API.
    • If not provided, value can be supplied externally from the environment variable AWS_PROFILE.
    • If provided together with aws_access_key_id and aws_secret_access_key, the access key will be used instead.

  4. refresh_interval

    • If provided, specifies the interval in seconds after which the cached DB credentials will be refreshed.
    • Default value of 3600 (1 hour) if not provided.
Order of precedence for configuration sources

Configuration options that are directly provided in settings.py will always take precedence over environment variables and other sources.

Example configurations​

./<PROJECT_NAME>/settings.py
# Using just the access key
DATABASES = {
    'default': {
        'ENGINE': 'govtech_csg_xcg.secretsmanager.db.mysql',
        'AWS_SECRETSMANAGER_CONFIG': {
            'secret_arn': 'mysecret/name',
            'region_name': 'ap-southeast-1'
            'aws_access_key_id': ACCESS_KEY_ID,
            'aws_secret_access_key': SECRET_ACCESS_KEY,
            'refresh_interval': 600
        },
    }
}

# Using an access key with session token (temporary credentials)
DATABASES = {
    'default': {
        'ENGINE': 'govtech_csg_xcg.secretsmanager.db.mysql',
        'AWS_SECRETSMANAGER_CONFIG': {
            'secret_arn': 'mysecret/name',
            'region_name': 'ap-southeast-1',
            'aws_access_key_id': ACCESS_KEY_ID,
            'aws_secret_access_key': SECRET_ACCESS_KEY,
            'aws_session_token': SESSION_TOKEN,
            'refresh_interval': 600
        },
    }
}

# Using a named AWS profile
DATABASES = {
    'default': {
        'ENGINE': 'govtech_csg_xcg.secretsmanager.db.mysql',
        'AWS_SECRETSMANAGER_CONFIG': {
            'secret_arn': 'mysecret/name',
            'region_name': 'ap-southeast-1',
            'profile_name': 'xcg-test',
            'refresh_interval': 600
        },
    }
}

Management of Django secret key​

This is achieved through the use of a custom Django middleware: govtech_csg_xcg.secretsmanager.middleware.SecretKeyRefreshMiddleware.

The middleware delivers the following functionality:

  • Transparent refreshing of Django secret key from AWS Secrets Manager.
  • (For Django version >= 4.1) Transparent rotation of Django secret key without affecting existing sessions and other functionality that depend on the secret key.

Basics​

To use the middleware, add it to your MIDDLEWARE list in settings.py. You also minimally have to provide a key secret_arn within the setting XCG_DJANGO_SECRET_KEY_REFRESH_CONFIG that points to your secret in AWS Secrets Manager (this can be the full ARN or just the human-readable name).

./<PROJECT_NAME>/settings.py
MIDDLEWARE = [
    'govtech_csg_xcg.secretsmanager.middleware.SecretKeyRefreshMiddleware',
    # ...
]

XCG_DJANGO_SECRET_KEY_REFRESH_CONFIG = {
    'secret_arn': 'mysecret/key',
}

The AWS Secrets Manager secret must be of JSON format with a key DJANGO_SECRET_KEY that refers to the value of the secret key.

Structure of secret in AWS Secrets Manager
{
    'DJANGO_SECRET_KEY': 'REPLACE_WITH_YOUR_SECRET_KEY_HERE'
}

Additional configuration for Django secret key​

Configuration of settings.XCG_DJANGO_SECRET_KEY_REFRESH_CONFIG is identical to that of AWS_SECRETSMANAGER_CONFIG for the database backends. Refer to the database backend configuration section for more details.

NOTE: The AWS Secrets Manager secret containing the Django secret key must be stored in the JSON format (i.e. key-value pairs in the UI), and must contain a key "DJANGO_SECRET_KEY" that references the secret key value.

{
    ...
    "DJANGO_SECRET_KEY": "secret-key-value"
}

IAM permissions for accessing secrets​

Below is a sample IAM policy that provides the appropriate permissions for the database backends and secret key refresh middleware to access secrets in AWS Secrets Manager.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "secretsmanager:DescribeSecret",
                "secretsmanager:GetSecretValue",
            ],
            "Resource": [
                "arn:aws:secretsmanager:ap-southeast-1:073624714755:secret:xcg/rds/mysql-LyDNma",
                "arn:aws:secretsmanager:ap-southeast-1:073624714755:secret:xcg/django-njSFlJ"
            ]
        }
    ]
}
tip

The example above combines permissions for both secrets into one policy. For increased granularity of permissions, you may choose to create 2 similar policies each targeting one specific secret.

Logging​

The package uses its own logger with the name secretsmanager. By default, this logger uses a stream handler and emits messages with level INFO and above. The log message format looks something like this:

2023-06-22 09:50:43,348 [INFO][XCG][secretsmanager]: This is an informational message.

To safely override the default logging configuration, you can write code in settings.py to import the logger from the package and modify it directly. The example below sets the logger's level to DEBUG:

settings.py
from govtech_csg_xcg.secretsmanager import logger
import logging

logger.setLevel(logging.DEBUG)

For more information on the Python logging library, see the official documentation.

Usage​

Once set up, the database backends and secret key refresh middleware will operate transparently in the background. Hence, there are no specific usage instructions for these components.

However, the Secrets Manager package also exposes 2 underlying client classes used by the backends and middleware for developers who want to directly call the AWS Secrets Manager API. The usage of these classes are documented in the sections below.

SecretsManager client class​

The SecretsManager client class provides basic secrets retrieval functionality, and can be imported from the module govtech_csg_xcg.secretsmanager.clients.

The code snippet below provides an example of basic usage:

import json

from govtech_csg_xcg.secretsmanager.clients import SecretsManager

sm = SecretsManager(region_name='ap-southeast-1', profile_name='xcg')
secret_string = sm.get_secret_string('xcg/randomsecret')
secret_dict = json.loads(secret_string)

The stubs below document the public API of this class:

class SecretsManager:
    # Creating an object
    # Note: If any of the arguments are not provided, underlying code will attempt
    # to pull them from the environment or config files, similar to the DB and refresh
    # modules above
    def __init__(
        self,
        region_name: str = None, # Optional - AWS region name
        aws_access_key_id: str = None, # Optional - AWS access key ID
        aws_secret_access_key: str = None, # Optional - AWS secret access key
        aws_session_token: str = None, # Optional - AWS session token
        profile_name: str = None # Optional - AWS named profile name
    ):

    # Retrieving the specified version of a secret
    def get_secret_string(self, secret_name: str, version_stage: str = None):
        """Returns a secret string for a given AWS Secrets Manager secret.

        Args:
            secret_name: String that identifies the secret to be retrieved. This can be the secret's
                full ARN, the secret's partial ARN (e.g. 'secretname-nutBrk'), or the secret's human
                readable name. The recommendation is to use the full ARN if using ARN at all.
            version_stage: String that identifies the staging label of the secret value to be retrieved.
                Possible values are "AWSPREVIOUS", "AWSCURRENT", and "AWSPENDING". Default is "AWSCURRENT".

        Returns:
            A string representing the value of the specified secret. Note that this could be in the
            format of a JSON string if the secret was defined as key-value pairs, or it could just
            be a normal string if the secret was defined in plaintext. See examples below:

            JSON string: '{"secret_name":"secret_value"}'
            Normal string: 'secret_value'

        Raises:
            SecretRetrievalError: Raised if any errors occur while attempting to retrieve the secret
                from AWS Secrets Manager.
        """

CacheSecretsManager client class​

The CacheSecretsManager client class inherits from SecretsManager, providing additional caching and secrets refresh capabilities on top of basic secrets retrieval. It can also be found in the module govtech_csg_xcg.secretsmanager.clients.

The code snippet below provides an example of basic usage:

import json

from govtech_csg_xcg.secretsmanager.clients import CacheSecretsManager

csm = CacheSecretsManager(region_name='ap-southeast-1', profile_name='xcg', refresh_interval=300)
secret_string = sm.get_secret_string('xcg/frequently_rotated')
# Force a refresh of the cached secret before the refresh interval is up.
secret_string = sm.get_secret_string('xcg/frequently_rotated', force_refresh=True)
secret_dict = json.loads(secret_string)

The stubs below document the public API of this class:

class CacheSecretsManager(SecretsManager):
    # Creating an object
    # Note: If any of the arguments are not provided, underlying code will attempt
    # to pull them from the environment or config files, similar to the DB and refresh
    # modules above
    def __init__(
        self,
        region_name: str = None, # Optional - AWS region name
        aws_access_key_id: str = None, # Optional - AWS access key ID
        aws_secret_access_key: str = None, # Optional - AWS secret access key
        aws_session_token: str = None, # Optional - AWS session token
        profile_name: str = None, # Optional - AWS named profile name
        refresh_interval: int = 3600 # Optional - default is 3600 seconds
    ):

    # Retrieving the specified version of a secret
    def get_secret_string(
        self,
        secret_name: str,
        version_stage: str = None,
        force_refresh: bool = False
    ):
        """Returns a (cached) secret string for a given AWS Secrets Manager secret.

        Args:
            secret_name: String that identifies the secret to be retrieved. This can be the secret's
                full ARN, the secret's partial ARN (e.g. 'secretname-nutBrk'), or the secret's human
                readable name. The recommendation is to use the full ARN if using ARN at all.
            version_stage: String that identifies the staging label of the secret value to be retrieved.
                Possible values are "AWSPREVIOUS", "AWSCURRENT", and "AWSPENDING". Default is "AWSCURRENT".
            force_refresh: Boolean value that indicates whether or not to force a refresh before the
                regular refresh interval.

        Returns:
            A string representing the value of the specified secret. Note that this could be in the
            format of a JSON string if the secret was defined as key-value pairs, or it could just
            be a normal string if the secret was defined in plaintext. See examples below:

            JSON string: '{"secret_name":"secret_value"}'
            Normal string: 'secret_value'

        Raises:
            SecretRetrievalError: Raised if any errors occur while attempting to retrieve the secret
                from AWS Secrets Manager.
        """

    # Retrieving the previous version of a secret
    def get_previous_secret_string(self, secret_name: str, force_refresh: bool = False):
        """Convenience function for retrieving previous version of a secret.

        Equivalent to get_secret_string(secret_name, version_stage='AWSPREVIOUS')

        Note: Refreshing one version of a secret will refresh the cached values of all versions of
            the same secret.
        """