While writing an update the a reusable app django-nyt (a framework for handling notifications), I came across a simple issue: Django's built-in decorator @override_settings couldn't change my app's runtime settings, a feature that I needed to build test cases based on a variety of available app settings.
On top of that, I noticed that the documentation for configuration django-nyt had a funny pattern: A setting would be named ENABLED but to override it with Django project settings, it should be called NYT_ENABLED. This made the documentation look messy.
I proposed a new pattern and immediately had great inputs to improve the proposal from the Fediverse, especially thanks to Adam Johnson and Josh Thomas.
Here's a narrated version of how the pattern came to be - if you want to skip the rest of this blog post, you can see the changes and a before/after screenshot of the documentation in this pull request.
We're gonna go through how the properties of the implementation looks and finally show the full example.
@override_settingsThe first property for the settings pattern that we want is for most IDEs to be able to see the available settings and suggest them when for instance the programmer has typed the "." (dot) after the settings object, i.e. settings.<AUTO-SUGGEST-HERE>.
Since most IDEs will not run code but inspect the code statically, we would need to find a pattern that explicitly adds settings as properties of a module or a class.
That's why the solution implements a Python class with named properties. We are going to use a dataclass to declare this in a more type-explicit fashion and add the frozen parameter, to guard against changing values directly on the object. This allows us to dictate the defaults and add a strict logic about how to override these defaults. We're also going to add the docstrings of each setting.
from dataclasses import dataclass
@dataclass(frozen=True)
class AppSettings:
MYAPP_EXAMPLE = True
"""How to use the MYAPP_EXAMPLE setting."""
app_settings = AppSettings()
Going further with dataclasses, you might see an emerging pattern that isolates each app's settings in its own dictionary, rather than a prefix-based approach to the namespace. Josh Thomas shared an implementation for django-email-relay for exactly this: https://github.com/westerveltco/django-email-relay/blob/main/src/email_relay/conf.py. In this blog post, I'm sticking to the prefix-based approach - in part because I wanted to be backwards compatible for django-nyt, but also because I think that storing all your app's django settings in a dictionary can be left optional.
An example of benefiting from using a frozen dataclass - we NEVER want to assign values directly to this class, but always go through django.conf.settings:
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django_nyt.conf import app_settings
>>> app_settings.NYT_ENABLED
True
>>> # Let's change the value the wrong way
>>> app_settings.NYT_ENABLED=False
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "<string>", line 4, in __setattr__
dataclasses.FrozenInstanceError: cannot assign to field 'NYT_ENABLED'
>>> # Now let's change the value the right way
>>> from django.conf import settings as django_settings
>>> django_settings.NYT_ENABLED=False
>>> app_settings.NYT_ENABLED
False
Don't fancy dataclasses? It's fair to think that it's a bit YAGNI, you can also just use a normal class with a custom __getattr__ and a __setattr__ that raises RuntimeError or similar.
Let's also make sure that something as prominent and important as an application's configuration interface has type hints. This makes it clear for developer's what data types are acceptable, rather than having to write this manually in the docstring. As you will see later, typehints are automatically visible with Sphinx autodoc.
Note that Python currently doesn't have any built-in way to verify types during runtime. So even though it would be nice to fail during runtime if a user-setting isn't of the right type, then it's not currently feasible.
from dataclasses import dataclass
@dataclass(frozen=True)
class AppSettings:
MYAPP_EXAMPLE: bool = True # <= ADDED TYPE HINT
"""How to use the MYAPP_EXAMPLE setting."""
app_settings = AppSettings()
@override_settingsNow we're going to get to the part that's really nice.
Here is some outdated code that I used a lot in Django applications in the good old days:
# my_app/settings.py
from django.conf import settings
MY_SETTING = gettattr("MY_APP_MY_SETTING", settings, "default value")
Not good enough! It will only read from Djangos settings once, and if they change, for instance while executing a test suite full of different scenarios, then we will never see the new value once the module has been instantiated.
So we expand our new implementation with a custom "getter":
from dataclasses import dataclass
# All attributes accessed with this prefix are possible to overwrite
# through django.conf.settings.
settings_prefix = "NYT_"
@dataclass(frozen=True)
class AppSettings:
NYT_EXAMPLE: bool = True # <= ADDED TYPE HINT
"""How to use the EXAMPLE setting."""
def __getattribute__(self, __name: str) -> Any:
"""
Check if a Django project settings should override the app default.
In order to avoid returning any random properties of the django settings, we inspect the prefix firstly.
"""
if __name.startswith(settings_prefix) and hasattr(django_settings, __name):
return getattr(django_settings, __name)
return super().__getattribute__(__name)
app_settings = AppSettings()
Noticed that we have introduced settings_prefix? That's just to avoid accessing other random properties of django.conf.settings and force using a settings prefix for all our own settings. So all settings, HAVE to be called NYT_<something>, and you cannot access some other setting app_settings.SITE_ID (use django.conf.settings for that).
Now we're at my pet peeve (dictionary: an opportunity for complaint that is seldom missed). In my experience, there are too many settings that go undocumented. Because there's an additional "penalty" of manual documentation work for using public settings, we will see negative consequences:
So let's use locality of behavior: Where the setting is defined, is where we want to document it, using docstrings, as we've already done in our previous examples. Thus, the documentation will be immediately available for an IDE. And with the right pattern, we can list all available settings in our documentation without doing any work!
Luckily, Sphinx has autodoc and this way, we can generate documentation for our app.conf module easily. In django-nyt, we have a configuration.rst that simply reads:
Configuration
=============
.. automodule:: django_nyt.conf
:noindex:
.. autoclass:: AppSettings()
:members:
And voila!
All of this fits in a simple file that I encourage you to call conf.py just like django.conf.
"""
These are the available settings, accessed through ``myapp.conf.app_settings``.
All attributes prefixed ``MYAPP_*`` can be overridden from your Django project's settings module by defining a setting with the same name.
For instance, to enable the admin, add the following to your project settings:
.. code-block:: python
MYAPP_ENABLE_ADMIN = False
"""
from __future__ import annotations
from collections import OrderedDict
from dataclasses import dataclass
from dataclasses import field
from typing import Any
from django.conf import settings as django_settings
from django.utils.translation import gettext_lazy as _
# All attributes accessed with this prefix are possible to overwrite
# through django.conf.settings.
settings_prefix = "MYAPP_"
@dataclass(frozen=True)
class AppSettings:
"""Access this instance as ``my_app.conf.app_settings``."""
MY_APP_TABLE_PREFIX: str = "myapp"
"""The table prefix for tables in the database. Do not change this unless you know what you are doing."""
MYAPP_ENABLE_ADMIN: bool = False
"""Enable django-admin registration for django-nyt's ModelAdmin classes."""
MYAPP_LABEL: str = _("Subject")
"""Some label that is translateable"""
def __getattribute__(self, __name: str) -> Any:
"""
Check if a Django project settings should override the app default.
In order to avoid returning any random properties of the django settings, we inspect the prefix firstly.
"""
if __name.startswith(settings_prefix) and hasattr(django_settings, __name):
return getattr(django_settings, __name)
return super().__getattribute__(__name)
app_settings = AppSettings()
When we write from my_app.conf import app_settings, we can read the following out loud: "Get me my application's settings from my application's configuration".
That is kind of confusing since there is already something in Django called AppConfig that would offer us to put our configuration in the application's registry entry.
We would then perhaps access our settings instance like this?
apps.get_app_config("admin").settings.MY_SETTING