โ๏ธCoding Patterns
Coding patterns we follow on our back end
The 3 Layers
Each backend consists of 3 layers that build on top of each other. Starting from the bottom layer, the layers are:
Models: Defines the data tables (a.k.a. models) and columns (a.k.a. fields).
Serializers: Handles converting data to objects and vice versa.
Views: Defines our API endpoints.
Because [data] models are our bottom layer, it can be said that our backend is model-centric. This means (almost) all our serializers and views and are model-serializers and model-views.
Models
Below are the conventions we follow when modelling our data.
First, understand how to define a Django model.
File Structure
TL;DR
For each model, create a file named after the model in the directory models.
class Person(WarehouseModel): ...class Car(WarehouseModel): ...All models should be imported into models/__init__.py to support importing multiple models from models.
from .car import Car
from .person import Person# some other py file can now import one or more models at a time.
from path.to.models import Car, PersonAny custom logic defined in a model-file should be tested in the directory models, where each model has its own test-file following the naming convention {model}_test.py. Model-tests should inherit ModelTestCase and set their type parameter to be the model they are testing. The name of the model-test-case should follow the convention Test{model}.
from codeforlife.tests import ModelTestCase
from ...models import Person
class TestPerson(ModelTestCase[Person]): ...from codeforlife.tests import ModelTestCase
from ...models import Car
class TestCar(ModelTestCase[Car]): ...Warehousing Data
TL;DR
Determine whether the data stored in a model needs to be warehoused. Data should be warehoused if it can contribute to analyzing users' behavior. By default, new models should be warehoused as it's a rare occurrence that meaningful insights cannot be gained from analyzing its data.
Defining a model as a warehouse-model allows us to sync data from our database to our data warehouse before it's deleted from our database. See our data deletion strategy.
Defining a warehouse-model:
from codeforlife.models import WarehouseModel
class Example(WarehouseModel): ...Defining a non-warehouse-model:
from django.db import models
class Example(models.Model): ...Defining Fields
TL;DR
When defining fields of any type, always set the verbose name and help text. These arguments aid future developers and super-users on the Django admin to understand the purpose of these fields.
from django.utils.translation import gettext_lazy as _
class CompanyMember(WarehouseModel):
is_exec = models.BooleanField(
verbose_name=_("is executive"),
default=False,
help_text=_(
"Whether or not this company member is an executive member."
" Executive members have elevated permissions to perform sensitive actions."
),
)Defining Foreign Keys
TL;DR
When defining a foreign key between 2 models, a few steps need to be taken to inform our static type checker of backward relationships between objects.
Say we have the following models:
class Person(WarehouseModel):
passfrom .person import Person
class Car(WarehouseModel):
owner = models.ForeignKey(Person, on_delete=models.CASCADE)From this we understand that an instance of Car has the attribute owner of type Person. Django will create an attribute for "backward relationships" on each related object with the naming convention {model}_set at runtime. Therefore, Person has the attribute car_set which is a set of type Car.
The problem is static type checkers do not know attributes for backward relationships are going to being auto-generated. Therefore, we need to add type hints.
import typing as t
from django.db.models.query import QuerySet
if t.TYPE_CHECKING:
from .car import Car
class Person(WarehouseModel):
cars: QuerySet["Car"] from .person import Person
class Car(WarehouseModel):
owner = models.ForeignKey(
Person,
on_delete=models.CASCADE,
related_name="cars",
)We import the model we are type hinting inside of if t.TYPE_CHECKING to avoid circular-imports. We also define the name of the backward relationships to be the plural of the model's name to improve readability.
Defining Meta Classes
Models' meta class should inherit TypedModelMeta to support type hinted Meta classes.
from django_stubs_ext.db.models import TypedModelMeta
class Person(WarehouseModel):
class Meta(TypedModelMeta): ...Defining Managers
TL;DR
If you're defining a model with a custom manager:
If you're defining an abstract model with a custom manager:
A model's manager should be defined within the model's class as Manager. If the model inherits WarehouseModel, the manager should inherit WarehouseModel.Manager, providing a string of the model's name as the type parameter to inform the manager of the type of model it will be managing. Below the manager's definition, create a class-level attribute on the model called objects which has its type set to Manager and its value set to an instance of the Manager class.
class Person(WarehouseModel):
class Manager(WarehouseModel.Manager["Person"]): ...
objects: Manager = Manager()If you need to create an abstract model with a custom manager, add a generic type parameter to the manager which is bound to the manager's model. A type variable will need to be defined before the abstract model, following the naming convention Any{model}. However, to bind the type variable to the model before the model is defined, use a lazy binding by referencing the model's name as a string. Below the abstract model's manager, set the type of objects to be Manager[t.Self] so that the manager is managing the type of the inherited model (not the abstract model).
import typing as t
AnyPerson = t.TypeVar("AnyPerson", bound="AbstractPerson")
class AbstractPerson(WarehouseModel):
class Meta(TypedModelMeta):
abstract = True
class Manager(WarehouseModel.Manager[AnyPerson], t.Generic[AnyPerson]): ...
objects: Manager[t.Self] = Manager()
class Musician(AbstractPerson):
class Manager(AbstractPerson.Manager["Musician"]): ...
objects: Manager = Manager()Defining Constraints
TL;DR
When defining constraints, the naming convention is {field}__{condition}.
class Person(WarehouseModel):
years_old = models.IntegerField()
class Meta(TypedModelMeta):
constraints = [
models.CheckConstraint(
check=models.Q(years_old__gte=18),
name="years_old__gte__18"
)
]Test your custom constraint in models/{model}_test.py. Your test name should follow the naming convention test_constraint__{constraint_name}. The unit test should leverage CFL's assertion helper assert_check_constraint, which will check the constraint is enforced under the expected conditions.
class TestPerson(ModelTestCase[Person]):
def test_constraint__years_old__gte__18(self):
with self.assert_check_constraint("years_old__gte__18"):
Person.objects.create(years_old=17)Define Verbose Names
When defining a model, set its verbose names in its meta attributes.
from django.utils.translation import gettext_lazy as _
class Person(WarehouseModel):
class Meta(TypedModelMeta):
verbose_name = _("person")
verbose_name_plural = _("persons")QuerySets as Properties
TL;DR
If there are query-sets that are filtered by a model instance's attributes, create properties on the model's class that return the query-sets. This achieves a shorthand that makes the code base less repetitive and more readable. It's only worth doing this if the query-set is used in 2 or more locations in the code base.
class Person(WarehouseModel):
favorite_music_genre = models.TextField()
@property
def recommended_songs(self):
"""Songs recommended for this person based on their favorite music genre."""
# pylint: disable-next=import-outside-toplevel
from .song import Song
return Song.objects.filter(genre=self.favorite_music_genre)We import Song outside the top level to avoid circular-imports.
Defining Settings
If a model has business logic that impacts its functionality based on custom conditions, add class-level settings on the model in all caps to configure the conditions.
For example, if the Person model has a property which checks if the person is too young:
class Person(WarehouseModel):
MIN_YEARS_OLD = 16
years_old = models.IntegerField()
@property
def is_too_young(self):
return self.years_old < self.MIN_YEARS_OLD This will also help to make tests more robust as values can be dynamically calculated.
class TestPerson(ModelTestCase[Person]):
def test_is_too_young(self):
assert Person(years_old=Person.MIN_YEARS_OLD - 1).is_too_youngSerializers
Below are the conventions we follow when serializing our data.
First, understand how to serialize a Django model.
Model Serializers
For each model, we create a file with the naming convention {model}.py in the directory serializers. Within a model's serializer-file, we create one or more serializers. It's advised to create one serializer per model-view-set action. Each model-serializer should inherit CFL's ModelSerializer by default and set the type parameter to the model being serialized.
from codeforlife.serializers import ModelSerializer
from ..models import Person
class CreatePersonSerializer(ModelSerializer[Person]):
class Meta:
model = Person
class UpdatePersonSerializer(ModelSerializer[Person]):
class Meta:
model = PersonTo avoid repetitively setting the model being serialized to Person, a base serializer may be created with the naming convention Base{model}Serializer.
from codeforlife.serializers import ModelSerializer
from ..models import Person
class BasePersonSerializer(ModelSerializer[Person]):
class Meta:
model = Person
class CreatePersonSerializer(BasePersonSerializer): ...
class UpdatePersonSerializer(BasePersonSerializer): ...All serializers should be imported into serializers/__init__.py to support importing multiple serializers from serializers.
from .person import CreatePersonSerializer, UpdatePersonSerializerAny custom logic defined in a model-serializer-file should be tested in the directory serializers, where each model has its own serializer-test-file following the naming convention {model}_test.py. Model-serializer-test-cases should inherit CFL's ModelSerializerTestCase, set the type parameter to be the model being serialized and set model_serializer_class to the model-serializer being tested. The name of the model-serializer-test-case should follow the convention Test{model}Serializer.
from codeforlife.tests import ModelSerializerTestCase
from ...models import Person
from ...serializers.person import PersonSerializer
class TestPersonSerializer(ModelSerializerTestCase[Person]):
model_serializer_class = PersonSerializerModel List Serializers
When defining model-list-serializers, follow the naming convention {model}ListSerializer. Model-list-serializers should inherit CFL's ModelListSerializer by default to support bulk-creating and bulk-updating model instances.
from codeforlife.serializers import ModelListSerializer
from ..models import Person
class PersonListSerializer(ModelListSerializer[Person]): ...When overriding validate(self, attrs), always call super().validate(attrs) first before implementing any additional validations. This is necessary to ensure the data is valid before bulk-creating or bulk-updating.
class PersonListSerializer(ModelListSerializer[Person]):
def validate(self, attrs):
super().validate(attrs)When overriding update(self, instance, validated_data), both instance and validated_data are of equal length and sorted in the correct order. Therefore, to get the data per model, use zip on instance and validated_data.
class PersonListSerializer(ModelListSerializer[Person]):
def update(self, instance, validated_data):
for person, data in zip(instance, validated_data): ...Validation Errors
When defining validation errors, always set the code of the validation error so that it may be asserted in a test. Error codes must always be unique per validation function so that they may be individually asserted in tests.
class PersonSerializer(ModelSerializer[Person]):
years_old = serializers.IntegerField()
country = serializers.CharField()
def validate_years_old(self, value: int):
if value < 18:
raise serializers.ValidationError(
"You are too young.",
code="too_young",
)
def validate(self, attrs):
if attrs["country"] == "GB" and attrs["years_old"] < 16:
raise serializers.ValidationError(
"A person in GB must be at least 16 years old.",
code="gb_too_young",
)When testing validation errors, the error_code that is expected to be raised must be provided. When testing the validations of a field, the test must follow the naming convention test_validate_{field}__{validation_error_code} and use CFL's assert_validate_field helper. When testing general validations, the test must follow the naming convention test_validate__{validation_error_code} and use CFL's assert_validate helper.
class TestPersonSerializer(ModelSerializerTestCase[Person]):
model_serializer_class = PersonSerializer
def test_validate_years_old__too_young(self):
"""A person must be at least 18."""
self.assert_validate_field(
name="years_old",
value=17,
error_code="too_young",
)
def test_validate__gb_too_young(self):
"""A person in GB must be at least 16."""
self.assert_validate(
attrs={"country": "GB", "years_old": 16},
error_code="gb_too_young",
)Testing create() and update()
If create(self, validated_data) was overridden in a model-serializer, a test named test_create will need to be created and use CFL's assert_create helper. Likewise, if update(self, instance, validated_data) was overridden in a model-serializer, a test named test_update will need to be created and use CFL's assert_update helper.
class PersonSerializer(ModelSerializer[Person]):
years_old = serializers.IntegerField()
def create(self, validated_data):
return Person.objects.create(**validated_data, created_at=timezone.now())
def update(self, instance, validated_data):
instance.years_old = validated_data["years_old"]
instance.last_updated_at = timezone.now()
instance.save(update_fields=["years_old", "last_updated_at"])
return instanceclass TestPersonSerializer(ModelSerializerTestCase[Person]):
model_serializer_class = PersonSerializer
def test_create(self):
"""A person is successfully created."""
self.assert_create(validated_data={"years_old": 18})
def test_update(self):
"""A person is successfully updated."""
self.assert_update(
instance=Person.objects.get(pk=1),
validated_data={"country": "GB", "years_old": 15},
)If create(self, validated_data) was overridden in a model-list-serializer, a test named test_create_many will need to be created and use CFL's assert_create_many helper. Likewise, if update(self, instance, validated_data) was overridden in a model-list-serializer, a test named test_update_many will need to be created and use CFL's assert_update_many helper.
class PersonListSerializer(ModelListSerializer[Person]):
def create(self, validated_data):
return Person.objects.bulk_create([
Person(**person_fields, created_at=timezone.now())
for person_fields in validated_data
])
def update(self, instance, validated_data):
for person, data in zip(instance, validated_data):
person.years_old = validated_data["years_old"]
person.last_updated_at = timezone.now()
person.save(update_fields=["years_old", "last_updated_at"])
return instance
class PersonSerializer(ModelSerializer[Person]):
years_old = serializers.IntegerField()
class Meta:
model = Person
list_serializer_class = PersonListSerializerclass TestPersonSerializer(ModelSerializerTestCase[Person]):
model_serializer_class = PersonSerializer
def test_create_many(self):
"""Many people are successfully created at once."""
self.assert_create_many(
validated_data=[{"years_old": 18}, {"years_old": 25}]
)
def test_update_many(self):
"""Many people are successfully updated at once."""
self.assert_update_many(
instance=[Person.objects.get(pk=1), Person.objects.get(pk=2)],
validated_data=[{"years_old": 67}, {"years_old": 49}],
)Testing to_representation()
If to_representation(self, instance) was overridden in a model-serializer, a test named test_to_representation will need to be created and use CFL's assert_to_representation helper.
class TestPersonSerializer(ModelSerializerTestCase[Person]):
model_serializer_class = PersonSerializer
def test_to_representation(self):
"""A data-field designating if a person is too young is included."""
person = Person.objects.filter(age=17).first()
assert person is not None
self.assert_to_representation(
instance=person,
new_data={"is_too_young": True},
)class PersonSerializer(ModelSerializer[Person]):
def to_representation(self, instance):
representation = super().to_representation()
representation["is_too_young"] = instance.years_old < 18
return representationViews
Below are the conventions we follow when viewing our data.
First, understand how to view a Django model.
Model View Sets
For each model, we create a file with the naming convention {model}.py in the directory views. Within a model's view-file, we create one view-set following the naming convention {model}ViewSet. Each model-view-set should inherit CFL's ModelViewSet by default and set the type parameter to the model being viewed.
from codeforlife.views import ModelViewSet
from ..models import Person
class PersonViewSet(ModelViewSet[Person]): ...All views should be imported into views/__init__.py to support importing multiple views from views.
from .person import PersonViewSetAny custom logic defined in a model-view-file should be tested in the directory views, where each model has its own view-test-file following the naming convention {model}_test.py. Model-view-set-test-cases should inherit CFL's ModelViewSetTestCase, set the type parameter to be the model being viewed, set model_view_set_class to the model-view-set being tested and set basename to the basename used to register the model-view-set in the urls. The name of the model-view-set-test-case should follow the convention Test{model}ViewSet.
from codeforlife.tests import ModelViewSetTestCase
from ...models import Person
from ...views import PersonViewSet
class TestPersonViewSet(ModelViewSetTestCase[Person]):
model_view_set_class = PersonViewSet
basename = "person"Register URLs
Each model-view-set needs to be registered in urls.py using DRF's DefaultRouter. Each registration needs to set prefix to be the plural name of the model with kebab-casing, viewset to be the model-view-set being registered and basename to be the singular of the singular name of the model with kebab-casing.
from rest_framework.routers import DefaultRouter
from .views import PersonViewSet
router = DefaultRouter()
router.register(
prefix="persons",
viewset=PersonViewSet,
basename="person",
)
urlpatterns = router.urlsIf a model has a foreign key to another model, it's also acceptable to set prefix to be a sub-path of the related model, where each model's name is plural with kebab casing.
from .views import PartyInvitationViewSet, VipPartyInvitationViewSet
router.register(
prefix="parties/invitations",
viewset=PartyInvitationViewSet,
basename="party-invitation",
)
# in practice, `is_vip` would likely be a field of `PartyInvitationViewSet` but
# this example is purely to demonstrate the naming convention of `prefix`.
router.register(
prefix="parties/vip-invitations",
viewset=VipPartyInvitationViewSet,
basename="vip-party-invitation",
)Permissions
Permissions must at the very least be set per action in get_permissions, but further conditions can be specified. All permission must be imported from CFL's package to support unit testing.
from codeforlife.permissions import AllowNone
from codeforlife.user.permissions import IsTeacher
class SchoolViewSet(ModelViewSet[School]):
def get_permissions(self):
if self.action == "list":
return [AllowNone()]
if self.action == "destroy":
return [IsTeacher(is_admin=True)]
return [IsTeacher(in_school=True)]If get_permissions is overridden, a test will need to be created for each action the model-view-set provides where each test follows the naming convention test_get_permissions__{action}. Each test should use CFL's assert_get_permissions helper.
from codeforlife.permissions import AllowNone
from codeforlife.user.permissions import IsTeacher
class TestSchoolViewSet(ModelViewSetTestCase[School]):
def test_get_permissions__list(self):
"""No one is allowed to list schools."""
self.assert_get_permissions(
permissions=[AllowNone()],
action="list",
)
def test_get_permissions__destroy(self):
"""Only admin teachers can destroy schools."""
self.assert_get_permissions(
permissions=[IsTeacher(is_admin=True)],
action="destroy",
)
def test_get_permissions__retrieve(self):
"""Only school teachers can retrieve schools."""
self.assert_get_permissions(
permissions=[IsTeacher(in_school=True)],
action="retrieve",
)
...Limiting Default Actions
If any of a model's default actions should not be allowed, they can be disallowed in one of 2 ways.
The first is by excluding an action's HTTP method from http_method_names. The is the preferred way if no action with that HTTP method should be allowed.
The second is to not permit some or all users to trigger the action. The is the preferred way if only some actions for a HTTP should be allowed.
class PersonViewSet(ModelViewSet[Person]):
http_method_names = ["get"]
def get_permissions(self):
if self.action == "list":
return [AllowNone()]
return [IsAuthenticated()]In the above example, actions that do not use HTTP GET are not allowed (e.g. The destroy action which uses HTTP DELETE is not allowed). Furthermore, of the 2 actions which use HTTP GET: list and retrieve, list is not allowed and only authenticated users are allowed to retrieve.
Custom Actions
If a custom action needs to be created for a model-view-set, use CFL's action decorator. A custom action should set detail and methods.
from codeforlife.views import ModelViewSet, action
class PersonViewSet(ModelViewSet[Person]):
@action(detail=True, methods=["post"])
def send_reset_password_email(self, pk: str): ...Testing Actions
Each action provided by a model-view-set should have one test case that calls the action as it is intended to be called. Normally, unit tests can be said to test the "unhappy scenarios" - where we are asserting that the bad scenarios we are expecting to encounter are handled as expected. However, we will also test the "happy scenario" of each action to assert that the action does indeed work.
CFL has provided client-helpers to call and assert each of the default actions. Each action-test should follow the naming convention test_{action}.
class TestPersonViewSet(ModelViewSetTestCase[Person]):
def test_create(self):
"""Successfully creates a person."""
self.client.create(
data={
"first_name": "John",
"last_name": "Doe",
"email": "[email protected]",
},
)Custom Update or Bulk-Update Actions
If a custom update or bulk-update action needs to be created for a model-view-set, use CFL's ModelViewSet.update_action or ModelViewSet.bulk_update_action helpers and pass the name of the action as the first argument. As these actions require a serializer to be provided, you'll also need to override get_serializer_class.
NOTE: Update actions use HTTP PUT and so it needs to be added to http_method_names.
class PersonViewSet(ModelViewSet[Person]):
http_method_names = ["put"]
def get_serializer_class(self):
if action == "reset_password":
return ResetPersonPasswordSerializer
if action == "block_login":
return BlockPersonLoginSerializer
return PersonSerializer
reset_password = ModelViewSet.update_action("reset_password")
block_login = ModelViewSet.bulk_update_action("block_login")To test the "happy scenarios" of each update action, use CFL's client helpers update or bulk_update.
class TestPersonViewSet(ModelViewSetTestCase[Person]):
def test_reset_password(self):
"""Successfully resets a person's password."""
self.client.update(
model=person,
data={"password": "example password"},
action="reset_password",
)
def test_block_login(self):
"""Successfully block the specified persons from logging in."""
self.client.bulk_update(
models=[person1, person2],
data=[{}, {}], # no additional data required
action="block_login",
)Testing get_queryset()
If you are overriding a model-view-set's get_queryset callback, a test will need to be created for each action at the very least where each test follows naming convention test_get_queryset__{action}. Each test will need to use CFL's assert_get_queryset helper. Additional test dimensions can be specified if other factors affect a queryset.
class CarViewSet(ModelViewSet[Car]):
def get_queryset(self):
queryset = Car.objects.filter(is_insured=True)
if action == "drive":
return queryset.filter(owner=self.request.user)
if action == "list":
return queryset | queryset.filter(is_insured=False)
return querysetIn the above example, the default queryset is all insured cars. For the drive action, only insured owners can drive their cars. For the list action, insured or uninsured cars can be listed.
class TestCarViewSet(ModelViewSetTestCase[Car]):
def test_get_queryset__drive(self):
"""Only cars owned by the requesting user can be driven."""
self.assert_get_queryset(
values=Car.objects.filter(is_insured=True, owner=user),
action="drive",
request=self.client.request_factory.post(user=user),
)
def test_get_queryset__list(self):
"""All cars can be listed."""
self.assert_get_queryset(
values=Car.objects.all(),
action="list",
)
def test_get_queryset__partial_update(self):
"""Only insured cars can be partially updated."""
self.assert_get_queryset(
values=Car.objects.filter(is_insured=True),
action="partial_update",
)
...Testing get_serializer_class()
If you are overriding a model-view-set's get_serializer_class callback, a test will need to be created for each action at the very least where each test follows naming convention test_get_serializer_class__{action}. Each test will need to use CFL's assert_get_serializer_class helper.
from ..serializers.person import (
CreatePersonSerializer,
ListPersonSerializer,
PersonSerializer
)
class PersonViewSet(ModelViewSet[Person]):
def get_serializer_class(self):
if self.action == "create":
return CreatePersonSerializer
if self.action == "list":
return ListPersonSerializer
return PersonSerializerfrom ...serializers.person import (
CreatePersonSerializer,
ListPersonSerializer,
PersonSerializer
)
class TestPersonViewSet(ModelViewSetTestCase[Person]):
def test_get_serializer_class__create(self):
"""Creating a person has a dedicated serializer."""
self.assert_get_serializer_class(
serializer_class=CreatePersonSerializer,
action="create",
)
def test_get_serializer_class__list(self):
"""Listing persons has a dedicated serializer."""
self.assert_get_serializer_class(
serializer_class=ListPersonSerializer,
action="list",
)
def test_get_serializer_class__partial_update(self):
"""Partially updating a person uses the general serializer."""
self.assert_get_serializer_class(
serializer_class=PersonSerializer,
action="partial_update",
)
...Testing get_serializer_context()
If you are overriding a model-view-set's get_serializer_context callback, only the actions that have additional context will need to have a test created. The test should follow the naming convention test_get_serializer_context__{action}. Each test will need to use CFL's assert_get_serializer_context helper.
class PersonViewSet(ModelViewSet[Person]):
def get_serializer_context(self):
context = super().get_serializer_context()
if self.action == "create":
context["favorite_color"] = "red"
return contextclass TestPersonViewSet(ModelViewSetTestCase[Person]):
def test_get_serializer_context__create(self):
"""Includes the person's favorite color."""
self.assert_get_serializer_context(
serializer_context={"favorite_color": "red"},
action="create",
)Last updated
Was this helpful?