HTTP Caching and CDNs for Django: ETags, Cache-Control, Surrogate Keys, and Purging

Application caching makes your code faster. HTTP caching makes your code unnecessary — the response is served by a CDN node near the user, and your server never learns the request happened. The mechanism is a handful of headers, and getting them slightly wrong is how you end up serving one customer's dashboard to another.

Four caches, and who obeys what

Between your view and the user there are four places a response can be stored:

• The browser cache — private to one user, obeys Cache-Control.
• The CDN / shared cache — serves many users, obeys Cache-Control plus its own rules.
• The reverse proxy (nginx, Varnish, Traefik) — same idea, closer to you.
• Django's cache framework — inside the process, unrelated to HTTP semantics.

The word that governs the dangerous ones is private versus public. private means "one user may store this"; public means "a shared cache may serve this to anyone". Mislabel a personalised page as public and a CDN will happily hand it to strangers.

Cache-Control, decoded

Cache-Control: public, max-age=300, s-maxage=3600, stale-while-revalidate=86400

• max-age=300 — browsers may reuse for 5 minutes.
• s-maxage=3600 — shared caches may reuse for an hour. Overrides max-age for the CDN only.
• stale-while-revalidate=86400 — after expiry, serve the stale copy instantly and refresh in the background. This one is the single biggest perceived-speed win available, and it is one word.

The split between max-age and s-maxage is the useful trick: short in the browser so users see changes quickly, long at the CDN so your server stays idle. A purge fixes the CDN instantly; you cannot purge a browser.

from django.views.decorators.cache import cache_control

@cache_control(public=True, max_age=300, s_maxage=3600,
               stale_while_revalidate=86400)
def tutorial_detail(request, slug):
    ...

@cache_control(private=True, no_store=True)
def account_dashboard(request):
    ...

no-store is the strong one: do not write this anywhere, ever. Use it on authenticated pages, payment flows and anything with a token in it. no-cache does not mean that — it means "store it, but revalidate before reuse", which surprises people at exactly the wrong moment.

Vary: the header that decides who shares a cache entry

Vary tells shared caches which request headers change the response. Get it wrong in either direction and you have a bug.

Vary: Accept-Encoding, Accept-Language, Cookie

• Missing Vary: Cookie on a page that reads the session → the first visitor's personalised page is served to everyone. This is the classic catastrophic caching bug.
• Vary: Cookie on a public page → every distinct cookie set becomes its own cache entry, so your hit rate collapses to roughly zero. Analytics cookies alone guarantee it.

The resolution is architectural: keep public pages genuinely cookie-free. Serve anonymous visitors pages with no session cookie at all, and fetch the personalised fragment — cart count, username — with a small separate request that is never cached.

<span id="cart-count" hx-get="/cart/count/" hx-trigger="load"></span>

Now the page itself is one cache entry shared by every anonymous visitor, and the personal bit costs one tiny uncached request.

ETags and conditional requests

Freshness lifetimes avoid requests. ETags make the unavoidable ones cheap: the client sends the tag it holds, and you answer 304 Not Modified with an empty body.

from django.views.decorators.http import etag, last_modified

def tutorial_etag(request, slug):
    row = Tutorial.objects.filter(slug=slug).values("updated_at", "views").first()
    if not row:
        return None
    return f'"{row["updated_at"].timestamp()}"'

@etag(tutorial_etag)
def tutorial_detail(request, slug):
    ...

Two rules. The tag function must be cheaper than rendering — one indexed query on a couple of columns, not the same work as the view. And the tag must change whenever anything visible changes: template edits and CSS bumps included. Hashing your deploy SHA into it costs nothing and prevents the "the site is stale after deploy" mystery.

Django's ConditionalGetMiddleware will generate ETags from response bodies automatically. That saves bandwidth but not rendering time — the view already ran. Explicit tags skip the work; automatic tags only skip the transfer.

Surrogate keys: purge by meaning, not by URL

You edit one tutorial. Which URLs are now wrong? The detail page, the category listing, the tag pages, the home page, the sitemap, the feed. Enumerating them is a losing game.

Instead, tag responses as you build them and purge by tag:

class SurrogateKeyMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        request.surrogate_keys = set()
        response = self.get_response(request)
        if request.surrogate_keys and response.status_code == 200:
            response["Surrogate-Key"] = " ".join(sorted(request.surrogate_keys))
        return response


def tutorial_detail(request, slug):
    tutorial = get_object_or_404(Tutorial, slug=slug)
    request.surrogate_keys |= {
        f"tutorial-{tutorial.pk}",
        f"category-{tutorial.category_id}",
    }
    ...

Then a single signal invalidates every page that mentioned the object, wherever it appeared:

@receiver(post_save, sender=Tutorial)
def purge_tutorial(sender, instance, **kwargs):
    purge_surrogate_keys([f"tutorial-{instance.pk}",
                          f"category-{instance.category_id}"])

Not every CDN supports surrogate keys, and the header name varies. Where it is unavailable, the fallback is short s-maxage plus stale-while-revalidate — less precise, but it degrades gracefully instead of serving week-old content.

Static assets: the easy win nobody should skip

Hashed filenames make assets immutable, and immutable assets can be cached effectively forever:

STORAGES = {
    "staticfiles": {"BACKEND": "django.contrib.staticfiles.storage.ManifestStaticFilesStorage"},
}

location /static/ {
    add_header Cache-Control "public, max-age=31536000, immutable";
}

immutable tells the browser not even to revalidate on reload. Safe only because the filename changes when the content does — which is exactly what the manifest storage guarantees. Never put immutable on an unhashed path.

Caching for logged-in users

The received wisdom is "you cannot cache authenticated pages". You can cache most of them, if you separate the layers:

• Shared shell, private fragments. The product page is identical for everyone; only the cart badge and the "you own this" notice differ. Cache the page publicly, load the differences separately.
• Per-user CDN caching by key. Some CDNs can vary on a signed cookie or header, giving each user their own entry. Powerful, and easy to misconfigure — only worth it at real scale.
• Cache the expensive query, not the page. When personalisation is deep, HTTP caching is the wrong tool; use Django's cache framework on the pieces.

And whatever you choose: on login, logout and any state change, send Cache-Control: no-store and change the session cookie. The bug that leaks a logged-in page to the next visitor almost always starts with a cached redirect.

Verifying it actually works

curl -sI https://example.com/tutorials/some-slug/ | grep -iE 'cache|vary|age|etag'

What to look for:

• Age greater than zero — the CDN is serving from cache. If it is always 0, you are not caching at all.
• An X-Cache style header showing HIT or MISS — request the same URL twice and expect the second to hit.
• Vary containing exactly what you intended and nothing more.
• A Set-Cookie on a page you believe is public — that is your hit rate leaking away, and most CDNs refuse to cache such a response at all.

Measure hit rate per route rather than overall. A 90% average can hide the one expensive page that never caches — and that page is your load.

Checklist

• Authenticated and payment pages: private, no-store.
• Public pages: public, short max-age, long s-maxage, plus stale-while-revalidate.
• No session cookie on anonymous public pages; personalisation loaded separately.
• Vary set deliberately — never accidentally including Cookie.
• Static files hashed and served immutable.
• Surrogate keys on content pages, purged from model signals.
• Deploy SHA folded into ETags so a release invalidates cleanly.
• Hit rate monitored per route, not as one number.

Done properly, most of your traffic stops arriving. The server you were about to upgrade turns out to be idle — which is the cheapest performance work available, and it is mostly headers.