Self-Hosted Object Storage for Django with MinIO: Buckets, Policies, Durability, and Honest Trade-offs

Object storage has become the default home for user uploads. The interface won: a single HTTP API, implemented by every cloud provider and by several products you can run yourself, which means the same application code works against all of them.

That portability is what makes self-hosting reasonable. This guide covers running an S3-compatible store on your own infrastructure, wiring Django to it, and — the part usually skipped — being honest about what a self-hosted store does and does not protect you against.

Why object storage rather than a directory

A local directory works until the day it does not. It ties uploads to one machine, so scaling to a second application server means either shared network storage or an awkward synchronisation problem. It grows without limit on a disk sized for an operating system. And it makes migration painful, because the data lives outside both the repository and the database.

Object storage separates the data from the machine. Files are addressed by key over HTTP, accessible from any number of application servers, backed by storage that grows independently, and straightforward to mirror elsewhere.

Compatibility is the feature

The reason to care about the S3 API specifically is that it is the lingua franca. Your Django application talks to one interface; behind it can sit a hyperscaler, a European provider, or a process on your own server. Switching means changing an endpoint and credentials, not rewriting file handling.

That is worth designing for even if you never switch. A dependency you can replace in an afternoon is a fundamentally different kind of dependency from one that would take a quarter.

Running a store of your own

A single-node deployment is a container, a data directory and two ports — one for the API, one for the browser console.

services:
  storage:
    image: quay.io/minio/minio:latest
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: ${STORAGE_ROOT_USER:?}
      MINIO_ROOT_PASSWORD: ${STORAGE_ROOT_PASSWORD:?}
    volumes:
      - storage_data:/data
    networks: [internal]
    restart: unless-stopped

Note what is absent: no published ports. The application reaches it over the internal network, and if browsers must fetch objects directly, a reverse proxy exposes the API endpoint over TLS. An object store reachable from the internet on a plain port is an invitation.

Root credentials are for setup only

The root user exists to create other users. It should never appear in your application's configuration. Create a dedicated account per application, grant it access to exactly one bucket, and store those credentials — not the root ones — in the application's environment.

mc alias set local http://storage:9000 "$ROOT_USER" "$ROOT_PASSWORD"
mc mb local/myapp-media
mc admin user add local myapp-service "$SERVICE_SECRET"
mc admin policy attach local readwrite --user myapp-service

The stock read-write policy is broader than you want, because it covers every bucket. Write a policy scoped to the single bucket instead — the difference matters the day an application key leaks, when the question is whether the attacker reached one bucket or all of them.

A bucket policy with actual boundaries

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket"],
    "Resource": ["arn:aws:s3:::myapp-media", "arn:aws:s3:::myapp-media/*"]
  }]
}

Two resource entries are required and people routinely supply one. The bucket itself is the target for listing; the wildcard path is the target for object operations. Omit the first and listing fails with a permission error that looks like a credential problem.

Wiring Django to it

Django's storage backend abstraction means the application code does not change. Configure the backend and the endpoint, and FileField keeps working exactly as before.

STORAGES = {
    "default": {
        "BACKEND": "storages.backends.s3.S3Storage",
        "OPTIONS": {
            "bucket_name": env("STORAGE_BUCKET"),
            "endpoint_url": env("STORAGE_ENDPOINT"),
            "region_name": "us-east-1",
            "signature_version": "s3v4",
            "addressing_style": "path",
            "file_overwrite": False,
            "querystring_auth": True,
            "querystring_expire": 300,
        },
    },
    "staticfiles": {"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage"},
}

Two settings deserve attention. Path-style addressing is usually required for self-hosted stores, because virtual-host style expects wildcard DNS for bucket names. And file_overwrite set to false prevents a new upload from silently replacing an existing object with the same name.

Private by default, shared by signature

With querystring_auth enabled, Django generates time-limited signed URLs rather than permanent public links. The bucket stays private and access is granted per request, for a few minutes.

This is the right default for anything a user uploaded. It is the wrong default for a public marketing image, where every page render produces a fresh URL that no cache or CDN can reuse — and the link stops working when it expires, which is confusing if someone copied it.

Use two buckets: one private with signed access for user content, one public with plain URLs for assets that are genuinely public. Trying to serve both from one bucket produces a compromise that is wrong for both.

Uploading directly from the browser

By default an upload travels twice: browser to your application, application to storage. For large files that consumes a worker for the duration and doubles bandwidth.

A presigned upload URL removes the middle hop. Your application authorises the upload; the browser sends the bytes straight to the store.

import boto3

def presigned_upload(request):
    client = boto3.client("s3", endpoint_url=settings.STORAGE_ENDPOINT)
    url = client.generate_presigned_url(
        "put_object",
        Params={"Bucket": "myapp-media", "Key": f"uploads/{uuid4()}.bin"},
        ExpiresIn=300,
    )
    return JsonResponse({"url": url})

Constrain what you sign: generate the key server-side rather than accepting one from the client, and set a short expiry. A presigned URL is a bearer token — anyone holding it can perform the operation you authorised.

The honest part: one node is not durable

This is where self-hosted object storage is most often oversold. A single-node deployment on one disk has exactly the durability of that disk. The API is identical to a cloud provider's; the guarantees behind it are not.

Managed object storage typically replicates every object across multiple independent failure domains. Your single node does not. It is convenient, portable and cheap — and if the disk fails, the objects are gone.

Multi-node deployments with erasure coding do provide real redundancy, tolerating the loss of several drives. That requires several machines to be meaningful; erasure coding across directories on one disk protects against nothing.

Mirror to somewhere else, on a schedule

Whatever your redundancy story, keep a copy elsewhere. Mirroring is one command and can run from cron.

mc mirror --overwrite --remove local/myapp-media remote/myapp-media-backup

Think carefully about --remove. It makes the mirror an exact copy, which means a deletion — accidental or malicious — propagates to your backup on the next run. For a genuine backup, omit it and rely on lifecycle rules to control growth, so that deleting an object locally does not erase your only other copy.

Versioning and object lock

Versioning keeps previous copies when an object is overwritten or deleted, which turns "someone overwrote the wrong file" from an incident into a two-minute recovery.

mc version enable local/myapp-media
mc ilm rule add --expire-delete-marker --noncurrent-expire-days 90 local/myapp-media

Object lock goes further, making objects immutable for a retention period — even the account that wrote them cannot delete them early. That is the property that makes a backup survive a compromise of the system it is backing up.

Lifecycle rules keep storage from growing forever

Uploads accumulate. Temporary exports, thumbnails, abandoned drafts and old versions all persist unless something removes them.

Express retention as a rule on the bucket rather than as a management command you will forget to run. Expire temporary prefixes after days, non-current versions after months, and incomplete multipart uploads after a week — that last one silently consumes real space and appears in no listing.

TLS on the storage endpoint

If the store is only reachable on an internal network, plain HTTP between application and storage is defensible. The moment browsers fetch objects directly, or the store is reachable across a network you do not fully control, it needs TLS — those requests carry signed URLs, and a signed URL is a credential.

Terminate TLS at your reverse proxy and route to the store internally. That reuses the certificate handling you already have rather than configuring certificates twice.

Performance characteristics worth knowing

Object storage is optimised for whole-object reads and writes, not for partial updates. Listing is comparatively expensive and gets slower as a prefix grows, so avoid designs that list a directory of millions of keys on every page load — keep the index in your database and use the store only to fetch by key.

Many small objects are also less efficient than fewer large ones, since each carries per-object overhead. For thumbnails and similar, that is usually acceptable; for logging individual events as objects, it is not.

When a managed provider is the better answer

Self-hosting makes sense when you need data in a specific jurisdiction, when egress charges dominate your bill, when you are already operating servers competently, or when the store is part of the product you sell.

A managed provider wins when durability matters more than control, when nobody on the team wants to be responsible for a storage system at three in the morning, or when the volume is small enough that the managed cost is trivial. Both answers are defensible; what is not defensible is running a single node and describing it as highly available.

Migrating between providers without downtime

The portability argument only pays off if you have actually rehearsed it. The good news is that a migration is mostly a copy followed by a configuration change.

mc alias set old https://old-endpoint.example ACCESS SECRET
mc alias set new https://new-endpoint.example ACCESS SECRET
mc mirror --preserve old/myapp-media new/myapp-media
mc ls --recursive old/myapp-media | wc -l
mc ls --recursive new/myapp-media | wc -l

Copy while the application still writes to the old store, then run the mirror a second time immediately before switching the endpoint — the second pass moves only what changed and takes seconds. Compare object counts on both sides before you cut over, and keep the old store intact for a few days so a rollback is a configuration change rather than a restore.

Because Django stores only the key in the database, none of this touches your data. The path uploads/2026/invoice.pdf means the same thing to every provider, which is precisely why the abstraction is worth respecting: resist the temptation to store full URLs in your models, or you will rewrite every row during a migration.

Monitoring a store you are responsible for

Running storage yourself means you own its failure modes. Three things are worth watching, and none of them are visible from the application.

Disk headroom first: an object store that fills up returns errors on write, which surfaces in your application as failed uploads with an unhelpful message. Alert at seventy percent, not ninety — freeing space in an object store means deciding what to delete, and that takes longer than you think.

Then the mirror. A backup job that has not run for three weeks is the normal way this goes wrong, so alert on the absence of a successful run rather than on failures alone.

df -h /var/lib/storage | awk 'NR==2 {print $5" used"}'
mc admin info local
find /var/log/mirror.log -mmin -1440 -size +0 || echo "MIRROR DID NOT RUN"

And finally latency from the application's perspective. A store that has become slow — a failing disk, a saturated link — shows up as pages that take seconds to render an image, long before anything reports an actual error. Measuring from the application catches the degradation that a health check on the store itself will happily report as fine.

Summary

The S3 API is the portability layer — design against it and switching providers stays cheap. Run the store on an internal network, never expose it directly, and use per-application credentials scoped to a single bucket rather than root. Keep user content private with short-lived signed URLs and public assets in a separate public bucket. Sign uploads server-side with keys you generate. Be honest that a single node has single-disk durability, mirror elsewhere without propagating deletions, and use versioning and object lock where the data matters. Express retention as lifecycle rules. And choose managed storage without embarrassment when durability matters more than control.