Build production RPA bots in Python: log into vendor portals, scrape dashboards, fill forms, and persist sessions — orchestrated from Django via Celery. The modern, license-free alternative to UiPath and Automation Anywhere.
A surprising amount of business still runs on humans clicking through websites — downloading reports from a portal that has no API, re-keying data between systems, checking a supplier's site for stock. Robotic Process Automation is the practice of letting software do that clicking, and with Python and Playwright you can build commercial-grade automations integrated directly into your Django app, at zero licensing cost. This tutorial covers RPA done right: reliable browser automation, async workflows, scheduling from Django, and the failure-handling that separates a robust bot from a fragile script.
Robotic Process Automation means automating the repetitive, rule-based tasks people perform through user interfaces — the work that exists because two systems do not talk to each other and a human bridges them by clicking. RPA bots drive a browser or application the way a person would: navigating, filling forms, clicking buttons, reading results. It is the pragmatic answer when there is no API to integrate against and you cannot change the system on the other side. The value is enormous in back-office processes, data entry, report gathering, and reconciliation, where automating a daily hour of manual clicking frees real time and removes human error. Playwright is the modern engine for doing it in Python.
Playwright is a browser automation library that drives real browsers — Chromium, Firefox, WebKit — programmatically. Compared to older tools like Selenium, it is faster, more reliable, and far less flaky, largely because of how it waits for the page. It handles modern JavaScript-heavy sites well, runs headless for servers, and has a clean Python API including full async support.
pip install playwright
playwright install chromium
Its reliability is the key selling point: the single biggest problem with browser automation is flakiness — bots that work one run and fail the next — and Playwright's design directly attacks that, which is why it has become the default choice for serious automation.
The number-one cause of flaky automation is timing: the script tries to click something before it has appeared or become interactive. Older tools forced you to litter code with manual sleeps and waits, which were both slow and unreliable. Playwright's locators auto-wait — before acting on an element, it automatically waits for the element to be present, visible, and actionable, up to a timeout.
page.get_by_role("button", name="Download report").click() # waits, then clicks
This single feature eliminates most flakiness. You express intent — click this button — and Playwright handles the timing, so you stop writing fragile sleeps and your automations become dramatically more dependable across runs and varying network conditions.
How you find elements determines how durable your automation is. Brittle selectors tied to a page's exact structure break the moment the site changes its markup. Playwright encourages robust locators that find elements the way a user would — by their visible role and text, their label, their placeholder — which are far more stable than CSS paths or generated class names. Preferring get_by_role, get_by_label, and get_by_text over fragile structural selectors means your bot keeps working through cosmetic site changes. Locator strategy is where the long-term maintainability of an automation lives: good locators survive redesigns, brittle ones generate a stream of breakages every time the target site is touched.
Most useful automations operate behind a login, and logging in fresh on every run is slow, fragile, and likely to trip security measures. Playwright lets you save the authenticated browser state — cookies and storage — after logging in once, and reuse it on subsequent runs so the bot starts already signed in. This makes runs faster and far more reliable, and avoids repeatedly hammering a login flow that may have rate limits or CAPTCHAs. Managing authentication state deliberately — log in once, persist it securely, refresh it when it expires — is a core pattern for production RPA, turning a slow and brittle re-login on every run into a quick resumption of an existing session.
Browser automation is heavily I/O-bound — most of the time is spent waiting for pages to load and respond — which makes it a natural fit for async. Playwright's async API lets you run multiple browser contexts concurrently, so you can process many items in parallel rather than one slow sequence:
async with async_playwright() as p:
browser = await p.chromium.launch()
# run several isolated contexts concurrently
Each browser context is an isolated session, like a separate incognito window, so parallel automations do not interfere. Bounding the concurrency so you do not open too many browsers at once or overwhelm the target site, async lets a bot that would take an hour sequentially finish in a fraction of the time.
Running RPA inside your Django app means the data the bot gathers flows straight into your models, and the bot can use your existing business logic. A bot that downloads a supplier's stock levels can write them directly to your database; one that submits forms can be driven by records in your app. This integration is the advantage of building RPA in the same Python codebase rather than as a separate tool: there is no export/import boundary, no second system to maintain, and the automation is governed by the same code, models, and configuration as the rest of your application. The browser automation becomes just another part of your Django project.
Automations are rarely run by hand — they run on a schedule (gather the report every morning) or in response to events (a new order needs submitting to a supplier). Celery is the natural home for this: an automation is a background task, scheduled with Beat for recurring runs or triggered on demand, executed by a worker off the request cycle. This keeps long-running browser sessions out of your web requests and gives you retries, monitoring, and concurrency control for free. Pairing Playwright with Celery is the production pattern: Celery decides when and how often the bot runs and handles its failures, while Playwright does the actual browser work within each task.
Automations fail partway — a page times out, the site changes, the network drops — so design them to be safely re-runnable. An idempotent automation can run again without duplicating work or causing harm: use update_or_create rather than blind creation so a re-run updates rather than duplicates, and track progress so a resumed run can skip what it already did. This matters because RPA interacts with external systems you do not control, where partial failure is common. Designing every workflow so that re-running it is safe turns a failed run from a data-corruption incident into a simple retry, which is essential when the bot's reliability depends on systems outside your control.
External sites are unreliable, and robust automation expects failure rather than assuming success. Wrap steps so a failure is caught, logged with context, and handled — retried with backoff for transient issues, escalated for persistent ones. Respect the target: if a site returns a rate-limit response, back off rather than hammering it. The difference between a script someone wrote once and a production automation is almost entirely in this failure handling — the script assumes the happy path and breaks at the first hiccup, while the production bot anticipates timeouts, missing elements, and site changes, and degrades gracefully instead of crashing or, worse, doing the wrong thing silently.
When an automation fails on a server, you cannot see what the browser saw, which makes debugging hard — unless you capture evidence. Playwright can take a screenshot at the moment of failure and record a full trace of the run, a step-by-step recording you can replay to see exactly what happened. Capturing a screenshot on every failure, and a trace for runs that go wrong, transforms debugging from guesswork into reviewing a recording. This is indispensable for headless server automation, where the bot's view is otherwise invisible. Building this evidence capture in from the start means that when a run fails at 3am, you wake up to a screenshot and a trace showing precisely where and why, not a bare error message.
Automating someone else's website raises real questions you must consider. Respect terms of service, which may prohibit automated access; honor robots directives and rate limits; do not overload a site with aggressive requests; and be careful with data you collect, especially anything personal. Automating your own systems or those of partners who have agreed is straightforward; scraping or driving third-party sites without permission can breach contracts or laws. RPA is a powerful capability, and using it responsibly — within the rules of the systems you interact with, at a respectful request rate, with proper handling of any data gathered — is part of doing it professionally rather than recklessly. The technical ease of automation does not remove the obligation to use it appropriately.
For a handful of automations integrated with your Django app, plain Playwright plus Celery plus your models is simpler and entirely sufficient — everything stays in one codebase under your control. If you grow to dozens of bots needing shared infrastructure — an orchestration UI, audit logs, a secrets vault, business users editing workflows — a dedicated RPA framework or platform starts to earn its keep by providing that scaffolding. The decision mirrors build-versus-buy elsewhere: start with the lightweight, code-first approach that keeps you flexible and cheap, and adopt heavier tooling only when the scale and organizational needs genuinely exceed what a few well-written tasks provide. Most teams need far less than the enterprise RPA platforms market would suggest.
The hardest reality of automating third-party sites is that they change without warning, and a change can break your bot overnight. Beyond using robust role- and text-based locators, build in resilience: detect when an expected element is missing and fail loudly with a clear diagnostic rather than proceeding blindly, monitor your automations so a breakage is noticed immediately, and structure code so adapting to a site change touches one place. Some teams add a verification step that confirms the page looks as expected before acting. Accepting that target sites will change, and engineering so that changes produce clear failures and quick fixes rather than silent wrong behavior, is central to running RPA in production.
Many sites actively resist automation with CAPTCHAs, bot detection, and rate limiting, and how you handle this is partly technical and partly ethical. Where you have permission to automate — your own systems or a partner's — work with them to allowlist your bot or use a provided API path. Where a site deliberately blocks automation, that is often a signal you should not be scraping it, and circumventing protections may breach terms or law. Technically, respecting rate limits and behaving like a considerate client reduces friction, but the honest position is that aggressive anti-automation measures are a boundary to respect, not always an obstacle to defeat. Automate where you are welcome to.
As automation needs grow from one bot to many, scaling becomes a real concern. Browser automation is resource-heavy — each browser instance uses meaningful memory and CPU — so running many in parallel requires capacity planning. Containerized browsers, a pool of workers, and bounded concurrency let you scale throughput without overwhelming either your infrastructure or the target sites. Running automations in containers also solves the dependency problem, since a prebuilt browser-automation image has everything installed. Thinking about how your automation workload scales — how many browsers run at once, on what infrastructure, against what rate limits — is what lets a handful of useful scripts grow into a dependable automation platform.
An honest assessment of RPA includes its ongoing cost: because bots depend on interfaces that change, they require maintenance in a way that API integrations do not. A site redesign can break a bot, and you must fix it. This is the inherent tradeoff of automating through a user interface rather than a stable API — you gain the ability to automate where no API exists, but you take on the burden of keeping up with interface changes. Budgeting for this maintenance, monitoring bots so breakages surface fast, and preferring an API wherever one becomes available are part of using RPA realistically rather than treating it as a build-once-and-forget solution.
RPA automates the repetitive, UI-driven work that exists because systems lack APIs and humans bridge the gap by clicking, and Python with Playwright lets you build it at commercial quality for free. Playwright's auto-waiting locators are the key to reliability, eliminating the timing flakiness that plagues browser automation, while robust role- and text-based locators keep bots working through site changes. Persist authentication state to make runs fast and stable, use the async API to process many items concurrently, and integrate directly with Django so gathered data flows into your models. Schedule and run automations through Celery for recurring or event-driven execution with retries and monitoring, design every workflow to be idempotent and resumable so failure means a safe retry, and handle errors gracefully with backoff and respect for the target. Capture screenshots and traces so headless failures are debuggable, stay within the legal and ethical bounds of the systems you automate, and reach for a full RPA platform only when scale truly demands it. Done this way, RPA turns hours of manual clicking into reliable, integrated, self-running automation.