Playwright for Beginners: Automate Any Website with Python

Why Playwright?

Here's a common frustration: you write a Python script to scrape a website, but the data doesn't show up. The page loads content with JavaScript after the HTML arrives. Your requests + BeautifulSoup script sees an empty shell.

Enter Playwright — a browser automation library built by Microsoft. Unlike requests (which just downloads HTML), Playwright opens a real browser. It clicks buttons, fills forms, waits for JavaScript to finish, and grabs the fully-loaded page.

In 2026, Playwright has officially surpassed Selenium in adoption. It's faster, has a cleaner API, and comes with batteries included: auto-waiting, built-in screenshot and video recording, and a code generator that records your clicks.

If you're new to web scraping in general, start with our BeautifulSoup web scraping guide — it covers the basics of parsing HTML. Then come back here when you hit a page that requires JavaScript.

Installation

pip install playwright

# Download the browsers (Chromium, Firefox, WebKit)
playwright install

That's it. You get Chromium (Google Chrome's engine), Firefox, and WebKit (Safari's engine) — all real browsers, not simulations.

Your First Script: Open a Page and Take a Screenshot

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)  # headless=False to see the browser
    page = browser.new_page()
    page.goto("https://byteleveltech.com")
    page.screenshot(path="screenshot.png", full_page=True)
    print(page.title())  # "ByteLevelTech — Learn AI tools, Linux, Python..."
    browser.close()

Run this and you get a full-page screenshot saved to your disk. Five lines of code. Try doing that with requests.

headless=True means the browser runs invisibly in the background. Set it to False during development so you can watch what Playwright is doing.

Core Patterns You'll Use Every Day

Clicking Buttons

# By visible text (best method — survives UI redesigns)
page.get_by_role("button", name="Submit").click()

# By CSS selector (fragile — breaks if the class name changes)
page.locator(".btn-primary").click()

# By text content
page.get_by_text("Learn More").click()

Playwright encourages user-facing locators — "the button that says Submit" — because they match how humans find things on a page. CSS classes change; a button's label rarely does.

Filling Forms

# Text inputs
page.get_by_label("Email").fill("hello@example.com")
page.get_by_placeholder("Enter your name").fill("Alice")

# Checkboxes
page.get_by_label("I agree to terms").check()

# Dropdowns
page.get_by_label("Country").select_option("Japan")

# Submit
page.get_by_role("button", name="Sign Up").click()

Waiting for Content

Here's where Playwright shines. In Selenium, you'd write time.sleep(3) and pray the page loaded. Playwright auto-waits before every action — it pauses until the button is actually clickable, the input is visible, or the text appears.

For dynamic content that appears after an action:

# Click a button, then wait for the result to load
page.get_by_role("button", name="Load More").click()

# Wait for specific text to appear
page.get_by_text("Showing 50 results").wait_for()

# Or wait for an element to be visible
page.locator(".search-results").first.wait_for(state="visible")

No more sleep(). No more guessing how long to wait.

Extracting Data

# Get all product titles from a listing page
titles = page.locator(".product-card h3").all_text_contents()
for title in titles:
    print(title)

# Get text from a specific element
price = page.get_by_test_id("price").text_content()

# Get attribute values (like image URLs)
img_url = page.get_by_role("img", name="Product photo").get_attribute("src")

Taking Screenshots

# Full page
page.screenshot(path="full-page.png", full_page=True)

# Just the visible viewport
page.screenshot(path="viewport.png")

# A specific element
page.locator(".chart-container").screenshot(path="chart.png")

Real-World Example: Scrape a JavaScript-Powered Search Page

Let's tie it all together. This script: 1. Opens a job listing site 2. Types a search query 3. Clicks Search 4. Waits for results to load 5. Extracts all job titles and company names

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()

    # Go to the job board
    page.goto("https://example-jobs.com")

    # Search for Python jobs
    page.get_by_placeholder("Job title or keyword").fill("Python developer")
    page.get_by_role("button", name="Search").click()

    # Wait for results — the page loads them with JavaScript
    page.get_by_text("jobs found").wait_for(timeout=10000)

    # Extract all job cards
    cards = page.locator(".job-card")
    print(f"Found {cards.count()} jobs:\n")

    for i in range(cards.count()):
        card = cards.nth(i)
        title = card.locator("h2").text_content()
        company = card.locator(".company-name").text_content()
        print(f"  {title} — {company}")

    browser.close()

Run this and you get structured data from a page that requests couldn't touch. The key difference? Playwright ran the JavaScript, waited for the results, and then read the fully-rendered page.

The Code Generator: Record Clicks, Get Code

Playwright has a built-in tool called codegen that records your browser actions and writes the Python for you:

playwright codegen https://example.com

A browser window opens. Click around — fill a form, navigate, hover. Playwright watches and generates working Python code in a side panel. It's the fastest way to prototype a script when you're not sure what selectors to use.

Playwright vs BeautifulSoup: When to Use Which?

Playwright is more powerful but slower — it has to launch a real browser. For static pages, BeautifulSoup is the right tool. But when you hit a wall with JavaScript, Playwright is the door you walk through.

Automating Logins

Many useful scripts need to log in first. Playwright makes this painless:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)  # Show browser for login
    context = browser.new_context()
    page = context.new_page()

    # Log in once
    page.goto("https://example.com/login")
    page.get_by_label("Email").fill("you@example.com")
    page.get_by_label("Password").fill("your-password")
    page.get_by_role("button", name="Log In").click()
    page.wait_for_url("**/dashboard")

    # Save the logged-in session
    context.storage_state(path="auth.json")
    browser.close()

# Later: reuse the saved session — no login needed
with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    context = browser.new_context(storage_state="auth.json")
    page = context.new_page()
    page.goto("https://example.com/dashboard")  # Already logged in!
    # ... do your automated work ...
    browser.close()

Security note: auth.json contains your session tokens. Add it to .gitignore immediately. Never commit authentication files.

Common Gotchas and Fixes

"Element not found" when the element is right there. The page probably hasn't finished rendering. Use page.wait_for_selector(".my-element") or page.get_by_text("...").wait_for() before interacting.

Playwright is slow compared to requests. Yes — it launches a real browser. For bulk static-page scraping, use BeautifulSoup. Reserve Playwright for pages that actually need a browser.

The site detects me as a bot. Add a realistic viewport and user agent:

browser = p.chromium.launch(headless=False)
context = browser.new_context(
    viewport={"width": 1920, "height": 1080},
    user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
)
page = context.new_page()

And be respectful — don't hammer a server with requests. Add page.wait_for_timeout(2000) between actions.

Next Steps

Playwright is the Swiss Army knife of browser automation. Once you've mastered the patterns in this guide — launch, navigate, click, fill, extract — you can automate almost any web task.

Here's what to build next:

• Price tracker: Check a product page daily, email yourself when the price drops. Use Playwright for the scraping and our Python email automation guide to send the alerts.
• Form filler: Automate repetitive data entry. Record your workflow with playwright codegen, tweak the output, and never fill the same form twice.
• Screenshot monitor: Take daily screenshots of a dashboard and back them up automatically.
• The official docs: playwright.dev — the Python API reference is excellent and well-organized.

The best way to learn? Pick one annoying website task you do manually, and automate it this week.

All code in this article was tested and runs successfully on Python 3.8.10 (Ubuntu 20.04) with playwright 1.48.0 — verified August 2026.