How to Compare Pages Before and After a Website Migration

A redesign, a new CMS or a new domain can look right in the browser and still lose what search engines read: hand-written titles replaced by a template, a canonical tag that points at the staging host, a noindex left over from staging, FAQ markup the new theme drops, a page that lost half its text. Migration checklists say to compare each old page with its new one, which by hand means two tabs, two inspections and two screenshots per page.

This guide does it in two runs of a Python script. Before launch, it saves each old page's title, meta description, H1, canonical and robots meta tags, structured data, word count and a full-page screenshot. Once the new pages are up, it captures each one, compares it with the saved page and writes a report with both screenshots side by side. A curl loop then checks the redirects.

We ran it on 10 November 2026 on our own pages: two blog articles stand in for old pages, the copies we republished on dev.to for their new ones, and /agents/ is an address we did move. The script compares the pages you list; it is not a crawler.

Compare one page in the browser, free

1. Open site-shot.com and paste the old page's URL. Capturing is free and needs no signup.
2. Tick full page, and leave No ads / No cookie ticked; it is on by default.
3. Press Shot and click the download arrow under the picture (its tooltip says Download) to save the PNG. Do the same for the new page.
4. To read the tags, open each page in your own browser, right-click it, choose Inspect, and search the Elements panel (Ctrl+F, or Cmd+F on a Mac) for <title, name="description", rel="canonical", name="robots", <h1 and application/ld+json.

The browser tool shows each page; it does not compare them or read their tags. A list of pages is a job for the API.

Save the old pages before launch

You need Python 3.9 or newer, the Site-Shot SDK and an API key. Install the SDK:

pip install site-shot

For the key, a confirmed account's API key comes with the free plan: 100 API captures a month, no card needed; paid plans start at $5/mo for 2,000. The API quickstart gets you one. Each row costs two captures, one before and one after; each further after run, such as a staging pass, costs one more.

Put the list in pages.csv, one row per old page and the address it moves to:

old,new
https://www.example.com/services/web-design.html,https://www.example.com/services/web-design/
https://www.example.com/about-us.php,https://www.example.com/about/
https://shop.example.com/products/blue-mug,https://www.example.com/shop/blue-mug/

This is the redirect map a migration needs anyway; a page that keeps its address goes in both columns. The first line must name the two columns, old and new, or the script stops before any capture; keep old first, because the curl loop below reads the cells in order. Blank lines are skipped. A CSV downloaded from Google Sheets works as it is, and so does one saved from Excel as CSV UTF-8 with commas between the columns.

Save the script as migration_check.py in the same folder:

"""Compare every page on pages.csv before and after a site migration.

python migration_check.py before   # while the old pages are live: save each one
python migration_check.py after    # once the new pages are up: compare, write the report
"""
import base64
import csv
import html
import json
import re
import sys
import time
from html.parser import HTMLParser
from pathlib import Path
from urllib.parse import unquote, urljoin, urlsplit

from site_shot import SiteShot, SiteShotError

client = SiteShot()  # reads SITESHOT_API_KEY from the environment
OUT = Path("migration")
BASELINE = OUT / "before.json"
SETTINGS = dict(
    width=1280, height=800,
    full_size=True, max_height=2400,
    no_cookie_popup=True, no_ads=True,
    fresh_session=True, source_code=True,
    markdown=True,
)
# Writes the address the browser ended up on, after any redirect, into the page source.
FINAL_URL = "document.documentElement.setAttribute('data-final-url', location.href);"
MIN_TEXT = 0.8  # flag a new page with fewer than 80% of the old page's words
NOINDEX = {"noindex", "none"}  # robots values that keep a page out of search results
FIELDS = (("status", "Status"), ("title", "Title"), ("description", "Meta description"),
          ("h1", "H1"), ("canonical", "Canonical"), ("robots", "Robots"),
          ("types", "Structured data"), ("words", "Words"))


def same(a, b):
    """Whether two addresses name the same page, ignoring the case of scheme and host."""
    def parts(url):
        p = urlsplit(url if "//" in url else "https://" + url)
        return p.scheme.lower(), p.netloc.lower(), unquote(p.path or "/"), p.query
    try:
        return parts(a) == parts(b)
    except ValueError:  # a malformed address, such as "https://[bad"
        return a == b


def types_in(data):
    """Every @type in a JSON-LD block, nested ones included."""
    found = set()
    if isinstance(data, dict):
        kinds = data.get("@type") or []
        found |= {str(kind) for kind in (kinds if isinstance(kinds, list) else [kinds])}
        data = list(data.values())
    if isinstance(data, list):
        for item in data:
            found |= types_in(item)
    return found


class Page(HTMLParser):
    """What a migration should keep, read from the page's rendered HTML."""

    def __init__(self):
        super().__init__(convert_charrefs=True)
        self.title = self.description = self.canonical = self.final_url = None
        self.robots, self.h1, self.types = set(), [], set()
        self.in_body, self.inside, self.text = False, None, ""

    def handle_starttag(self, tag, attrs):
        attrs = dict(attrs)
        name = (attrs.get("name") or "").lower()
        if tag == "html":
            self.final_url = attrs.get("data-final-url")
        elif tag == "body":
            self.in_body = True
        elif tag == "meta" and name == "description" and self.description is None:
            self.description = " ".join((attrs.get("content") or "").split())
        elif tag == "meta" and name in ("robots", "googlebot"):
            self.robots |= set((attrs.get("content") or "").lower().replace(",", " ").split())
        elif tag == "link" and "canonical" in (attrs.get("rel") or "").lower().split():
            self.canonical = self.canonical or (attrs.get("href") or "").strip()
        elif self.inside is None and (
                tag == "h1" or (tag == "title" and not self.in_body) or (
                    tag == "script"
                    and (attrs.get("type") or "").strip().lower() == "application/ld+json")):
            self.inside, self.text = tag, ""

    def handle_data(self, data):
        if self.inside:
            self.text += data

    def handle_endtag(self, tag):
        if tag != self.inside:
            return
        text, self.inside = self.text, None
        if tag == "title":
            self.title = " ".join(text.split()) if self.title is None else self.title
        elif tag == "h1":
            self.h1.append(" ".join(text.split()))
        else:  # a JSON-LD block
            try:
                self.types |= types_in(json.loads(text))
            except ValueError:
                self.types.add("(unreadable JSON-LD)")


def capture(url, image):
    """Capture one page, save its screenshot as OUT/image and return its fields."""
    reply = client.capture_json(
        url,
        javascript_code=FINAL_URL,
        **SETTINGS,
    )
    (OUT / image).write_bytes(base64.b64decode(reply["image"].split(",", 1)[1]))
    page = Page()
    page.feed(reply.get("source_code") or "")
    words = None
    if reply.get("markdown_status") == "ok":
        words = len(re.findall(r"\w+", reply.get("markdown") or ""))
    return {
        "status": reply.get("response", {}).get("status_code"),
        "source": reply.get("source_code_status"), "final_url": page.final_url,
        "title": page.title, "description": page.description, "h1": page.h1,
        "canonical": page.canonical, "robots": sorted(page.robots),
        "types": sorted(page.types), "words": words, "cut": bool(reply.get("truncated")),
        "image": image, "checked": time.strftime("%Y-%m-%d %H:%M UTC", time.gmtime()),
    }


def compare(old, new, address):
    """What to look at on the new page, against the saved old one."""
    notes = [] if new["status"] == 200 else ["status %s" % new["status"]]
    if new["source"] != "ok":
        return notes + ["no page source (%s)" % new["source"]]
    landed = new["final_url"] or address
    if not same(landed, address):
        notes.append("lands on %s" % landed)
    if NOINDEX & set(new["robots"]) and not NOINDEX & set(old["robots"]):
        notes.append("the new page asks search engines not to index it")
    try:
        canonical = urljoin(landed, new["canonical"]) if new["canonical"] else ""
    except ValueError:  # a malformed address, such as "https://[bad"
        canonical = new["canonical"]
    if canonical and not same(canonical, landed):
        notes.append("canonical points to %s" % canonical)
    elif not canonical and old["canonical"]:
        notes.append("no canonical tag")
    for key, label in (("title", "title"), ("description", "meta description"), ("h1", "H1")):
        if new[key] != old[key]:
            notes.append(("%s changed" if new[key] else "no %s") % label)
    lost = sorted(set(old["types"]) - set(new["types"]))
    if lost:
        notes.append("structured data lost: %s" % ", ".join(lost))
    if old["words"] is None or new["words"] is None:
        notes.append("no text to count")
    elif new["words"] < MIN_TEXT * old["words"]:
        notes.append("text is %d%% of the old page's" % (100 * new["words"] / old["words"]))
    if old["cut"] or new["cut"]:
        notes.append("Markdown cut at its size limit")
    return notes


def rows():
    """(row number, old address, new address) for each line of pages.csv."""
    with open("pages.csv", newline="", encoding="utf-8-sig") as f:
        reader = csv.DictReader(f)
        reader.fieldnames = [(name or "").strip().lower() for name in reader.fieldnames or []]
        if not {"old", "new"} <= set(reader.fieldnames):
            raise SystemExit("pages.csv needs the header old,new")
        return [(n, (row.get("old") or "").strip(), (row.get("new") or "").strip())
                for n, row in enumerate(reader, 1)]


def before():
    """Save every old page that loaded and that before.json does not hold yet."""
    saved = json.loads(BASELINE.read_text(encoding="utf-8")) if BASELINE.exists() else {}
    taken = {page["image"] for page in saved.values()}
    (OUT / "before").mkdir(parents=True, exist_ok=True)
    for n, old, new in rows():
        if not old or old in saved:
            continue  # no old address, or saved by an earlier run
        if "before/%02d.png" % n in taken:
            raise SystemExit("row %d would overwrite a saved page's screenshot: keep earlier "
                             "rows in place and add new ones at the end" % n)
        try:
            page = capture(old, "before/%02d.png" % n)
        except SiteShotError as error:
            print(n, old, "capture failed: %s" % error)
            continue
        if page["status"] != 200 or page["source"] != "ok":
            print(n, old, "not saved: status %s, page source %s"
                  % (page["status"], page["source"]))
            continue
        saved[old] = page
        BASELINE.write_text(json.dumps(saved, indent=1), encoding="utf-8")
        landed = page["final_url"] or old
        print(n, old, "saved" if same(landed, old) else "saved; already lands on %s" % landed)


def show(value):
    """A field's value as report text."""
    if isinstance(value, list):
        value = ", ".join(value)
    return "(none)" if value in (None, "") else str(value)


def card(page):
    """One report cell: the screenshot, when it was taken and every field."""
    if not page:
        return "<td></td>"
    fields = "".join("<br><b>%s</b>: %s" % (label, html.escape(show(page[key])))
                     for key, label in FIELDS)
    return "<td><a href='%s'><img src='%s' width='300'></a><br>%s%s</td>" % (
        page["image"], page["image"], page["checked"], fields)


def after():
    """Capture every new page, compare it with its old page and write the report."""
    if not BASELINE.exists():
        raise SystemExit("run 'python migration_check.py before' first")
    saved = json.loads(BASELINE.read_text(encoding="utf-8"))
    (OUT / "after").mkdir(parents=True, exist_ok=True)
    cells = []
    for n, old, new in rows():
        if not old and not new:
            continue  # a blank line
        old_page, new_page = saved.get(old), None
        if not new:
            notes = ["no new address"]
        elif not old_page:
            notes = ["no saved old page"]
        else:
            try:
                new_page = capture(new, "after/%02d.png" % n)
            except SiteShotError as error:
                notes = ["capture failed: %s" % error]
            else:
                notes = compare(old_page, new_page, new)
        check = "; ".join(notes) or "OK"
        print(n, new or old, check)
        cells.append("<tr><td>%d</td><td><a href='%s'>%s</a><br>&rarr; <a href='%s'>%s</a></td>"
                     "<td>%s</td>%s%s</tr>" % (
                         n, html.escape(old), html.escape(old), html.escape(new),
                         html.escape(new), html.escape(check), card(old_page), card(new_page)))
    (OUT / "report.html").write_text(
        "<meta charset='utf-8'><table border='1' cellpadding='6'><tr><th>Row</th>"
        "<th>Old page &rarr; new page</th><th>Check</th><th>Before</th><th>After</th></tr>"
        + "".join(cells) + "</table>",
        encoding="utf-8",
    )


command = sys.argv[1] if len(sys.argv) > 1 else ""
if command == "before":
    before()
elif command == "after":
    after()
else:
    raise SystemExit("usage: python migration_check.py before|after")

Run the first half with the key in the environment. Typing it at a hidden prompt keeps it out of your shell history; this works in bash and zsh:

printf 'Site-Shot API key: '; read -rs SITESHOT_API_KEY; echo
export SITESHOT_API_KEY
python migration_check.py before

The script runs on your own computer and writes into a migration folder next to it: each old page's screenshot in migration/before/, named after its row, and its fields in migration/before.json. Only pages that loaded are saved; a status other than 200 or missing HTML prints not saved, and a failed capture prints capture failed. Running before again captures only the rows not saved yet. Because the screenshots are named after their rows, add new pages at the end of pages.csv between runs and clear a row's cells instead of deleting it; the script stops rather than write over a saved page's screenshot. To capture a saved page again, delete its entry from before.json.

The line says already lands on when the old address already redirects, so the saved page is the one it lands on. Before launch, the address on your list is probably out of date. After launch, the old page is gone and the row will compare the new page with itself.

Compare the new pages

When the new pages are up, run the second half:

python migration_check.py after

It captures each new address, compares it with the saved old page, saves the screenshot in migration/after/ and prints a line per row. Then it writes migration/report.html: a row per page with the check result and both screenshots side by side, each with the time it was taken and every field underneath. Click a screenshot to open the full PNG.

To check staging before launch, put the staging addresses in the new column and run after. After launch, copy migration/report.html and migration/after/ aside if you want to keep the staging report, change the addresses to the live ones and run after again; before.json stays as it is.

What each setting does

• source_code=True returns the page's HTML in the reply's source_code field, as the browser rendered it, so tags added by the page's own scripts are there too; source_code_status is ok when it came back. Every field except the status and the word count is read from it.
• markdown=True adds the page's text as Markdown in the markdown field, converted from the same rendered page: its one <main> element, otherwise the whole body, without navigation and search landmarks, with links reduced to their text. The script counts its words. markdown_status is ok when the text came back, and truncated means it was cut at the size limit.
• javascript_code runs your JavaScript in the page before the screenshot is taken and before the HTML is read. Here FINAL_URL copies location.href, the address the browser ended up on after any redirect, onto the page's <html> tag, because the reply itself does not say where the page landed.
• full_size=True with max_height=2400 captures the page from the top down to 2,400 pixels. The HTML and the Markdown are not cut at 2,400 pixels, but the picture is; raise max_height in SETTINGS to see more of every page.
• no_cookie_popup=True and no_ads=True are the two blockers behind the browser tool's No ads / No cookie box; fresh_session=True starts each capture without the cookies and local storage that earlier captures saved.

Which rows to look at

The Check column says OK, or lists what to look at on the new page:

• status …: the new page answered with a status other than 200, such as 404 for a page that was never built. A 504 usually means the capture ran out of time.
• lands on …: the new address redirected elsewhere, such as the home page, a sign-in page or the address with a trailing slash. The redirect map should name where the page ends up.
• the new page asks search engines not to index it: a robots meta tag on the new page says noindex or none, and the old page's did not. On staging that is expected; after launch it is the classic leftover. The report's Robots line shows every value, nofollow included. A noindex sent in an X-Robots-Tag response header is not in the HTML and does not show here; curl -sI on the new address shows it.
• canonical points to …: the new page's canonical tag names another address, often the old domain or the staging host. On staging, a canonical that names the future live address is normal; after launch it should name the page itself. no canonical tag: the old page had one.
• title changed, meta description changed, H1 changed, or no title, no meta description, no H1: compare the two values in the report. A rewrite can be deliberate; the same title on many new pages is a template default.
• structured data lost: …: JSON-LD types the old page had and the new one lacks, nested ones included, so a lost FAQ reads Answer, FAQPage, Question.
• text is …% of the old page's: the new page has fewer than 80% of the old page's words. The count covers only the part of the page the API converts (see markdown=True above), so a template that adds or drops a <main> element, or puts more text inside it, moves the number; a page at half has usually lost a section. MIN_TEXT sets the line. no text to count and Markdown cut at its size limit mean the text did not come back or was cut.
• no page source (…): the reply had no HTML for the new page, so its fields could not be read.
• no saved old page, no new address or capture failed: …: the row was not compared.

A bot check, consent wall or error page on either side changes the title, the H1 and the text at once, so look at the screenshots before you blame the migration.

Check the redirects after launch

This step needs no API and costs no captures. After launch, each old address should answer with a permanent redirect, 301 or 308, to its new address. Google follows the temporary ones, 302 and 307, too, but does not treat them as a sign that the new address should replace the old one. This loop reads the same pages.csv and prints, for each old address, the status of its first answer and where it ends up:

tail -n +2 pages.csv | tr -d ' \r' | while IFS=, read -r old new _ || [ -n "$old" ]; do
  [ -n "$old" ] || continue
  code=$(curl -s -o /dev/null -m 30 -w '%{http_code}' "$old")
  landed=$(curl -s -o /dev/null -m 30 -L -w '%{url_effective}' "$old")
  if [ "$landed" = "$new" ]; then result=OK
  elif [ "$landed" = "$old" ]; then result="does not redirect"
  else result="lands on $landed"; fi
  echo "$code $old $result"
done

A row ends in OK when the old address ends up on the row's new address; the code in front of it should still be 301 or 308. A 404 with does not redirect is an old address nobody redirected, and 000 means curl got no answer. The loop compares addresses as text, so write each new address in full, with https:// and the slash after a bare host name, as in https://www.example.com/. It reads the first two cells of each row, not quoted ones.

What the run returned on 10 November 2026

In a real migration most of these notes would need a fix. A canonical tag that points back at the old page is right for a republished copy and wrong for a moved one: it tells search engines to keep the old address. Neither copy carries our FAQ structured data, and the Python copy is the April version of a tutorial that has grown since. The third row is a baseline taken after a move: /agents/ has redirected to /ai-agents/ since June.

The redirect loop on the same rows printed 200 and does not redirect for the two articles, which have not moved, and 301 https://www.site-shot.com/agents/ OK.

Where the honest limits are

• The old pages must be saved while they are live. After launch, the old addresses show the new pages or nothing, so keep the migration folder until the new site has settled.
• Each page is read once, as a new visitor. Each capture runs in a clean browser, 1,280 pixels wide, not logged in, so pages that change with location, A/B tests or a login can differ for reasons the migration did not cause. Google indexes the mobile version of a page; for a mobile pass, run a copy of the script with width and user_agent in SETTINGS set as in the sitemap guide and OUT set to another folder, such as Path("migration-mobile"), so the two baselines stay apart.
• The capture follows redirects but does not report them. The script reads only the final address, and the curl loop only the first status code.
• A list, not a crawler. The script checks the rows on your list, not pages you left out, internal links, images, hreflang tags, the XML sitemap or robots.txt. Crawlers do whole sites: Screaming Frog's SEO Spider, for one, compares two crawls with a paid licence and can map a staging site's addresses to the live site's.
• The word count is a prompt, not a verdict. A page can keep its count and still lose its key paragraph, so read the pages the report flags and spot-check some that pass.
• The same HTML is not the same ranking. The script compares what the pages say. Search Console's page indexing report and URL Inspection tell you what Google has indexed.
• Staging behind a login page or an IP allowlist is out of reach. The API renders pages as an anonymous visitor from the internet.

FAQ

What should I compare before and after a website migration? For each old address and its new one: where the old address redirects, the new page's status, its title, meta description and H1, its canonical and robots meta tags, its structured data, how much text it has, and how it looks. The Site-Shot API can return a full-page screenshot, the rendered HTML and the page's text as Markdown in one JSON reply (source_code=1 and markdown=1 with response_type=json), so a script can collect all of that for every page on a redirect map.

Can I compare a staging site with the live site? Yes, if staging is reachable from the internet: save the live pages first, then capture the staging addresses and compare them. A canonical tag that names the future live address is normal on staging; after launch it should name the page itself. Staging behind a login page or an IP allowlist is out of reach of an outside capture service such as Site-Shot.

Does Site-Shot follow redirects? Yes. A Site-Shot capture follows redirects as a browser does and returns the page it lands on, but the JSON reply does not include the final address or the redirect chain. A javascript_code snippet that copies location.href into the page records the final address, and curl shows the status code of the first answer, such as 301 or 302.

Why does the word count change when the article text is the same? Because Site-Shot converts only part of the page to Markdown: the main element when the page has exactly one, otherwise the whole body, in both cases without navigation landmarks. A new template that puts other text inside that part, or adds or drops a main element, changes the count. A drop to half the old count usually means a section is missing, and the two screenshots show where.

Related reading

• How to Screenshot Every Page of a Website (From Its Sitemap) — a record of the whole old site.
• How to Verify Backlinks and Screenshot Them for a Link-Building Report — another check on the rendered HTML.
• How to Schedule Automatic Website Screenshots and Keep a Dated Archive — a dated log of key pages after launch.

Comparing one page? Do it free — no signup — at site-shot.com. For a list of pages, get an API key from the API quickstart or see the paid plans on the pricing page.