Hi friends 👋🏻

Planning a big website change usually starts with knowing what's on your site now. This week, I'll show you a couple of ways to speed up the task of generating a content list to start with.

This is part of a series in which I'm writing about the process of refreshing the Blue Hills Digital website.​ I introduced the project in my Digital Landscape email on May 20. If you missed it, read that email here!

Want links to recent emails or a sign-up link to share? Go to The Digital Landscape sign-up page​

---

Plan changes, track changes.

A big website refresh or migration will impact every URL on the website.

• Even for a project that doesn't change the structure of a website (URLs stay the same, changes are just design/visual) you're going to want to check each page template or separately-constructed page to ensure nothing has broken.
• And for a more complex project that involves migrating to a new platform or moving content around, every URL will need careful attention to make sure your redirects work, that search engines can still find all that previously-indexed content, and that users can still navigate the website.

For these reasons, I recommend starting a project by wrapping your head around all the content that's currently on your website.

I've been doing that over the past week for the Blue Hills Digital site. This refresh is a combination of the two examples I gave above:

• I'm changing the design in some areas, which means each of the pages on the site will need to be adjusted. For example, the updated design uses a header and navigation without the current dark background, which means the hero/title section of each page will need to be adjusted, since most currently start with a matching dark background hero section.
• I'm also moving content around. I'm moving free and paid product offerings (mostly digital downloads) into a separate Resource content type, which will mean archiving and redirecting some pages, and inserting new dynamic promotions for those resources into existing articles and guides.

To help plan these changes, and track the changes as they are implemented, I need a list of URLs on the site in a Google Sheet, so I can sort them into categories based on what kind of updates they need.

Generating a list of website content

Here are four options:

1. Use a site crawling tool

Site crawlers like Screaming Frog or Sitebulb will attempt to crawl every URL at your domain, similar to Google. They do this by following links on the site, so so long as every page can be found by following links, you'll get a complete list. You can also add the URL for your sitemap, which might help the tool find additional pages that aren't linked.

Pros: you can get lots of additional columns of data in your report, and even connect to GA4 or Search Console to pull in additional data points per URL.

Cons: the tools can be complicated to use for the first time, and the crawl may not pick up URLs that are not linked and marked with a noindex tag, so you can't be sure every single URL will be included.

2. Use your sitemap file

Most websites include an XML Sitemap file, which is an automatically-generated list of files that are available for indexing. Platforms like Squarespace generate this automatically. On WordPress, you'll need an SEO plugin like Yoast to generate the sitemap.

In my case, the Yoast plugin generates a sitemap for each content type on the website, so there are four sitemaps available here: https://www.bluehillsdigital.com/sitemap_index.xml

This means I can grab the Page sitemap and download it to my computer. Catch: it's an XML file, so not ready to import to my Google Sheet yet. Save: You can quickly convert an XML file to a CSV with ChatGPT. Just use this prompt:
"Convert xml sitemap file to a downloadable csv" and upload the XML file

Pros: Quick and easy, especially with ChatGPT's help to convert the output to a CSV.

Cons: Might involve extra steps if your website platform isn't generating a sitemap already. And remember that the sitemap will only include URLs that are available for indexing. So any pages with a noindex tag will not be included – you'll have to add those another way. You also get limited content here: the CSV will only include columns for page title, URL, and a couple of other datapoints.

3. Use your website platform's export tool

Availability varies by platform. WordPress includes a basic "Export" tool that spits out an XML file, which you could convert to a CSV as described above. Squarespace does something similar, although with some content limitations.

Pros: You'll get all the pages using this approach, regardless of index or noindex status, and often full page content too.

Cons: The output is in XML format, so it will need converting.

4. Use a custom plugin

This option only applies if you're working with an advanced website platform like WordPress or Drupal. In those cases, there are third-party plugins specifically for exporting content that allow you to completely customize the export file content and format. If you really need full control, I recommend the WP All Export / All Import plugins for WordPress sites. They're excellent plugins, and I use them for every migration I work on that involves WordPress.

Pros: Full control, super customizable

Cons: Complex, and only available for more advanced website platforms.

Here's a preview of the Pages list I generated for the Blue Hills Digital website. My next step is to work this list and flag pages that:

• Can be archived
• Need a redesign to match the updated site header
• Need content edits to match the updated brand (less corporate, more me)

Stay tuned for more updates in the coming weeks 👋

---