Dynamic sitemaps, SEOs, and empty pages

[Banjaman]

Does anyone know what is the best way of dealing with dynamic pages that have been deleted in a Bubble app but are still being crawled by Google and returning soft 404 errors? I have several pages in my app that are linked to a ‘thing’ (eg, user profiles, blog posts, job listings, etc). Those ‘things’ are sometimes created and deleted by users (for example, a user deleting their profile). The problem I’m having is that if I ask Bubble to ‘expose a sitemap file’ under the SEO settings, all of those dynamic pages are added to my sitemap and crawled by Google. That’s a great thing as long as the page is live, but once it’s deleted, Google continues to crawl those pages and adds a ‘Soft 404’ error. The graph below from the Google Search Console shows that the number of soft 404 errors in my site keeps growing, which is bad for SEO:

Bubble has already identified this issue in its SEO documentation:

Watch out!
If an earlier version of your sitemap contains a page with a Thing assigned to it (e.g. mybubbleapp.com/product/great-widget-1), and later that Thing is deleted, Google may still list that page online even after you upload an updated sitemap. This is because regardless of what your most recent sitemap says, Google will try to visit old URLs to see if they’re still alive. Since a Bubble page will still technically load even if the assigned Thing doesn’t exist (it just doesn’t show any Thing’s data), Google will think that old URL is still alive."

But there is no recommendation on how to solve this problem.

One solution I thought about is to redirect users to my 404 page whenever a page associated with a ‘thing’ is empty on loading (see below). That is probably better than having soft 404s, but it’s still not great from an SEO perspective:

Another solution that has been suggested is to turn off the ‘expose a sitemap file’ option under the SEO tab in Bubble’s settings and upload a sitemap manually. The problem with that solution is that dynamic pages created by users that could be quite useful for SEO are no longer added to my sitemap (and it’s not really feasible for me to add them manually every time).

Does anyone know if there’s a good solution to this issue? Thanks!

[adamhholmes]

You can include a noindex tag on the page when the Current Page User is empty - so when Google next crawls that URL, if the Page User is empty, it will remove it from their index.

[Banjaman]

Interesting, thanks. Is there a way to do that dynamically (without having to identify each page)? And would you mind explaining how? Thanks!

[adamhholmes]

Just add it to the Page HTML header using a Boolean expression, formatted as text…

e.g.:

Current Page User is Empty: formatted as text:

For Yes: <meta name="robots" content="noindex">

For No:

[Banjaman]

Thanks a lot, I’ll give it a try! In case it helps others with the same problem, I implemented Adam’s solution as follows: