Why aren't my product pages appearing in search?

I have a price comparison website that uses Bubble as the frontend and Xano as the backend. Ever since moving my products from Bubble’s DB to Xano’s DB I’ve not been able to get those product pages to appear in Google search.

Here is a visual example of what is happening:

On this page you can see a list of products which is a Repeating Group that pulls in external data via an API:

If my products were hosted on Bubble, the user would click on a product and then I would simply do an internal link to the page “/product” and pass data on to that page to populate it.

However, since my products are on an external DB, the way I currently have it set up is to direct the user to an “external page” to which I then customize the URL to an ID number that matches the product ID in my Xano database:

So in this case, the product I clicked on takes me to the page techaudit.tv/product/40. The page knows to make an API call looking for the product details of ID #40 and then displays the data sent back. Works great!

The problem? That page (and all other product pages) won’t show up in search. You can see that I have the /product page in my sitemap and is just like how I had it set up when the products were pulling from my Bubble DB:

image

So what is the deal? Is it not possible to have a page that relies on an external DB to be crawled? If I were to say build an app to track stock prices and pulled from an API, would I not be able to have a page for Apple, Google, Netflix, etc simply because the data is external? There has to be a way around this that I’m missing…

By Search do you mean Google Search? Or other search engines like it.

If so take a look a setting up data types from Xano for Schema. I had to Rebuild my entire database to setup Schema so that pages would show up in google searches properly.

Also you Schema would go in the product HTML header. Heres a good reference tool https://www.google.com/webmasters/markup-helper/u/0/

1 Like

Yes Google search, thanks for clarifying that (I’ve updated my original post to reflect that).

Here is a quick update that may add to the discussion:

When I run a site: search on Google to see what is indexed, I can see that some product pages actually do show up in search, but it’s only a handful of them.

Is it possible that the only reason those pages are ranking is because they were shared elsewhere on the web (therefore allowing Google was able to crawl them)?

If so, this seems to tell me that when Google’s crawler hits the product search page it doesn’t actually have a way to access those individual product pages, likely due to them being dynamic links, but then again, it seems no different than when I was dynamically directing to a product page from Bubble’s DB…

Perhaps a solution would involve me populating all of the thousands of product pages via a custom sitemap documentation that syncs directly with all of the product ID’s on Xano…

I’d imagine you have a Google search console setup so you can take pages in copy and paste them here


page

You can use this to start to see where you SEO breaks down

I personally use Links for all my Text & Buttons to set up internal database links. Also using Url reformatting so instead of Unique ID’s or Slugs using the product name. This has made google pretty happy with my site. I’m not sure how much this might differ with Xano. But doing all of this I’ve had Google crawl the majority of my product pages.

I’m curious, is there a reason you’re going with xano for backend rather than bubble? Since it’s just an e-commerce site it may be better to run it all through bubble.

The thing is I have about 12,000 products in my DB so the only realistic way would be to populate my sitemap with all of the specific product URLs, right?

I’m guessing that is what Bubble was doing for me when the products were internal (auto populating the sitemap with the product data type slugs). So perhaps I need to run an export of my Xano DB with all of the product URLs and then load them into a custom sitemap.xml file that I can either try to sync with Bubble or just go directly to Google Search Console…

@chris.williamson1996: To answer your question, the reason I moved to Bubble for this is because I’m pulling prices from a ton of different API sources and then need to do transformations to that data to sync the prices to the right products, use the nessesary crytopgraphy to access some of them, get all the data cleaned up and looking uniform, etc and doing that multiple times a day across 10k+ products is just not something Bubble is ideal for whereas that is Xano’s bread and butter.

1 Like

Yeah, that could definitely help. It’s always a good idea to give Google as much data as you can about your website. In the past, I had issues with pages getting “sandboxed” for months in Bubble. However, I found that using proper Schema markup and formatting with H1, H2, and other relevant tags helped resolve this issue. So because your using Xano might need to do a mixture of both of these things if you havent. Out of curiosity under Indexing in Google Search console where it says “Pages” how many pages has it reported Indexed?

Consider putting ld-json on your pages to describe your products. Especially if you can load that content from Xano, and avoid WU.

https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data