How to Make Blog Post Crawlable?

Anyone confirmed for a fact if dynamic content appears in chatgpt or google?

Again, we know for a fact that static content can be picked up. My own site appears in chatgpt using static content directly built into the page’s text elements.

@asked111

What would qualify as factual evidence? What qualifies as in Google? What qualifies as in chatGPT?

From my experience dynamic content does appear in Google and chatGPT.

I made a test for you to try but because the idea was to find the magic hidden password in the pages, some of the AI’s refused to do it for security reasons thinking it’s looking for actual passwords for naughty reasons , so i’ll tweak the test slightly and repost in a bit)…

Ok here is a quick test for you guys to try…

On each of these pages is a “Magic word of the day”. One is dynamic and the other static. See if you can get the ai to scan the content and tell you what the magic word of the day is on each:

Dynamic:
https://hitvocals.co.uk/blog_post/singer-vocal-layering-tips

Static:
https://hitvocals.co.uk/static_test

I’ll do my own tests later with various LLMs and would be interested to know what results you get too.

1 Like

Great test, but when using ClaudeAI, both pages returned completely empty content. AI can not render no code pages because they hide text behind Java. I will now update on my extensive work over the last 2 weeks..

I’ve been working on a service to make Bubble apps visible to AI search engines (ChatGPT, Perplexity, etc.) by serving pre-rendered HTML to AI crawlers.

What I built:

Apify integration to capture fully-rendered Bubble pages (this worked)
Cloudflare Workers to detect AI bots and serve clean HTML (this worked)
Dashboard for managing URLs (fully completed and tested)

Why it won’t work:
Sadly I have decided to not pursue this project because the solution requires customers to:

1.Transfer their domain nameservers to Cloudflare
2.Manually configure DNS records
3.Install Worker code

For non-technical Bubble users, this is too complex and risky. Asking someone to change their domain’s nameservers is a huge barrier - one mistake could break their site or email.
The core problem: You can’t intercept traffic to someone’s domain without controlling their DNS.

My reasoning:

Initially I wanted to solve this pain point for myself - I have a blog I wanted to make AI-searchable. Then I realized there could be a huge market (300+ million Bubblers with a similar pain point).

But my gut tells me that anyone willing to configure DNS records to make their Bubble pages searchable will likely decide it’s easier to build SEO content on a normal non-Bubble site and simply link to their main app - this is something I’ve done successfully with a separate Bubble app.

If anyone has ideas for a simpler approach, go for it! Personally, I’m shelving this after ~30 hours of work :frowning:

Thank you for going through this - i have actually gone through the exact same steps and have set up cloudflare workers to serve html version of some pages (slightly different approach to yours) both for pageload speed and also for ai crawability, but likewise i then just though - why am i going through all this hassle :melting_face:- bubble should implement this natively somehow - I am sure they could.

For me the pageload speed aspect and the ai search crawl-ability are the only negative aspects of bubble that i can not defend or workaround easily and it does bug me. Give me these fundamental things over ai prompt building any day!! All we can do I think is appeal to the company and hope they listen…

Very ironic also that Cloudflare has currently just gone down :sweat_smile::grimacing:

1 Like

Yes, I agree it is a lot of hassle and I think maybe Bubble can’t solve this issue, it must be a big challenge for them or they would have surely solved it?

The way I see it now is that if I want my site to “rank” high, or have any chance of being mentioned in AI search results, then I have to pay for another website which can be visible by AI, so Bubble is going to cost me more.

Haha, I was working on Cloudflare this morning and then the crash happened and I literally wondered if I had broken the internet - oops! lol

Neither of these work through chatGPT. All cited JS as the reason why. Interestingly, my own page appears, but the LLM says that it can only see the page’s “meta/preview text.” Basically it doesn’t see any of the content unless you use the SEO headers/structured data markup. I’m guessing this is why @boston85719 sees his dynamic content in search results/AI. I think he made a plugin to insert dynamic content into the structured data (although I’m not sure why you wouldn’t be able to do this without a plugin).

You can do it without a plugin. I had been using Bubble settings on the page since 2020 adding structured data directly via page header.

I do have a plugin Data Jedi that has lots of structured data tools. Part of the reason is to reduce the need for adding the same type of structured data I might or might not need conditionally on different pages across the app. You can add directly in the app editor in settings area structured data as well, but that can not be turned on or off conditionally. The elements in the plugin also make it so I can inject structured data conditionally so as to support SEO for Single Page Applications.

From my experience using structured data directly in the header, you can only go 2 levels deep with inline conditionals in attempts to get good SEO for an SPA, so my plugin elements solve that problem for me too. And especially for SPAs, you can only use the ‘current page thing’ or ‘do a search’ for the values necessary for Structured data in page header on page, so my plugin solves for that.

I also like to use a lot of structured data, especially FAQs now that AI is around, since AI does not have built in browsers, so they can not really browse websites and see all content on page (I think the AI companies themselves are aiming to solve that and might already have rolled out something for it), the structured data is most important thing to get AI to pick up your dynamic content, and FAQs add a lot of dynamic content.

It is also important to be able to conditionally turn on or off things like Reviews on a product page, because if you inject Review structured data on a dynamic product page and that product has no reviews, that is bad for SEO and can get penalized by Google, so a big need to make conditionally injecting or removing structured data based on the actually ‘things’ data.

I also have a plugin element that solves the issue of Bubble not reading URL path lists well or at least easily. In fact, just yesterday I updated the element URL Extractor in the Data Jedi Plugin to make it even more versatile in extracting the URL path lists based on any text field of the data type, so it is not just slug or unique ID but can be the email, name or any other text based field you want in the URL, and as it works for all 4 types of data I use in Bubble (custom types, option sets, api objects, and plugin objects), it makes it really useful. It even autodetects the URL and website details so that if you choose to conditionally inject breadcrumbs or website schema, you don’t need to worry, and it autodetects the correct values to inject.

Part of the reason as well for the plugin use is that I build flexible, maintainable and extendable apps using a Global Data Center, and having all the data right where all the structured data gets injected from, just makes building great apps easier.

1 Like

Hi mate,

Can you give me a web URL for a page you believe is searchable by AI? I would be interested to test it.

Don’t want to be the bearer of bad news but I tested add content in headers/structured data markup and that fails too. The only reliable solution I found was to serve the scrapped HTML to the search engine before it hits your webpage.

This is not an adequate test. You need to ask an AI to go to the URL link and tell you exactly what text is on the page. Also, you should update the text so that we know the AI is not reading from Google because sometimes it does, but other times is fails (unreliable).

The goal for me was to:

1. ensure the AI’s can read all the latest text

2.provide a AI first schema of the page text in question/answer format (which AI’s love!)

This fails with bubble pages, all the content is hidden behind js. I tried a few things and could not get AI crawlers to access the content on the page. So gave up.
FYI, Gemini is able to access a bit of content. You can try with that.

1 Like

Have you tested with an html built site to verify an ai chatbot is accessing the site and not reading google? Or changing structured data in bubble to see if ai can read the newest structured data?

For me, all I care is AI can when queried reference my site page, since most people only use AI as search engine anyway. I’m not needing AI to provide full content of site. I just want AI to give traffic to site via references.

Can confirm, Gemini sees my entire page, not just the stuff in the page header. I’m assuming Google’s crawler is just superior compared to whatever OpenAI is using since they’ve been in the game for decades.

on bubble app? structured data and all content not in structured data?

Gemini (even v3) can’t see the bubble test pages I made above.

My guess is that if there was an indexed version that had previously been crawled by the Google bot then it would have picked up & used that (dated) version as a fallback. Fine for some cases.

1 Like

Yeah, Bubble app. No structured data, plain text fields.

I think that’s what’s going on. I submitted it to google search console a long time ago, it’s been indexed for sure.

1 Like