[Seeking Alpha testers] A/B testing via Google Optimize

Announcements
1

Hi Bubble community,

We are recruiting volunteers for an Alpha test for a new feature: A/B testing via Google Optimize! (As a PM, I can barely contain my excitement about this feature!)

A/B testing is a core tactic used by tech companies to experiment with different variations of their website to see what converts users better. If you’ve ever tried a third party A/B testing tool with your Bubble app, you may have noticed that it doesn’t work very well - this is because of the rather unique way that Bubble renders pages. Trust us - we tried a couple tools ourselves and weren’t satisfied!

So, we set out to build this feature in a way that would work with Bubble apps. Rather than build a full-blown native A/B testing solution (maybe someday in the future, but we have enough on our plate!), we chose to integrate with Google Optimize, which is part of the same family as Google Analytics.

Here are some details about this feature:

  • It’s effectively a plugin, i.e. it’ll be found in the plugin gallery when it’s launched
  • It enables you to use “Get data from an external API” to get the current variant assignment for the given user for the given experiment. This allows you to create conditionals like “If the user’s experiment assignment is variant 1, show this element”
  • Google Optimize relies on Google Analytics for metrics. This means that in order to use this, you must also be piping data from your app to Google Analytics (e.g. via the Google Analytics or Segment plugins)
  • Currently, it only allows you to run 1 A/B test at a time; you supply the experiment ID in the plugin’s settings right next to your API key

This feature will only be available to apps on the current Professional plan and above, so only those apps will be eligible for the Alpha.

If you’re interested in participating, please send me a DM with the ID of the app you want this feature on.

Thanks!
Allen

(Friendly reminder: it’s up to you to make sure that using Google Analytics and Google Optimize are allowed in your app’s Terms)

14 Likes
2

That would be helpful to have optimize for bubble apps. I don’t have any app build yet, otherwise would have applied for this.

3

This post should have more love - THIS IS HUGE!! Can’t wait :slight_smile:

4

Hi there, how does this feature (will) works if user is blocking GAnalytics via adblocker or vpn ? On certain tech savy populations that goes ~30% of traffic

5

Also, what is the time frame for Beta or production level service ?

6

We don’t do anything special to try to get around adblocking. We’re relying on an integration with Google Analytics / Optimize and optionally Segment here.

In terms of timeline, we’re making a couple adjustments based on testing, so hopefully will be ready for a Beta in a few weeks (with A/B testing especially, we’ve noticed more things as we’ve run actual A/B tests ourselves, which unfortunately can take a little while!)

7

Ok, but does the plugin show some signal abouth this kind of behaviour/information (users blocking to enter the experiment) ? Maybe just same ratio of split (and i guess statistically neutral results) is done with blocking Ganalytics users ?

8

The way we’ve built it out, if a user has an ad-blocker, the randomized assignment of that user into a variant will still happen (i.e. they still have the same chance of seeing the variant), but none of the data about that user will flow through to the analytics tools (because ad-blockers block analytics tools, in general). Ad-blockers should impact both control and variant(s) equally, given it’s a randomized assignment.

1 Like
9

Hey @allenyang can we test this feature as well? :slight_smile:

10

Hey @allenyang

Do you have any timescale of when this is coming out of closed alpha? Cheers

11

I’m going to guess we’ll be ready for Beta next week or the week after - if this current round of QAing goes well

1 Like
12

Hi @allenyang - did this go-live?

13

Yes - here’s the documentation.

Closing this thread to avoid future confusion!