Sudden WU Spike - My Customer is Being Overcharged

georgecollier · 2024-09-11T06:48:32.403Z

randomanon:

How has this not been answered by him this deep into the thread?

Probably because the answer is in there! I hope OP reports back whether it’s a bug or not with details of what actually caused it.

Efe · 2024-09-11T07:11:09.060Z

Sudden WU Spike - My Customer is Being Overcharged Bugs

Have you connected with the support team yet?

Yes, 5 days ago and it’s still under investigation.

boston85719:

The reason for it to have been my suggestion is that I have seen a bug in WUs that has still not been addressed by Bubble.

I’m sorry you’re experiencing this. Creating a bug report and getting results is really a tedious thing. I waste a lot of time trying to explain my problem in full detail. It’s also very disappointing that it’s not resolved.

boston85719:

This seems to be a bug that is causing things to be out of sync with the reality of the app setup. Unfortunately, I have not taken the time to investigate whether an app optimization run will actually delete the backend workflow or not in a similar fashion it does for reusable elements, data types and fields etc.

There must be something wrong here. One of the issues we often face as a team is losing parts of our work during merges. We’ve encountered this many times. For example, we have a page in our app that was updated weeks ago in terms of UI. But when we merge with a newly created branch, the page somehow reverts to its old UI. The page with the old UI doesn’t exist in either the main branch or the merged branch. There seems to be a synchronization issue somewhere.

boston85719:

Again, this is a known bug that I have raised and gotten no resolution to, and I believe recalling other users reporting via forum of this bug previously.

This is exactly why I created this forum post. I’m sure there are other developers suffering from WU bugs, and maybe they are experiencing very similar problems to mine. Contacting Bug Support alone doesn’t seem to enough. I hoped that this forum post could speed up the resolution process for these bugs, and you’ve contributed greatly to this topic. I’m grateful for that.

boston85719:

Likely, the reason for the usage spike, obviously the fact of the bug is the real case, but why such a high usage, is because since deleting the reusable element, the source of the data search was likely unconstrained and therefore searching and returning all entries (as support pointed out) because despite having been deleted, it likely was not deleted from the page in the app it was originally placed on, so the ‘element not found’ notice was likely not visible, as the reusable element itself was placed either in a group not visible or a popup (so on preview can not see the ‘element not found’)…also, probably an issue with the issue tracker not picking up things as consistently as it should, so the ‘app wide’ alert of the deleted reusable element not found on a page was not shown in editor.

I’m a bit confused on this point. My reusable element was only being in one place. The reason I made it a reusable element was to manage it more easily in the editor. The strange part is that this element had been there for months, yet suddenly it started consuming huge amounts of WU. Normally, if an element isn’t drawn on a page (when not visible), it shouldn’t fetch data, as we know. But in this case, I think a bug may have occurred about this thing.

boston85719:

Please make sure you report this as a bug to support.

I’m already in contact with Bubble Support. They are still investigating, but the process is very slow. In the meantime, I’ve already deleted and deployed the buggy reusable element. I hope they don’t respond with “we can no longer investigate because this element is not exist anymore” because I couldn’t sit while losing money, waiting for them to investigate.

Thank you for sharing your thoughts, @boston85719.

I hope we get to see the time when serious things like WU consumption are bug-free.

sem · 2024-09-11T07:49:05.144Z

@Efe

Although I do not have a solution obviously, I can say that we have the same experience as you. In any case we can not fully trust Bubble all the time when merging. We always need to double check. 95% of the time it goes fine. Those 5% where something is not working as it should can hit you hard.

I think a lot of users do not experience this either because their App is not heavily used in the real world or the owner is not measuring lots of things around user behavior, error logging and such so they simply do not see it.

Good to know that we also had issues with undo. We saw weird behavior with undo where after undo not all steps of a workflow were added back to the app. Eventually, after weeks, Bubble support said that they could reproduce the issue and that its behavior can occur in various situations. Since there was no easy fix, they advised only to undo small steps and check.

These small things can add up if you are not familiar with it.

I think the best approach is to make small changes and for each small change save it. If something weird happens you can at least go back to a state that you know.

As for WU’s, I am very happy that we decided to move data processing to Supabase. We started this transition when we got notified of the price change. I feel sorry for everybody still running heavy data processing on Bubble or backend workflows that go haywire.

eren · 2024-09-11T08:11:28.212Z

Hi all,

I believe there are also bugs in WU consumption and data fetching, too… I won’t go into all the metrics now, as it would only cause confusion. Instead, I’ll share the key points I found important within a specific date range.

For those who don’t want to read the entire post, here’s our observation: Bubble’s hidden elements consume data. At least for today, this is the case in my application.

Last 30 Days - 12 Agu to 11 Sep

image1587×747 47.7 KB

You can see the increase, which started in the evening hours of the 5th.

image1563×727 43.8 KB

On this evening, API Workflows were initially responsible for WU consumption. Then, “fetching data” took the lead with a significant increase.

image1873×901 71.9 KB

Now let’s go to the peak time of workload on the same day.

image1648×771 61.4 KB

Upon closer examination, I noticed that the “search” part was causing this.

image1647×771 46.7 KB

I investigated the element causing this issue. The only way this could happen was if the relevant conditions were met, leading to a significant increase in page loads.

Here is the expression:

Expression part 1: It checks if the page is working under a certain domain or vice versa.
Expression part 2: It checks a feature on data directly assigned to a group. If it is active, it searches for an LLM type that matches certain data and becomes visible. This search returns just over 10,000 results.

image866×834 79.9 KB

Is this possible? Yes! If this condition holds and the number of page views increases significantly, my app’s WU consumption will undoubtedly increase.

Let’s conclude by examining the page metrics. I pulled 30 days of data and analyzed it. When my WU usage was overloaded—and it still is—sometimes the page views were even lower.

image1690×634 63.5 KB

After these investigations, we concluded it was a bug and created a report. Note how the dates align closely with when @Efe started experiencing similar issues.

My teammate @Batuhan Merguz, who was investigating this problem yesterday, understood the situation. Once we realized this, the alarming part was that every user interacting with the chatbot on our client’s website was causing the chatbots on other users’ pages to also fetch data!

For instance, if this data stream consumes 10 WU, and 10 users interact with the chatbot simultaneously, it becomes 100 WU for each message. As long as they continue talking, it increases exponentially. If new users join, the situation results in an exponential increase in WU consumption, leading to the explosion in usage we observed.

The repeating group was fetching data even though it was not visible .

Here’s what we tried to resolve the issue:

1- We tried to figure out what went wrong by deleting the condition piece by piece, nope, it kept repeating in every case.
2- We unchecked the condition “visible”. It still keeps fetching data.
3- We completely hid the repeating group, it keeps fetching data.
4- We deleted the repeating group and it stopped fetching data.

We are waiting for the Bubble team to begin investigating, and we are still pulling WU like crazy.

Note: Efe is also my teammate, but he is talking about a different app.

georgecollier · 2024-09-11T08:17:46.630Z

eren:

it searches for an LLM type that matches certain data and becomes visible. This search returns just over 10,000 results.

150 WU in one search

eren:

The repeating group was fetching data even though it was not visible .

eren:

3- We completely hid the repeating group, it keeps fetching data.

Don’t know whether this is the root cause of the issue, but the idea that an invisible repeating group should not load data is a misconception. Making an RG non-visible does not mean that the data will not be fetched. Often, that will be the result, but as soon as you reference the hidden RG from anywhere else in a way that requires the RG’s data, the RG’s data will be fetched. This generally happens when you reference it somewhere that was visible (or some condition in another visible element required data from that RG).

So, for this:

eren:

here’s our observation: Bubble’s hidden elements consume data.

The answer is yes, they do in some cases, and that’s what’s always been the expected behaviour.

Whatever it is, I hope you’re able to get it fixed as nobody wants to be billed loads of WU if they’re not supposed to be. If it’s a bug then let us know if it gets confirmed, and if not… best leave this up as an example!

Efe · 2024-09-11T08:34:50.850Z

sem:

Those 5% where something is not working as it should can hit you hard.

Well, it definitely hits hard.

sem:

I think a lot of users do not experience this either because their App is not heavily used in the real world

I totally agree. We often encounter weird issues and I wonder, “If Bubble is so buggy, why isn’t anyone talking about it?” Then I thought, maybe there are just really a few heavily used apps.

sem:

Good to know that we also had issues with undo.

We have the same.

sem:

I think the best approach is to make small changes and for each small change save it. If something weird happens you can at least go back to a state that you know.

Isn’t that super exhausting? I mean, we’re using Bubble to move fast, but these kinds of situations really make me feel disappointed.

sem:

As for WU’s, I am very happy that we decided to move data processing to Supabase. We started this transition when we got notified of the price change. I feel sorry for everybody still running heavy data processing on Bubble or backend workflows that go haywire.

I’ve also thought about moving to a third-party service for database management, but I haven’t had the courage to start yet. For heavy apps like ours, moving all the data is a big step. When you decided to go with Supabase, did you consider any other alternatives? If so, what made you choose Supabase?

Thank you for your reply,
Best

eren · 2024-09-11T08:37:38.002Z

georgecollier:

150 WU in one search

How did you find this figure?

georgecollier:

Don’t know whether this is the root cause of the issue, but the idea that an invisible repeating group should not load data is a misconception. Making an RG non-visible does not mean that the data will not be fetched. Often, that will be the result, but as soon as you reference the hidden RG from anywhere else in a way that requires the RG’s data, the RG’s data will be fetched. This generally happens when you reference it somewhere that was visible (or some condition in another visible element required data from that RG).

So, for this:

georgecollier:

The answer is yes, they do in some cases, and that’s what’s always been the expected behaviour.

We have already checked the possibilities that I have not referred to elsewhere, those that are mentioned here, and those that are not mentioned here at all. In short, I hope it is understood that I am talking about retrieving data from an element that is not referenced anywhere. Otherwise, when writing a post, we would have to give examples from the entire Bubble manual, which would make it impossible to write a post.

A little advice: Please do not always feel obliged to give advice. There are better ways to provide guidance, such as blogging or appearing on different social platforms.

randomanon · 2024-09-11T09:12:27.786Z

eren:

2- We unchecked the condition “visible”. It still keeps fetching data.
3- We completely hid the repeating group, it keeps fetching data.
4- We deleted the repeating group and it stopped fetching data.

The only time visibility matters for data is in a Popup.

boston85719 · 2024-09-11T10:21:17.905Z

@eren best practice for hidden RG to not load unless necessary is to leave main data source empty. Put conditional onto RG for when data is needed and property to change is data source, and that is where you place search.

ihsanzainal84 · 2024-09-11T17:02:29.292Z

This is why 98% of the time I avoid putting Searches as data sources. I’ve experienced similar issues in the past, when WU first launched and I was learning to optimize for it.

Instead I will load data in a workflow to store into states. To keep things updated I have an expression somewhere in a group or a plugin that will retrieve and store the latest item. This will be a Search but it returns a single item instead. I’ll have some workflow that will detect if the latest item in the list state with that single item does not match then add the latest item into the list. A plugin like Floppy with it’s Expression Watcher has built in actions to trigger events when a value changes so that makes it easier to manage.

I’ve not had any unexpected behaviours in my apps with this approach since I have full control over when I want to load data.

sem · 2024-09-11T18:25:26.567Z

I think that it is exhausting but also that in many cases it is still for many users the best bet. We should not forget that managing complex hard and software ourselves also can mean problems. The feeling is different of course in that case. On the other hand, NoCode and AI is moving fast so more and more other options are very viable as replacement.

As for Supabase, we wanted a large community, backed by serious entrepreneurs/investors, open source, proven technology, full ownership, managed, good pricing structure. Honestly, there are not much other options and Supabase does many things very very good. And for us, it is golden that we can use Postgres as it is meant to be. Oh boy…what a power and what a speed.

We started with a few tables. Db triggers when something changes in Bubble and updated Supabase. Now almost everything is in Supabase. Planning to have data and business rules as much as possible out of Bubble. It will mean we can scale easier at much lower price points and are less vulnerable to any of the companies we do business with

eren · 2024-09-12T06:52:19.353Z

Thank you for your input.

As you suggested, there are dozens of ways to accomplish things. On this page, there are already elements that should hold a single item. Normally, that RG should already fetch the list. Of course, it should do this when it becomes visible.

For us, this is not just a process related to WU. For years, we have been building applications for our own projects and for our clients. In applications with many page loads or workflows, whether WU is present or not, we always had to optimize for high performance. I come from the days when Bubble was very slow, and we had to jump through hoops to show some things correctly(for UX, etc.). In many other applications, the way I used above has been used and is still being used. This usage has not caused any issues either from a WU perspective or a performance perspective.

For example, the most suitable usage to edit the current situation seems to be fetching data with a GET API. This way, Bubble will not make a new call without fully refreshing the page or straining it.

However, we are not applying these optimization techniques that come to mind. We are waiting for news from the Bubble team. Because we believe this is a bug related to data fetching and is important for the sustainability of the platform. Even if it is not a bug, we need to understand which behavior has changed.

You will see in the above metrics that the page load numbers fluctuate between 25K-50K. And WU consumption started to spike after a certain date. Of course, saying we made no changes doesn’t mean much, but it is an old feature, so we know we didn’t touch “that part” and there are no references to it in our release notes. As a less valid proof, I can show the date of Aicado’s GPT’s PH launch, and if you read the post, you will see that our “history” feature has been there at least since that day. We also have an old branch to show to the Bubble team.

image1179×230 18.2 KB

We also noticed a release note that aligns with the start date of overconsumption.

image1736×568 21.8 KB

That’s why I created a post here. Maybe there are people who are going through the same thing as us, and we thought the Bubble team could make it easier to clarify.

georgecollier · 2024-09-12T06:58:58.888Z

eren:

We also noticed a release note that aligns with the start date of overconsumption.

Do your search expressions use unique ID or :each item’s Thing in their expressions?

georgecollier · 2024-09-17T08:14:15.903Z

So, what was the bug? Has it been fixed? Or was it something in this particular app?

Efe · 2024-09-17T09:58:27.929Z

Yes, at first they thought it was a problem with the recursive workflow. Fortunately, they did not insist on it. They said that they started receiving other bug reports about this issue. As a result, the technical team confirmed that it was a bug. They are currently investigating further. I think this post encouraged other developers who had the same problems to create a bug report. So thanks to everyone who contributed to this post.

boston85719 · 2024-09-18T04:38:51.728Z

No worries. Glad it is getting addressed. The post spurred me to create a second report for my issue, which is also getting investigated as it is similar and may be connected. Last I heard from support is it’s being investigated but they are not sure if my issue of deleted workflows still incurring charges is related to your bug of a deleted reusable element still incurring charges.

prograds · 2024-09-19T21:09:28.588Z

Hey @eren . Thanks for sharing this and all the detective work you did and your patience in responding to the ‘experts’ who said it had to be a recursive workflow. If it helps, when I read this post top to bottom and I saw your description and then the response that it had to be a recursive workflow, I was annoyed like you must have been.

Anyway, I had a similar thing happen on July 8. I hadn’t deployed any changes. The same users were doing the same things. I did all that detective work you did, looking at logs, who was logged in, what pages, what data were being fetched, etc. The big consumption was “Individual Data Request”. I did a bunch of (actually useful) optimization, but I still got the WU spike happening from time to time in July.

image2136×702 31.9 KB

It’s happening a lot less now, both frequency and the size of the spike.

I’m also confident that there was a bug causing a bunch of data requests from elements that weren’t ‘active’ on the page.

What you say about deleting a reuseable that wasn’t being used is interesting. I will look at whether that’s involve in my app.

prograds · 2024-09-19T21:19:22.380Z

@Eram_BubbleSupport worked on this for me, but didn’t really find any answers.

Also, Bubble, don’t close this thread after five more days! It’s important!

Efe · 2024-09-19T21:31:31.365Z

Hey @prograds, thank you for your response.

Even though it’s hard to repeat the same explanations patiently, it’s what made this post get so much engagement, and I believe it brought us a bit closer to a solution. Therefore, I’m grateful to all the expert friends.

The reason I deleted the reusable element was that it was the source of WU consumption in the app metrics. I think your case is a bit different. I’m also working with @Eram_BubbleSupport. Hopefully, everything will get even better. soon

boston85719 · 2024-09-21T11:11:24.617Z

I believe they are sending it to engineering for further investigation and fix