Sudden WU Spike - My Customer is Being Overcharged

[Efe]

Hello everyone,

The issue is related to one of my client’s app. The app has had a stable daily WU consumption of 10-20K for months, but starting from September 6th, there’s been a spike in consumption, reaching up to 600K WU per day.

The strange part is: I haven’t done any development or new deployments on this app for weeks. There hasn’t been a significant increase in usage either. Here are the 3-month “page view” and “workflow runs” graphs:

As clearly shown in the image, the usage now is almost the same as it was 3 months ago, or even lower.

I reached out directly to Bubble Support, and after 3 days, they responded as follows:

I can see from your App Metrics that the workload has indeed spiked the last few days, and when looking at the pie charts showing the usage, it primarily stems from four third-party chart plugin elements in the reusable element "Screen - Analytics. When you click into the specific sections, it should take you directly to the element reference itself, and each one is referencing the PotentialCustomers data type, which as over 26,000 live data entries.

The third-party plugin they referred to is the Apex Chart plugin. It has 10.8K installs and a 4.9 rating. If there were an issue with the plugin, thousands of apps would be affected, and the forum would be post about complaints. So, I find this highly unlikely.

They also mentioned the number of entries in the app. However, if that were the issue, there wouldn’t have been a 20x WU consumption spike in a single day. Instead, it would have gradually increased, as this many data entries don’t get created in a single day.

Here is the count of data entry created from 10 Jul 24 to 09 Aug 24

Here is the count of data entry created from 10 Aug 24 to 09 Sep 24

I mean, this problem occurred despite the decrease in data entry growth rate.

Despite everyting, I disabled the “Screen - Analytics” reusable element they mentioned, so that screen is no longer in use. But still, when I check the app metrics, it shows 37K WU consumed in a single hour. When I look into the details, it shows only 3 WU consumed during the same hour. As you can see, there’s a major discrepancy.

Moreover, if I’m not mistaken, all this confusion started with the UI update in the “Logs” tab. So, I think it might be the source of problem.

As a result, my client is being charged unnecessarily every hour because they’re exceeding their WU limit. The app is a critical CRM app for a large organization, so we don’t want to deactivate the WU overage.

Has anyone experienced a similar issue? Or does anyone have an idea of what might be happening?

[georgecollier]

You have a recursive workflow that’s gone haywire. You can start by pausing the scheduler that will prevent any backend workflows from running.

You can debug it using the logs to work out which scheduled workflow is infinitely recurring.

It’s hard to advise any further without seeing the logs. I’m helping people out with WU for free @ Free workload optimisation | Not Quite Unicorns if you want me to take a look…

[Efe]

Thank you @georgecollier for your response. I appreciate your thoughts, but I’m confident in the approach I’m taking with my app. Could you explain why you believe there’s an out-of-control recursive workflow? It only consumes 3 WUs, so I don’t quite agree with your assessment. I’m just trying to better understand the relevance of this point. Were you able to go through the entire post and the details I provided?

[georgecollier]

@Efe

Could you explain why you believe there’s an out-of-control recursive workflow? It only consumes 3 WUs

Sure, it might only consume 3 WUs, but if you run it 600,000 times then you’ll get to 1.8 million WUs, which is what you have.

@Efe

Here you’re looking at the development version.

@Efe

Your logs here show it’s in the live version.

Again, there’s no way to know for certain without looking at your app but you’d struggle to consume that amount of WU without an infinitely recursive backend workflow.

[adamhholmes]

Yeah… without digging into your app it’s impossible to say for sure, but there really are only 2 likely causes here…

Either it’s an out of control, unconstrained recursive workflow (very likely) or a sudden 50-fold increase in traffic/usage (very unlikely, and easy to identify - just check your traffic analytics).

So my money would definitely be on a recursive workflow.

[Efe]

My mistake. I accidentally shared the “Dev” version on the Workload Usage specific graph. Thank you for pointing that out.

I wish the situation were as simple as you mentioned, but unfortunately, it’s not. Here’s the graph for the “live” version.

As you can see, 98.6% of it is from Fetching Data processes. Not from a scheduled workflow.

Scheduled Workflows, on the other hand, consume far less WU.

My recursive workflow simply makes a basic API call. It’s unrelated to my current data, so it’s definitely not running 600,000 times, @georgecollier.

And here, take a look. It’s still consuming a substantial amount of WU.

When I looked into what’s consuming all this WU, I arrived at the following graph:

These graphs are from the “Screen Analytics” reusable element.

However, this reusable element is nowhere to be found because I removed it last night. Yet, it’s still consuming a significant amount of WU.

Do you still thought it’s a recursive workflow? Is your money still on it? @adamhholmes

[georgecollier]

@Efe

image1642×681 52.8 KB

Then it’s likely to be the data searches.

Your WU per search is not high (about 0.3 WU per search, which is more or less as low as it can be) so something’s not adding up.

Again, impossible to tell without looking at the app as it could be so many things. Do you have a ‘Do when’ running every second? Or custom events that are setting each other off and displaying data? What are the data sources of the RGs references in the chart?

[adamhholmes]

@Efe

Do you still thought it’s a recursive workflow? Is your money still on it? @adamhholmes

Well, it’s got to be some kind of recursive/recurring/looping actions/datasource…but again, without seeing inside your app, it can only be a guess.

try and trace back each data retrieval mentioned in the logs.

Also, check the Dev tools to see what’s being loaded to the page.

Then check the server logs.

or feel free to share a link to your editor if you want someone to take a closer look.

[Efe]

Mates, I understand that it’s impossible for you to make a definitive statement without fully reviewing the app. I’m already in contact with Bubble support for this, but unfortunately, the process is moving very slowly, and in the meantime, the app is consuming a significant amount of WU.

Based on the analytics, it’s clear that the WU consumption is coming from the child elements of a reusable element called “Screen - Analytics.” I’ve used the “App Search Tool” to prove that this reusable element doesn’t exist anywhere in the app anymore, yet it continues to consume WU. Does this really make sense to you? Why do you refuse to acknowledge that this is an bug from Bubble?

@georgecollier @adamhholmes

[ihsanzainal84]

Just to clarify, this bug you’re claiming is a bug that causes your app to burn through WU for no reason?

A few more things to clarify:

• So after removing the element in question and deploying to Live, WU is still burning up?

• Have you tried removing the plugin? Some plugins will add headers to your app even with no related elements in the page.

[Efe]

@ihsanzainal84

So after removing the element in question and deploying to Live, WU is still burning up?

• So after removing the element in question and deploying to Live, WU is still burning up?

Yes it is after I remove it from all the pages. A few minutes ago, I’ve directly delete the reusable element from app. And see the results in a few hours.

• Have you tried removing the plugin? Some plugins will add headers to your app even with no related elements in the page.

No, but I use this plugin’s version from the initial version of the app. And as I mentioned it, the plugin is the most popular paid chart plugin that used in 10.8K apps. So, if the plugin cause it, I think we would see another reports on the forum or plugin page.

Thank you

[georgecollier]

@Efe

Does this really make sense to you? Why do you refuse to acknowledge that this is an bug from Bubble?

We’re not refusing to acknowledge it… it’s just not the most likely outcome. We’re trying to help but suit yourself if you don’t want it

[adamhholmes]

@Efe

Why do you refuse to acknowledge that this is an bug from Bubble?

As part of my work, I spend a lot of time investigating/auditing other people’s Bubble apps, often specifically in cases of sudden WU spikes. In 99% of cases it’s caused by developer error, despite how adamant they are initially that it’s not.

I’m not saying it’s impossible that there’s a bug here… there could well be - but I’ve got a lot of experience with this, and it’s almost never a bug.

But if you’re in talks with Bubble support already, then I guess that will confirm whether or not there’s a bug in this case.

But in the meantime, did you check your server logs yet? or your dev tools? Those will shed some more light on what’s going on.

There’s not really much more help anyone else can give if you’re not willing to share your app editor, so see what those things show, and I guess you’ll just have to wait for Bubble support to get back to you.

[Efe]

Thank you guys for all your replies.

Sudden WU Spike - My Customer is Being Overcharged

We’re not refusing to acknowledge it… it’s just not the most likely outcome. We’re trying to help but suit yourself if you don’t want it

Please don’t get me wrong. I truly appreciate your efforts to help, and your insights are very valuable to me. I’m just confused as to why there’s so much focus on a recursive workflow when the metrics I’ve shared clearly point to the root of the issue. I’m genuinely trying to understand why you continue to think this is the cause.

As part of my work, I spend a lot of time investigating/auditing other people’s Bubble apps, often specifically in cases of sudden WU spikes. In 99% of cases it’s caused by developer error, despite how adamant they are initially that it’s not. - @adamhholmes

You’re absolutely right. A misconfigured recursive workflow can lead to serious issues, and inexperienced developers may not realize this. However, I’ve been professionally developing Bubble apps for over 5 years, and I regularly experiment with WU consumption, sharing what I learn on X. I’ve also learned a lot from your replies on oher topics.

In summary, I’ve tried to explain my issue as clearly and in as much detail as possible. Even so, I’m struggling to understand why, despite all the information I’ve provided, there’s still such a strong focus on the possibility of a recursive workflow error.

When you encounter a serious issue, you know how challenging it can be to create a bug report and communicate with Bubble support. The process moves very slowly, and it takes a long time for them to understand and acknowledge the problem. That’s why I created this forum post, to be able to find a quicker solution.

[Efe]

For those interested, the root of the issue was in the fetching data actions within the child elements of a reusable element called “Screen - Analytics.”

This reusable element had been consuming a normal level of WU (daily 5-20k) for months, but four days ago, it suddenly started consuming 20x more WU (reached up to 600k). As I mentioned in my previous reply below, I removed this element from all pages.

Sudden WU Spike - My Customer is Being Overcharged

My mistake. I accidentally shared the “Dev” version on the Workload Usage specific graph. Thank you for pointing that out. I wish the situation were as simple as you mentioned, but unfortunately, it’s not. Here’s the graph for the “live” version. [image] As you can see, 98.6% of it is from Fetching Data processes. Not from a scheduled workflow. [image] Scheduled Workflows, on the other hand, consume far less WU. My recursive workflow simply makes a basic API call. It’s unrelated to my cu…

However, I noticed that despite being removed from all pages, it was still consuming WU. At this point, I suspect that somehow, even though this reusable element wasn’t visible or existing anywhere in the app, it was still triggering the data fetching process. But I have no idea how that was happening.

I then deleted the reusable element from the app entirely, and after doing so, WU consumption returned to normal.

Although WU consumption is back to normal, this doesn’t mean the issue is fully resolved because I had to remove a crucial feature from my app. For that reason, I will continue to work closely with Bubble Support to get further assistance.

So my money would definitely be on a recursive workflow. @adamhholmes

I think you might want to reconsider that! (Just joking, I absolutely see your point.)

Thanks again, everyone!

[georgecollier]

@Efe

I suspect that somehow, even though this reusable element wasn’t visible or existing anywhere in the app, it was still triggering the data fetching process. But I have no idea how that was happening.

Then it was definitely still somewhere in your app (perhaps you hadn’t deployed the change that really removed it from everywhere).

We’re trying to help but have very little to go on, so a recursive workflow is the most likely culprit, followed by a Do every X seconds, followed by a nested search, or some combination of all three.

@Efe

I’ve been professionally developing Bubble apps for over 5 years

For what it’s worth, years of experience doesn’t really count for anything on Bubble. There are people around that have been doing Bubble for six months that are better than people who’ve been around since 2014!

@Efe

the root of the issue was in the fetching data actions within the child elements of a reusable element called “Screen - Analytics.”

So what expression were you using? Were you using display data / display list actions anywhere on your page? It evidently caused 40,000 searches in one hour alone, so something must’ve been triggering that.

[fede.bubble]

Have you connected with the support team yet?

[randomanon]

@georgecollier

So what expression were you using?

How has this not been answered by him this deep into the thread?

[boston85719]

@Efe

I then deleted the reusable element from the app entirely, and after doing so, WU consumption returned to normal.

While reading through all the posts in this thread, this was going to be my suggestion. Glad you got to it.

The reason for it to have been my suggestion is that I have seen a bug in WUs that has still not been addressed by Bubble. I had a backend workflow, consuming a small amount of WUs which totaled around 1.8 WUs per day for an API call that was not necessary. I deleted the backend workflow, yet it still shows as running and consuming WUs every day. This is in development and when I look at the WUs it is in the Development.

This seems to be a bug that is causing things to be out of sync with the reality of the app setup. Unfortunately, I have not taken the time to investigate whether an app optimization run will actually delete the backend workflow or not in a similar fashion it does for reusable elements, data types and fields etc.

Again, this is a known bug that I have raised and gotten no resolution to, and I believe recalling other users reporting via forum of this bug previously.

Likely, the reason for the usage spike, obviously the fact of the bug is the real case, but why such a high usage, is because since deleting the reusable element, the source of the data search was likely unconstrained and therefore searching and returning all entries (as support pointed out) because despite having been deleted, it likely was not deleted from the page in the app it was originally placed on, so the ‘element not found’ notice was likely not visible, as the reusable element itself was placed either in a group not visible or a popup (so on preview can not see the ‘element not found’)…also, probably an issue with the issue tracker not picking up things as consistently as it should, so the ‘app wide’ alert of the deleted reusable element not found on a page was not shown in editor.

Please make sure you report this as a bug to support. I will be having to report again about my ‘deleted workflow’ still running and getting charged. The more users reporting about the buggy behavior around WUs the better and faster it will start to function appropriately all the time.

[georgecollier]

@randomanon

How has this not been answered by him this deep into the thread?

Probably because the answer is in there! I hope OP reports back whether it’s a bug or not with details of what actually caused it.