WU bugs - overcharged

[boston85719]

We need more community engagement on uncovering WU bugs

In one simple test, to verify if using recurring WFs is less costly than using schedule api workflow, I did a simple setup and test.

Setup below

1. Simple recurring trigger - No Conditions

2. Simple send email action - No conditions

3. Simple Set recurring action

Please keep in mind for the following, there are no conditions on any of the actions or triggers in this simple setup/test

This is what the WU metrics shows as being charged for this

1. Button click to set recurring action - Cost overall is 1.22 WUs for one run - this seems to be 0.62 more than it should be since, a button click trigger is not supposed to be charged, and there is only one actions, which should be only 0.6 WUs, and there is no indication in the charts that data retrieval (ie: individual data request took place, which, if it did should only cost 0.2 WUs) - so, where is 1.22 WUs coming from?

2. Running the recurring Action overall shows 0.2 WUs for fetching data (the individual data request, which is correct) and the 1 run of the actual backend recurring which is listed as 1.24 WUs, which is strange as there is just one action (ie: send email) which should just be 0.6 WUs.

3. Running the action shows evaluation of conditions (those conditions that are empty and do not exist so should not need to be evaluated)

Nowhere in the Bubble manual is there a reference to either conditions that do not exist, needing to be evaluated and therefore are charged, nor is there a section that details how recurring workflows are charged.

Based on the public details available, I would have expected to be charged only for the below plus the return of a individual data request and the data in terms of bytes…so something like 0.95 WUs, if conditions that do not exist even need to be evaluated, which they shouldn’t, so in reality, maybe 0.90 WUs, but not the 2.46 WUs the metrics show as having been charged.

@fede.bubble is Bubble devoting any of it’s own resources to uncovering bugs in the WUs other than relying on the user base to uncover and then notify of such?

Any other developers who can spare 10 minutes of their day to do the same simple test, can you confirm if you get the same results?

@Eram_BubbleSupport could you have the team look into this?

[Jici]

One thing is that you forget to calculate the fetching of Current user, but this if far from the difference that you have.

Will try to test later

[boston85719]

@Jici

One thing is that you forget to calculate the fetching of Current user,

I do not think I did, but believe Bubble did in the charts…that is the individual data request charged at 0.20 WUs I’ve included in the expected calculation…unless the ‘current user’ is not charged as an individual data request.

@boston85719

I would have expected to be charged only for the below plus the return of a individual data request and the data in terms of bytes

[boston85719]

Had to submit a bug report about this, as testing again today, I see another issue, which is that one first day of testing was charged only for 1 search, but on second day of testing was charged for 2 searches, even though nothing in the setup was changed.

I couldn’t imagine why it needed to fetch data two times on the second day of testing since it only needs to retrieve the current user at the time the recurring WF is set, or why it needed to search two times if the first day it only needed to search 1, although it could be that on first day of testing the chart was broken into two different bars, while on second day is was a single bar, so maybe the first day, the 2 searches were just split between the two bars…either way, still confused on why the seemingly overcharged amounts, and why there needs to be two searches to find the single item of current user in the one single place it is used as a dynamic expression…maybe we are charged for when we set the recurring WF and when the recurring WF runs?

[mghatiya]

It is really complex to debug all these WUs for different scenarios and then apply learnings to optimise, test etc. Takes away a lot of time and energy.

We all have spoken so much about why the WU based system is quite faulty. There are bound to be bugs in this, but they still continue with this system.

After a point we all need to think of how we utilise our times as well. For me now Bubble is just a way to create MVP to test, iterate etc and not serious long term apps, because that’s what they want us to do. In my one app that I have seriously built, I have got locked in because it is way to complex and not easy for me to move out. But I wish I could.

[boston85719]

An update to this thread based on the findings via Bubble support.

The first response from support as an ‘explanation’ of the issue to the 0.63/0.64 WUs charged for a ‘conditional evaluation’ was below

The “Condition evaluation” component refers to the necessity to evaluate whether a workflow has a condition. This can be seen as the general cost of the event, but our Product and Engineering teams have had discussions about changing this description to make it less obtuse and misleading, so I will incorporate this as feedback internally to emphasize this problem.

This obviously didn’t make any sense and so my reply to express the concern of the nonsensical ‘explanation’…For me, the 0.64/0.63 WUs to evaluate if a condition exists or not on the send email action doesn’t make any sense at all, not just because that is an excessive amount of WUs for checking if a condition exists or not, but because nowhere in the manual is there a depiction of this in the charge sheet, nor is this applied to all workflow actions or triggers.

The next ‘explanation’ from support is below

When setting a recurring event on a thing , such as the Current User, Bubble will first read the thing to see how many recurring events have already been set and then store information about the new recurring event on the relevant thing .

I’ve discussed this behavior with our engineering team and they informed me that there is a database read and write that occurs when setting a recurring event.

This didn’t address the ‘conditional evaluation’ charge of 0.63/0.64 WUs.

My reply: Understood that the cost of setting a recurring backend is affected by a read and write, causing the true cost to be higher than expected and documented. Thank you for confirming that piece to the puzzle.

Could you please address the issue of conditional evaluation.

About a month later I followed up and received this response below:

As I shared in our previous thread, setting a recurring event on a thing, such as the Current User, will first read the thing to see how many recurring events have already been set and then store information about the new recurring event on the relevant thing.

Our engineering team is working to update the workload usage charts so that this metadata update for recurring events is categorized properly.

To provide some context, the metadata update for setting recurring events is currently attributed to the “Conditional Evaluation” category, which is why you see 0.63 WU used for an empty condition.

Our engineering team is working to categorize this workload usage properly and I’ll keep you updated on their progress!

I then looked into the historical WUs of this recurring event and replied as such: I just checked into the WU metrics and it seems this ‘conditional evaluation’ is taking place every time the recurring event runs, and not just at the time it is set, which makes the explanation of the ‘conditional evaluation’ being mislabeled, and that it is really a ‘lookup do other recurring events exist on this data type’ seems inaccurate.

If the mislabeled conditional evaluation occurred only at the time of setting the recurring event, it would make sense that it is mislabled and actually attributed to the ‘lookup do other recurring events exist on this data type’, however, since the conditional evaluation is charged whenever the recurring event runs (ie: the scheduler scheduled it to run every day and on a day it runs).

The last response of the topic from support is below:

Our engineering team confirmed that the metadata update is labeled incorrectly as “Condition Evaluation” when a recurring event runs after it is already set. After the recurring event runs, it schedules itself for the next date/time, which also triggers the metadata changes.

So the lesson to learn from this is, that if you are testing WUs and attempting to find ways to optimize an application in regards to WUs and find yourself scratching your head at exceedingly confusing things, it might not be that Bubble is ‘overcharging’ you or that there is an actual bug in how the WUs are charged. It could be simply that Bubble still has not released a complete and thorough charge sheet and that Bubble has mistakes in their metadata and are mislabeling charges.

Another lesson too, is to never just accept the first response from support as the definitive answer, especially if it doesn’t add up to a logical explanation.

This is my most recent reply to support on this topic…Could you please help me understand why the mislabeld ‘conditional evaluation’ charge of 0.63 WUs is nearly 6 times higher than the 0.1 WUs in the charge sheet for adding a new item to the scheduler?

This is what the WUs metrics show after the first run (ie: when the event has to schedule itself again)

It really seems strange that I can’t get a valid explanation on this ‘conditional evaluation’ charge, since everything support as reported back to me doesn’t add up to logical explanations. The only explanation that could possible suffice is on the first ‘set recurring event’ which is where I can understand a mislabeld conditional evaluation charge of 0.63 WUs is to check if any other recurring events are attributed to the data type, but when you see for every single run of the recurring event the same mislabeld charge that is 6 times higher than latest explanation of having to be added to scheduler (which is only listed as 0.1 WUs in the charge sheet), it doesn’t make any sense.

@fede.bubble could you maybe help with getting some fresh eyes on this subject? Would love to either have a solid explanation or a resolution on two points. 1. Label the initial lookup for pre-existing recurring events properly and add that charge to the public charge sheet. 2. Why is it that adding a recurring event to the scheduler is 6 times higher than the pricing sheet shows that adding a new item to the API workflow scheduler is supposed to be 0.1 WUs and not 0.63/0.64 as I am experiencing in the application used for testing this.

[Jici]

@boston85719

Bubble will first read the thing to see how many recurring events have already been set and then store information about the new recurring event on the relevant thing

So basically, we are charged from Bubble to apply their own plan limits pricing structure (so we don’t go above the maximum of recurring workflow that can be scheduled on a thing according to plan) ? Doesn’t make sense. We shouldn’t get charged for that OR the limits should be removed since we are using WU (and at this moment, no need to check how many recurring event exist on a thing)

Also, Did you test if scheduling the workflow itself cost less? Because this avoid Bubble to check how many recurring event are scheduled already…

[boston85719]

@Jici

Did you test if scheduling the workflow itself cost less? Because this avoid Bubble to check how many recurring event are scheduled already…

When running recurring backend workflows for the same functionality of a recurring event the cost is for the action itself of the send email, the action of schedule backend workflow and the adding to the scheduler…so total of 1.3 WUs as to be expected as it is all clearly labeled properly, and all in the cost sheet.

The very first run of the backend workflow setup is 2.0 WUs because there is the cost of running the action to schedule and the adding to the scheduler (total of 0.7 WUs) plus the 1.30 WUs for the sending of email, scheduling of backend workflow and adding to scheduler.

The reason I ran the test to begin with was to get a baseline of best practices on how to build an optimal app in the age of WUs. Should we opt for a recurring event or setup a backend workflow that is recurring?

It is impossible to run these tests and come to a conclusion when the numbers from the test results are buggy, mislabeled and inaccurately explained by support/engineers.

Would love for other professional developers to test this and see if they get the same buggy numbers for the recurring event.

[boston85719]

Latest replies and responses…still nothing logical, all sounds made up in an attempt to sidestep the issue.

From Bubble:

Our engineering team confirmed that the metadata update is labeled incorrectly as “Condition Evaluation” when a recurring event runs after it is already set. After the recurring event runs, it schedules itself for the next date/time, which also triggers the metadata changes.

From me:

Could you please help me understand why the mislabeld ‘conditional evaluation’ charge of 0.63 WUs is nearly 6 times higher than the 0.1 WUs in the charge sheet for adding a new item to the scheduler?

From Bubble:

Our engineering team confirmed that the metadata update reads an item, which uses at least 0.015 WU + more for each byte, and then writes to the item, which uses 0.5 WU.

From me:

Now, I’m confused more. If this recent explanation about reading an item and then writing to the item is to explain the cost of the ‘lookup if existing recurring events exist’ that is okay, but the issue now is not that, it is the fact that the reply previously was about why after the first ‘set recurring event’ ran, which is when I’d expect a need to ‘lookup if existing recurring event exists’ would take place, so not during the ‘set recurring event’ but during the actual adding to the scheduler.

I showed the screen shot and indicated that the same mislabeled metadata for conditional evaluation shows up during the actual running of the recurring event, whether that be the second, third, fourth run of that event, there is still this charge of 0.63/0.64 WUs for conditional evaluation. The most recent message prior to this indicated that was for adding the event to the scheduler, so my question was why is that cost of adding the event to the scheduler as displayed publicly the charge sheet of 0.10 WUs 1/6 the cost of what is experienced here at 0.63/0.64.

Please do not conflate the two issues. I’m okay with the explanation that the conditional evaluation is mislabled, and is a ‘lookup if existing recurring event exists’ and that there may be a need to read an item at 0.15WUs and write to the item at 0.5WUs (even though that is 0.65 WUs and not the 0.63/0.64 expereinced) as happens when we run the set recurring event action, but that doesn’t seem necessary and is not the explanation for why that same charge is experienced on each run, as it was expressed it is for the adding item to scheduler once we get past the initial set recurring event.

So the question is why is it that on each successive run of the recurring event is there a charge of 0.63/0.64 WUs to simply add the event to the scheduler?

Part of the issue is that the numbers don’t add up. And I am the type that believes numbers don’t lie. If the pricing sheet says adding to scheduler is 0.10 WUs I’d expect the adding to the scheduler for recurring events to also be 0.10 WUs. And I could understand that there may be an additional charge of 0.60WUs for running the ‘action’ (If I schedule a backend workflow it costs 0.60 WUs to run the action that is schedule backend worklfow plus 0.10 WUs for adding it to the scheduler). So if I saw the recurring event with a mislabeled charge of 0.70WUs of conditional evaluation for each successive run, I’d understand, but it is 0.63/0.64 WUs and not 0.70 WUs…nor did the explanation of it is the cost for adding to the scheduler include the idea that perhaps an ‘action run’ charge is also applied.

I feel like I’m not getting logical explanations, and this issue is not being addressed properly.

Surely Bubble is not leaving any money on the table. So if there is a need to update the metadata for each successive run with a cost of 0.15 for read and 0.5 for write plus the 0.10 to add to the scheduler, the cost should be 0.75WUs and not 0.63/0.64…I’m just looking for a logical explanation that has numbers that can add up, because something is off and it is not being addressed by Bubble.

[boston85719]

So the final outcome is that Bubble has labeled properly the metadata, which should at least help a bit in reducing confusion around this example of WUs.

I’m following up here to let you know that our engineering team deployed a fix so the metadata update should now appear as Processing in the workload usage charts.

To clarify, the metadata update occurs both when the recurring event is initially set and when the recurring event is scheduling the next iteration. When scheduling a regular API workflow, we don’t need to keep track of the metadata because regular scheduling is a one-time operation.

The metadata update consists of a read and write to the relevant item. Reading an item uses at least 0.015 WU + more for each byte, and writing uses 0.5 WU. This workload usage is currently labeled as “Condition Evaluation” and our team is actively working to update this label!