Hoping that some of you can share your thoughts on a recent unpleasant surprise on the cost of some backend processes that I tried to run.
Last night I started a backend workflow using List Popper (thanks again for your great plugins @keith ) and concluded based on the first 10 records of 3,000 that it was going to cost me about 150,000 Work Units. The process was just tying a new table I had loaded to another table that it is related to (ie creating keys). I thought that it would just be easier and less time to configure a backend workflow than fiddle with a spreadsheet to get the unique ids in a single spreadsheet cell as a comma separated list. I guess I was wrong! I have gone the spreadsheet route for this load but the experience has me questioning my approach on all the processes that i have been using List Popper for.
I am curious to know if anyone else has experienced such a hi cost for using this plug in and how much people think the cost per transaction might come down if I stopped using List Popper, (which works great and is very easy to set up) and went 100% native Bubble components.
Thanks,
DJ
Before anyone chimes in with a recommendation to use Xano, I am almost certain that I do not want to increase complexity by introducing a separate backend like Xano for this particular project - in large part because much of the heavy lifting will be done up front in terms of ETL, so if List Popper is as expensive as it appears to be under the new pricing, I will probably keep it. However, I would consider rebuilding processes with just Native Bubble if there were substantial efficiency gains to be had.
You should (āshouldā) be able to drill into your recurring workflow and see what exact steps are accounting for what amount of WU.
One reason Iāve become extremely disenchanted with Bubble is the updated pricing scheme. What List Popper does is VERY simple: It takes your list of data, takes the first item off the list and forwards that item, along with the remaining list, to its outputs.
Server side plugins operate as AWS lambda functions (functions as a service) and the execution time (which is how AWS Lambda is billed) is then marked up by Bubble by what is still a usurious amount.
They had reduced that markup after various complaints, but if your list is āhugeā (Bubbleās idea of a huge list is different from anyone elseās) or the objects themselves are āhugeā, then execution time could result a large markup when turned into WU.
Of course, thereās almost nothing you could do in the allowed execution time that should cost anything but a small fraction of a cent. (A very small fraction.) But thatās Bubble these days.
Now, it could be that Bubble is incorrectly accounting for WU here, and Iād encourage you to file a support ticket with them.
I havenāt been keeping tabs on Bubbleās pricing shenanigans of late. Have been spending time in more worthwhile pursuits.
Backend (server-side) plugins are unfortunately not really a good thing anymore in Bubble due to the markup on AWS execution costs that Bubble actually incurs. Iāve written about this extensively in the past and wonāt belabor the points here, but Iām very sorry that List Popper isnāt helpful to you in your case.
ASIDE: Hi to anyone who has missed me and, yes, there are some issues with drag/drop in Floppy in certain scenarios that I still need to fix. Apologies for that not being solved yet.
I replaced many recursive workflows that would change a single thing at a time, to bigger batches of āMake changes to a list of thingsā (still recursive workflow). Saw a large reduction in WU.
Adding a serverside action ontop of that like List Popper I could see it getting more expensive for no reason.
Maybe you can restructure it a little bit and do things in overall less workflows? Like taking your list and doing ā:from item #2ā instead of List Popper doing it?
Hey @mikeloc! Yeah, Iām alive and yes am essentially in Bubble rehab. Whenever I consider returning to it seriously, I just get hives.
To the point of this thread, Iāve been very concerned about the potentially negative effects of List Popper and Friends and am still considering whether I will actually migrate them to the latest node version of the plugin API.
Those plugins are so simple that itās not ethical to move them to a paid model, so anyone who wants to ensure they remain available might want to make a fork while they still can.
At the moment, Iām leaning toward migrating them, but havenāt found a good chunk of time to focus on that just yet.
I have been working more on my day job (digital marketing/SEO for an enterprise SaaS thing Iāve long been involved with), using AI to help me learn Python much better, and generating an insane number of waveforms and wavetables for modern digital synthesizers. Not Bubble, but if you miss my dulcet voice:
@keith Thanks for the reply. I am happy to share the log file. Do you have a plugin that lets me scrape the Bubble Log Files Seriously though, if you are interested, I am happy to jump on a web conference to see what it going on here. It may be a stupid error on my part that is causing the WU spike but I donāt think so.
With regards to the new pricing scheme. I have to say that I do not envy Bubble their monetization challenges. I figure it must be very difficult to come up with a one scheme fits all pricing model that would keep that venture capital company that gave them $100M happy enough to stay out of their hair. It does seem like they may have missed the mark for a lot of use cases though. Hopefully they will find a way to tweak it.
@tylerboodman - I like the idea of doing bulk loads but frankly have an iterative mindset that is slow to adapt. Given that one cannot actually export the Bubble log files, my preference to use interative probably does not make a lot of sense - btw does anyone have a way of exporting Bubble Log Files?? - I am going to build in some additional fields to the current model to help with ensuring completeness - if you have an approach to check for completeness with your batch uploads/modifications, I would love to hear them!!!
I will say this - I do not have a lot of experience with backend, mostly bulk API list changes to a relatively small table (1000 items). Iāve also been doing a deep dive into how much more WU is actually used to āmake changes to a thingā (documentation says this is 0.5WU per thing). I have yet to encounter making a change to a thing that costs more than 2.5WU - this is updating data on a thing with approx 30 columns, and setting 3-4 lists on that thing with about 5 items each. Iām usually closer to the 1.2-1.4WU mark for changes like this. I can say, without having used List Popper or anything similar, that based on my testing, ~50WU per āthingā updated does seem abnormally high and certainly could warrant a deep-dive into what is accruing that much WU per thing.
Unfortunately I donāt have the experience to properly point you in the right direction, but if you reach out to Bubble Support with your WU usage concerns they have done some deep dives for me (takes a few weeks of back and forth testing).
You already said you know the WU per workflow. I guess you just divided the cost by number of items? You should also be able to drill into a single iteration and see what cost of each workflow step is. (I am unfortunately not available to hop on a call and watch you do that.)
However, once you do that, you can go to AWSās page about lambda pricing and multiply the execution time for a single iteration of your List Shifter executions and multiply it by the time-per-millisecond that AWS charges.
You can then take the number of WUs that Bubble charges you for those steps and multiply it by the dollar cost per WU that your current Bubble plan has.
The ratio of the resulting dollar values will tell you the markup that Bubble is charging.
You may find that the markup is exorbitant and you may want to take that up with Bubble (via a support ticket/bug report).
I canāt recall if this is something that relates to AWS Lambda executions, but Bubble also charges for data ingress/egress charges. They definitely do with EXTERNAL API calls.
For example, I have an API endpoint that I built in Google Cloud that does one of the most compute intensive aspects of one of my apps.
My Google Cloud bill for last month was LITERALLY $0.02 (thatās 2 cents) from Google. This includes compute and data ingress/egress.
However, Bubbleās charge for MAKING those external API calls and the associated data egress/ingress is a substantial portion of my monthly Bubble cost to run that app. Weāre talking DOLLARS (perhaps tens of dollars) of my monthly app cost. The actual markup may be upwards of 100,000%.
Iāve not done the exact math recently, but itās crazy.
Also, I should clarify, the situation with external API calls is in fact BETTER than with AWS Lambda (SSA plugin) executions. That is, external API calls are in fact more reasonable than SSA executions.
This isnāt reasonable or fair, but it is what it is.
@tylerboodman yeah you can do that if your list is a list of unique objects (e.g., Things). But if your list is just scalar values that might have duplicates, this will not work as when you remove one item with a duplicate value elsewhere in the list, you will lose all entries with that same value due to the set-wise nature of :minus list.
But if youāre iterating through a list of values, in order to create some number of things, well youāre sort of stuck in the case of duplicates. This is the sort of Bubble dumbness that gives me hives.
Itās a serious design deficiency for which there is no workaround except for my approach.
@keith They may be cheaper but they are still an expense. I am using the Vimeo API (which is horrible) and for some reason they leave out the list of domain in a list all video files CALL. As far as I can tell, that means to be sure that you have properly tagged all of your videos with the right domains, you have to make an individual call for each video to get that info. I suspect that Bubble LOVES vimeo. I did a fairly deep cost comparison and by sheer chance Vimeo put in a new pricing model just for Canada just before I made my final choice on supplier that seemed to make them the hands down winner. Had I known about the domain issue and the cost of API calls, I would probably not have chosen Vimeo. I canāt say for sure that my other short listed vendors API.Video, MUX, Dacast, Wowza and Kaltura would have been equally painful but I am willing to bet they are not.
I have gone the spreadsheet route for this load but the experience has me questioning my approach on all the processes that i have been using List Popper for.
Seriously though, if you are interested, I am happy to jump on a web conference to see what it going on here. It may be a stupid error on my part that is causing the WU spike but I donāt think so.
