Help. How to see what "Individual Data Request" is in workflow logs when you have overages happening?

We have a pretty large app and currently about half of our workload units are attributed to “individual data requests" but unfortunately when this portion is clicked it shows nothing… Therefore, no clue where to optimize…. especially when paying overages. I don’t know how to fix this and reduce whatever is happening within our app. Can someone help please?

I have no idea where to start or where to look because there’s genuinely too many places. It’s not practical to go through everything in the application and blindly guess what’s going on. What is someone like me supposed to do and why can’t we see where these workload units are coming from?

We also don’t really have a massive amount of users at the moment… Maybe 15-100 every week so far but at this rate it’s gonna be very hard to scale if we can’t see where our workload units are being spent the most. Most of our workload units are in “Fetching data”, with almost about 3/4 being “Individual data requests”.

2 Likes

@julpereira82 “Individual data requests” usually come from repeating groups, searches, or API calls fetching multiple things. If the audit tool shows nothing, it’s probably happening inside workflows or nested elements. Check repeating groups, searches, and backend workflows, and see where you can add constraints, limit data, or cache results. Use the Performance tab in debug mode, it won’t give exact units but will highlight the heaviest elements so you know where to optimize.

@julpereira82 the below is wrong

Since individual data requests are fetching one thing, not multiple, hence the individual in the naming convention.

What @connect10 might have been attempting to explain is it can come from repeating groups that are of a data type that has a related data type as field and inside the RG you have elements pulling field values from the related thing. Or, maybe a search that does a search with constraint of unique id is in.

@connect10 might have also been trying to say that they definitely do not come from api calls, since api calls do not fetch data from database, unless it’s an api call to your bubble database.

@fede.bubble is there any technical reason why we remain in the dark in WU charts what the individual data requests are coming from? Since they are tracked, surely they can be traced? It would be nice if the current user requests at least get their own special handle and not just individual data request.

@julpereira82 if you’ve followed some advice floating around the forum, for security, and added unnecessary conditionals onto your workflow even triggers or actions, those will add individual data request charges for verifying the condition on the backend. If that is a part of it, consider simpler ideas like privacy rules, role based redirects, proper constraints removing access to data a user should not be editing, and find areas to remove conditions on workflow events and actions.

They may also show up from do a search first item…but yeah, ridiculous that the WU charts/logs do not specify or split and group them, leaving us to only guess where in the app they may be coming from.

1 Like

I don’t have any new information on that front. I’ll share with the team

1 Like

Thanks, I didn’t know conditionals cost more WU. I’ll consider that, though I was hoping for a more concrete answer on working on it and being able to optimize.

I do have things related to things among some of the repeating groups, I’m not sure what can be best practice to structure this though and if that could be it.

For example, we have a Bag, that has a list of Events, that each can hold a list of Tickets.

I’m not sure the best way to do this though, if it is the issue. Would it be best to, instead of storing a list of Event things inside a Bag, storing a list of numbers/texts to store the unique IDs of the events they’re related to? I considered this, as we need to show them as related information (like, this Event comes from Bag A or Bag Z… or this is a Ticket from Event 1 from Bag 2). But then we’d have to Search Events by curren’t cell’s Bag’s Event filtered by the Bag’s Event text ID and not sure how much better that is than just storing it and referencing it (this bag’s Events vs. Search for Events whose bag_uniqueID is this bag’s unique ID). We might also have a profile page showing an Event’s Tickets in a repeating group.

Sometimes we also have conditionals on repeating groups to apply filters by the user’s preference or query, like say the user wants to see pink Bags only so the conditional will change the source of the repeating group like “if this custom state filter is pink, Search Bag’s filtered etc…”, “if custom state filter is green, Search Bags filtered by name green…”. But these aren’t used too often, so condition is mostly false. I also have conditions like Make changes only when… Current Ticket is not empty/a condition checks yes (which I do precisely to avoid WU for making changes when not applicable or sometimes for functionality). Make changes only when it’s been more than 7 days, etc…

/// An idea I had to avoid the constant referencing of Things of Things would be to store the first Ticket’s Title in a single Event text field (as, Event ticket_title) and entirely remove the list of Tickets from the Event, especially since we only need to show the first Ticket’s title to browsing users. So, avoiding lists stored anywhere at all costs. This would mean though that we’d also have to set an Event data field to each individual ticket, so when we show a particular Bag’s Event’s Tickets instead of doing this Bag’s Events:item #x’s Tickets: first item we’d just do for Search for Tickets, filtered by Event matches clicked Event. But that also means we’d have to elsewhere source the tickets differently, so instead of referencing Event’s Tickets we’d have to add a new Search for Tickets:filtered id matches this Event’s unique ID.

There’s lots ideas I’ve had and some things I have tried, unfortunately this is a lot of experimenting and I’m trying to make it feel less like a shot in the dark. To just try the idea above we’d have to re-structure a lot of our setup, risk many bugs (after finally stabilizing features), and we’d also have to make changes to thousands of things in our database which will thousands of WU long term (to update existing user creations) and who knows if it’s not that much better and we end up having to reverse it which is not great. Otherwise I’ll also recreate test pages with sections of the app… but what if it’s workflows? Or how the data is structured in the database? I have to somehow test this and I don’t know how without having to make such disruptive and uncertain changes. It’s certainly easier when you can click and pinpoint specific workflows or searches that are weighing things down unnecessarily out of inefficiency.

Not sure if this any of this may actually be it though. There’s also conditionals on elements across the app (such as, This Ticket is not clickable when current user is not Event’s creator/border color turns blue/not visible/etc. changes), which looks like might be part of it but who knows? There are so many angles. It’s a mystery… Currently this is what I’m looking at from just starting up the page itself… Trying to get them down because at this rate this cycle we might be paying overages again for sure (before considering upgrading our tier, 1-10 users shouldn’t rack up 1k WU per day at any point and we can’t scale that way). We can’t afford to pay hundreds for hardly 500 users.

Photos for reference:

Currently this is what I’m looking at from just starting up the page itself… App has a lot of work put into it, it is single-page and mobile friendly, has about 1,200 workflows, and some many dozens of groups. Hopefully I can figure something out for the sake of everything.

(In Search section largest search consumes1.4 WU maximum. Most WU coming from individual data requests, trying to test) Result of a single-person page load:

Below, a day we had spike in workload units less than a week ago. We had maybe 9 users that day according to google analytics, most just browsing information. Also had changes done to database by 1-2 users but those were not most of the WU as seen in top hour:



January 9 we had a huge spike too where we reportedly had 17 users or so (mostly reading for under 2 min), something caused a lot of WU and it wasn’t database changes. We removed a bad recurring workflow, which improved things slightly (on the right), but we’re still left with the individual data requests which is most of everything, same as first image.

If anyone has tips please help.. Many things I’ve been able to optimize as much as reasonably can, but this one I’m truly lost on. Will keep testing…

If there are elements in the repeating group that has a data type of ‘type A’ that has a related field of ‘type b’ and a text element for example says ‘current cells type A related type b field xyz’ then that could be it, as those would result in an individual data request for every data entry in the RG of ‘type a’ since each ‘type a’ has a related ‘type b’ whose field ‘xyz’ is being requested to be displayed. If that is something you are doing a simple fix if there is only one or two field values necessary from type b in the rg displaying type a, add those fields to type a as well.

Is your RG showing Bags, requesting details from the List of Events and their respective List of Tickets?

These are situations where Ticket A is in a repeating group of Tickets A-Z and in each cell you want to show which Event the current cells Ticket is from, so you are doing either a search for Events whose related ticket list contains current cells ticket…what is the actual data structure of your DB? from the example of

That means as I stated above, each cell of an RG of Tickets needs to perform a search of Events to check the Event list of Tickets to match to the current cell Ticket.

If that is what you are doing, and you just want to show the Event Name in the RG of Tickets, just put onto the Ticket data type a field of type text that is the Event Name, and display that. Or if you have a field of type Event as related field on the Ticket Data Type, and each RG cell is showing current ticket event event name, then again, put onto the ticket data type a field of type text that is the event name, since each cell of the RG showing tickets is doing an individual data request for the related event just to show the name of the event.

The issue is not having related fields, it is requesting a field value from the related field in order to display it.

An alternative to the DB structure tweaks of adding a field of text, could be instead of doing for an RG that is list of tickets a search for tickets, do a search for events:each events related list of tickets. But that still would result in individual data requests, but maybe fewer, I’m not sure.

Is that on an action to change the current ticket? If so that works to avoid WU for the action running and the WU for the database data changes. If no, it will incur WU for the condition check. When you run an action to make changes to a thing, and the fields in the action are not different from what is in DB, bubble still charges WU for the action to run, but no WU for database changes, as they do not actually make any.

1:1 relationships are generally better in my view…each ticket has a relation to one event…event has no relation to any tickets…need to know all tickets for the event, do a search for tickets whose event is equal to selected event…need to know which event a ticket belongs to, look at the 1:1 related field.

If the event is already on the page, that shouldn’t make a difference since current user is always downloaded as individual data request for each page and the events creator would be on page if the event is on page already. If however, this is on the ticket and it is ticket’s events creator, and the event is not on page, then event and creator are fetched I believe.

How many Bags are there? How many Events would each Bag have? How many tickets would each event have?