Reducing WU for fetching data on heavy objects

Just a quick one that may help someone in the future. (Im sure its been covered before but I couldn’t find it)

I was saving API responses to a “Data Type” and I was listing these in a Table. Same would apply for a Repeating Group.

Because the API responses were sometimes large chunks of JSON, some were extremely large. In the App Data view (database) I was getting a message regarding a 10mb size.

Even listing multiple records in a Table / RG was consuming large WU when fetching the data. As my use case was a list and then when a user clicks an item, they are shown the detail in a popup or another page, I didn’t really need to fetch the heavy offending saved API data at this point.

Solution:

By saving the heavy objects in another Data Type, I then simply save the ID of that Data Type record to the original Data Type rather than saving the whole object directly on it.

Then when I need to bring up the single heavy object, I just do a search for the Data Type ID attached to the main record and display it.

This drastically reduced WU fetching data requests by a long shot. Additionally, the original Data Type loads snappily in the Table / RG.

I know this won’t help all use cases but it can help quite a few when you just need to reference a Record in a list.

Hope that helps some.

4 Likes

Absolutely correct and great approach to reducing WU, as well as increasing performance.

Needlessly downloading large amounts of data is one of the main reasons for slowdowns on bubble apps I’ve seen, this is especially true in repeating groups.

A lot of people in the bubble community call this technique “satellite data types”. I’m pretty sure the term was pioneered by @petter in his book on bubble performance (great read, would recommend). But this technique is also very widespread in traditional non-bubble web development. It’s an essential part of database normalisation, and it’s generally referred to as horizontal expansion of the database (using multiple smaller tables) instead of vertical expansion (using fewer larger tables). Horizontal expansion is generally considered a better solution, but of course a balance needs to be struck between the two… can’t create a new datatype for every piece of information.

Congrats on figuring it out on your own :slight_smile:

3 Likes

But huge confusion is going on here , when you say satellite people often confuse it as do a search for needed to retrieve the data to increase performance but in relational data table this format works as good as well

Bubble uses Postgres, which is a relational database. There is no need to save the ID of the dataType as a text on the original datatype. You can save the new datatype onto the original dataType as a relationship. Under the hood, bubble is just saving the unique ID of the new dataThing , and fetching it only if any data in is referenced anywhere on the page. If you don’t reference the data of the ‘satellite datatype’ then only the unique ID is downloaded.

I don’t think there is a big performance difference between the two, as they are both essentially a lookup by primary key, however using Bubble’s in-built relationship will increase maintainability and speed up development because you can reference the data of the satellite datatype directly form the expression editor instead of having to set up a search & states to fetch the data.

1 Like

100% agree but I still see some people recommending when primary table is retrieved any foreign table is retrieved.