(JavaScript) How to actually load the data before taking any actions?

[vini_brito]

Hi,

Previously (some weeks ago) I had a function working properly, but now I can no longer make it work. I’m not sure what changed, but I don’t recall making any change to it. Anyway…

The goal is to grab a List of Things and to apply an action on each object, getting a value from each Thing in the List of Things.

The expected behavior of the code snippet below is to console.log() a field’s value (in this case a text) once for every Thing (hence the forEach!).

However it logs the first, then restarts, logging the first again then the second, then finally the three of them. Then if you activate the workflow a second (or third, fourth…) time, it will behave as expected, which is to log the three of them at once.

It seems to have something to do with the “load all the data before taking actions”, however I can’t be sure of that. Debugging on Firefox throws me to Bubble’s standard errors after the first element gets console.log()'ed by the forEach.

To see it live, open this test app, open your web browser console to see the console.log() results, then click the big left button to generate some data then the big right button to activate the custom code.

This is all the code used in that workflow:

function(instance, properties, context) {

// Successfully retrieves a field value within every and each Thing in the List of Things.	
let listOfThings = properties.fieldy_list.get(0, properties.fieldy_list.length());

let processOnEachItem = (element, index, array) => {
  			console.log(element.get("thing_name_text"))
}  
  
listOfThings.forEach(processOnEachItem);
  
}

And here’s a link to the app editor if you want to check something.

The problem is that this snippet will be part of a much bigger action and would be insane to have the entire action repeating once for every single element in a list of things.

Any suggestions on how to defeat this?

[mishav]

@vini_brito

load all the data before taking actions

What you describe is exactly as outlined in the manual: https://manual.bubble.is/building-plugins/loading-data.html

@vini_brito

Any suggestions on how to defeat this?

Your script is already handling it properly, processing after each item has been loaded. Consider those console logs as in the loading phase.

@vini_brito

The problem is that this snippet will be part of a much bigger action

You’ll want a similar approach, i.e. first load all the data for all the snippets, then start the processing phase.

[vini_brito]

@mishav

What you describe is exactly as outlined in the manual: https://manual.bubble.is/building-plugins/loading-data.html

It’s a relief to know I am not the only one thinking that!

@mishav

You’ll want a similar approach, i.e. first load all the data for all the snippets, then start the processing phase.

Do you mind telling more about that? I’m still a bit confused. My intention is to run this snippet in the middle of a bigger action, however that loading phase makes the whole action restart several times and that’s what I want to avoid.

I thought about having that loading phase in a previous action and write it to a instance_data_whatever or trying to change the script so I can position this snippet right at the start of the action so it won’t repeat everything else, however I’m afraid both these options will make it a bit clunky, so I wanted to see if someone else had thought of another way.

Also, regarding it initially not having to load the data through that mechanism (restart) and then suddenly having to load the data like this, have you (or anyone else) seen this before? That’s something I’m curious about, because I have other snippets working with lists and this is the only one that repeats for now.

I’m trying to see if there’s a “line” that if crossed will trigger this loading mechanism instead of having bubble send all the data to begin with. That’s a bit unclear for me, so I’m trying to tap into the collective knowledge in order to clarify this subject.

[exception-rambler]

I have always found that UPDATE fires multiple times on page load and have seen the behaviour you’re seeing.

In one plugin where this affected behaviour I added a counter to UPDATE to track number of executions, and only ran the main scripts on the Third execution. I would not advise this though as it assume that UPDATE will always run three times before all data is available. But it did correct the behaviour I was seeing on my machine at that time.

Ed

[mishav]

@vini_brito

My intention is to run this snippet in the middle of a bigger action, however that loading phase makes the whole action restart several times and that’s what I want to avoid.

You’ll need to think about the overall action a bit differently. Instead of it having independent components, conceptually it has a loading phase, processing phase and output phase.

From a black box point of view, it doesn’t matter if loading and processing is interrupted and run multiple times, so long as the output comes out correct.

From an efficiency point of view, you likely want to minimise how much gets rerun, so keeping all loading first makes more sense than a stream of loading, processing, loading, processing, output.

You can still keep things fairly modular, for example by each snippet supplying a loading function, and processing function, to the overall action, which can be a component manager.

@vini_brito

have you (or anyone else) seen this before?

Yes, pretty often on non-trivial data sets. I think it occurs relative to data size, client/server timing, network speeds, cache sizes, etc.

@exception-rambler

only ran the main scripts on the Third execution. I would not advise this though as it assume that UPDATE will always run three times before all data is available.

I agree, this approach isn’t reliable, update() is potentially run for multiple reasons, including screen resize, dynamic properties changed, position alltered in a repeating group.

But if it works, pragmatism rules! : )

[AliFarahat]

Hey guys found this thread instresting as I was plagued with it in the early days of development

There several this I do before manipulation of any data or DOM

I load all data.instances that I have.
I load all properties I am going to use.

I know the majority of us tend to pull data when we need it but having the data fetched in the early phases of the execution saved me plenty of headache.

In some senarios I need to excute a DOM code only once. But don’t mind and even need the data to updated from bubble. For that I have setup a runonce function using the instance.data as the flag for it.

Instance.data is a handy tool,. I had found that once the data is added it does not get removed even if updates are being pulled

[vini_brito]

@exception-rambler Hmm, that’s interesting, I still have not tinkered with the update function yet, but now I will keep it in mind! I’ll take your advice on how to use it.

@mishav

@mishav

so long as the output comes out correct.

That I will keep in my mind too! Focus on the outcome, of course without letting processing stuff repeat itself. I guess that moving these loading functions to the beginning will be the way forward in this case.

@AliFarahat

@AliFarahat

I load all data.instances that I have.
I load all properties I am going to use.

I do that too! Much better.

@AliFarahat

In some senarios I need to excute a DOM code only once. But don’t mind and even need the data to updated from bubble. For that I have setup a runonce function using the instance.data as the flag for it.

Hey, can you tell more about this “runonce” function approach? What is it, and what you do with it exactly?

[AliFarahat]

I use something along these lines

//Bubble - runOnce function
const runOnce= (function() {
  let executed = false;
  return function() {
    if (!executed) {
      executed = true;
      //Do something
    }
  };
})();


//Bubble - Execute runOnce Function
//Check in the Bubble instance is undefined (function did not run yet)
if (instance.data.runOnce=== undefined) {
  //Create the Bubble instance
  instance.data.runOnce = runOnce;
  //Run the function
  runOnce();
}

[JonL]

Still don’t get it! Why would the get(e) method call recursively the function i times?

function(instance, properties, context) {    
    
    console.log("I like to repeat myself");
    instance.data.columns = properties.columns.get(0,properties.columns.length());
    for (var i = 0; i < properties.columns.length(); i++) {
        instance.data.columns[i].get("id_text");
        console.log("Don't mind me. I'm tripping "+i);
    }    
    console.log("Fuck this shit I'm out");

[JonL]

function(instance, properties, context) {
    
    if (instance.data.count == null)
        instance.data.count = 1;
    else
        instance.data.count++;
    
    properties.columns.get(0,1)[0].get("_id");
    properties.columns.get(1,1)[0].get("_id");
    properties.columns.get(2,1)[0].get("_id");
    properties.columns.get(3,1)[0].get("_id");
    properties.columns.get(4,1)[0].get("_id");
    properties.columns.get(5,1)[0].get("_id");
    
    console.log("This madness needs to stop. I've iterated myself "+instance.data.count+" times");       

}

@bubble is this normal?

[vini_brito]

This last one seems to be normal, the other one I’m too lazy to actually look at

Anyway, this last one reloads everytime .get is called. Pretty much what it’s supposed to do.
As you can see, I pulled some hair too when I first started dealing with it.

[JonL]

Yeah. But why? Why doesn’t the get method just wait and deliver all the data at the same time? Why this madness and resource consumption?

[gustavopch]

But what’s wrong? It’s described in https://manual.bubble.is/building-plugins/loading-data.html. Whenever the function tries to get something that’s not already loaded, an error will be thrown, the thing will be loaded into memory in background, then the function will run again… It will repeat this as much as necessary to load everything. If you’re getting 20 pieces of data that are not already in the memory, the function will run 21 times (20 terminated with an error, 1 succeeded).

Basically:

• Function starts 1st time…
• Tries to get X: not loaded, throws error, loads the thing in background
• Function starts 2nd time…
• Tries to get X: great, already loaded
• Tries to get Y: not loaded, throws error, loads the thing in background
• Function starts 3rd time…
• Tries to get X: great, already loaded
• Tries to get Y: great, already loaded
• Tries to get Z: not loaded, throws error, loads the thing in background
• And so it goes…

Extremely important to have all the data loaded at the beginning of the function so that only the gets are subject to re-running. They are prepared for that. Otherwise you may inadvertently end up doing business logic (like sending an email) multiple times, like this:

• Function starts 1st time…
• Tries to get user data: not loaded, throws error, loads the thing in background
• Function starts 2nd time…
• Tries to get user data: great, already loaded
• Sends the email
• Tries to get some other stuff: not loaded, throws error, loads the thing in background
• Function starts 3rd time…
• Tries to get user data: great, already loaded
• Sends the email (AGAIN! )
• Tries to get some other stuff: great, already loaded
• And it proceeds…

Result: you’ve sent duplicated emails.

EDIT: Indeed, data loading could be batched at least. Better than that would be if Bubble used GraphQL.

[JonL]

Yes. But why? Not why Bubble does that, but why did they make it that way

Edit: Couldn’t they just use a promise before sending me the data?

[gustavopch]

You’re talking specifically about the function restarting multiple times, right? Bubble started in 2012. I guess there’s already a lot of legacy code they’re not happy to touch. They probably came up with the current solution before promises and async/await became popular, so current solution is at least better than callback hell.

Now talking about the real problem, the performance: it’s not easy to load relational data through REST APIs, especially when your API cannot imagine which combinations of different types of data the client is going to request. The best they could do would be to batch requests getting same type of data (from same endpoint). Best way to solve all this though would be if they migrated from REST to GraphQL.

Bubble should REALLY consider open sourcing as much of their code base as possible, so that we would be able to submit pull requests to improve this kind of thing that they don’t have the time to work on.