List Shifter: Reverse, Rotate, Swap and ITERATE (Loop) Over Bubble Lists | Now Part of Floppy

Is it possible to calculate the difference between two dates in the process step? The use case is that I have a repeating group that shows information about insect observations, and I’m showing the number of days between the first observation of a pest in a field and the last observation of a pest in a field. I’m interested in leveraging this tool to ultimately sort the repeating group by that calculated value.

Thanks in advance!

Hey @tim17, yes, you can do that. If an item referred to in a Process List step is a date, you can turn it into its milliseconds value using operator x.getTime(). Now the result of that step is a number (the “UNIX time” of the date as some would have it). Like so:

Do the same thing with your other date in another step. Now that step is the UNIX time of the other date.

In the next step, subtract these two values. Now you have the distance, in ms, between the two dates.

In the next step, you could convert that number of milliseconds to some other time unit, by dividing by some constant (e.g., 86400000 to convert to days).

Simple example page here: https://list-shifter-demo.bubbleapps.io/version-test/time-between-process?debug_mode=true

Edit mode: List-shifter-demo | Bubble Editor

The Process Output type is set to “number” in the main List Shifter interface and the workflow that executes Process List is this one:

Great! Thanks so much!

1 Like

Hi, is there a way to use list shifter as a pre-loader for a one page app?

What I’m trying to do is Have a Dashboard page with many reusable elements as App Sections.
I want to use list shifter in the Dashboard Page, and refer to the list shifter from reusable elements. Is that possible?

If not, will there be a benefit to use list shifter as a reference then also put that same search inside the reusable element? I think @boston85719 can relate to this as he does it to improve user experience.
I’m just thinking if referencing it in the Page level forces Bubble to load that data and cache, then using the same search inside the reusable element, bubble doesn’t actually need to search that data but rather, reference on the cache?

Does it make sense?

Bubble will perform a single search instance when on the same page there are multiple ‘do a search’ with the same constraints. What this means is if you have a reusable element with a list shifter element performing a search, and then on the page have another doing the same exact search, Bubble would only perform that search once. (Note, this is not known through testing: Bubble support told me they only perform one instance of a search when it is the same - but not in the context discussed here, but shouldn’t make a difference as on your single page, the reusable element will be on that same page)

One reason to maybe test for your exact use case is that a reusable will not be visible all the time, and until it is visible, Bubble will not populate for searches, but I’d expect Bubble is smart enough to know the search is the same as one performed on the page already, so it wouldn’t be necessary, and therefore Bubble wouldn’t perform the search again once the reusable element is visible.

I do this in my apps and they are definitely not slow, plus I don’t personally care so much if data on a dashboard page doesn’t load super fast, as to me, that is a very different situation compared to a search performed for an e-commerce product selection type of app, and the user is capable of waiting an extra second or two for the data to be loaded.

Overall, I think you may want to reach out to Bubble support for a clear answer to this, as they are really the only ones who would know exactly how Bubble works and how Bubble may cache or load data for a search.

3 Likes

Hey @boston85719 and @shu.teopengco - as far as Shu’s question: why not just test it?

Note that Bubble does cache search results and (for example), if you have some search that returns All Foobars:items to 100 and then later search All Foobars:items to 200, it doesn’t need to fetch Foobars 1 thru 100.

I’m not sure if a search in a reusable element might screw that up, but what @boston85719 says about element visibility is true. (Elements like List Shifter need to be visible for their code to run and toggling them off and on will cause them to instantiate/initialize again. Also if they go invisible you now longer have access to their outputs, which is code to how you work with List Shifter).

1 Like

A little further to what I wrote above… Understanding Searches:

What is a Search? When we send a simple Search (“Do a Search for…”) query to the database, Bubble responds incredibly quickly (nearly instantaneously). The Bubble programmer/user doesn’t really perceive this as being the case, but plugin builders do.

When you input a Search to a field in a plugin (in its main interface or an action’s dialog or what-have-you), what the plugin receives is very simple: It’s a single object with some methods (functions) on it. This object is called a List. One of the functions on it (.length()) tells us how many results were found (it’s really more like just a property as the value returns immediately, so basically a Search includes the info on how many results we could retrieve if we desire). The other of these functions is .get(), which we can use to retrieve some or all of the Search results (by their index numbers). The individual Search results themselves are, of course, Things (as that’s all we can store in the database).

It is the retrieval part of this operation that takes time in terms of interacting with the database. Unless we are doing some sort of pagination (which List Shifter can do, of course), we will typically want to fetch the entire List of results.

Depending upon how many things are in the list, it can take a certain amount of time to fetch all of the individual results. Once retrieved, what we have is an array of Things. Like Search results (Lists), Things are also very simple objects. They are an object with 3 functions on it:

One tells us what properties (fields) the Thing has on it (these are the custom fields you’ve defined in the Data part of your app along with several system fields that all Things have [created and modified dates, created by, Unique ID, etc.].

Another (again .get() ) allows us to retrieve the values for a certain property (which again may themselves be a Thing or Things).

There’s a third method on Things that we cannot access, but it’s all of the special Bubble secret sauce that goes with a Thing.

But anyway, properly structured Things themselves can be individually quite small. I’m not 100% sure of whether any of the data stored in a Thing actually travels along with the Thing object (it might in the case of things like the unique ID or single value fields that are core data types )-- it just gives us instructions on how to retrieve the properties/fields of the Thing. (Things should basically be composed of Things “all the way down”, as I like to say.)

In some plugin applications, you would want or need to go and fetch some of these field values, but in List Shifiter we do not do that (unless you invoke some special action like “Get Field of Interest”). What List Shifter does is just shoves the array of Thing objects to its Original List output, and a (possibly modified/re-ordered) array of Thing objects to the Shifted List output.

So, in some sense, List Shifter is the absolutely fastest way to fetch a list of Things because it does not fetch any fields. The “get” operations for the fields are only accessed (and not by List Shifter) when you use them somewhere, as in an expression. (For example, in a text element you compose: List Shifter's Original List: each item's Name. It’s at that point that Bubble fetches the values for the Name fields).

If you feed a Repeating Group with a source list List Shifter’s Original List (or Shifted List or whatever), again, no field values are fetched until you put some element in the cell that references that field.

(This is where things start to look slow to inexperienced Bubble devs. If your RG is showing hundreds of Things and trying to - for example - show a bunch of images that are stored on those Things, well, that can get to be a lot of data really fast.)

All of the above being said: Another thing to know about Searches is that, in a given page, they are not parallelized. They happen sequentially. If you have some initialization workflow that populates some Custom States, each with the results of different Searches, you will note that they execute in series. This is also true if you try to be tricky and have several workflows that trigger off the same event. You will find that one workflow completes before the other even starts. (The Debug Buddy action plugin included with List Shifter can be used to prove that this is so.)

And similarly, there is no free lunch with List Shifter: If you drop several List Shifters onto your page, they will of course start retrieving the Search results as soon as possible, but one List Shifter will start reading and become initialized before any of the others, and then some next one, and then some next one.

(List Shifter will often be marginally faster than some other way of loading Lists, and doesn’t result in any funny preloading of unneeded field values that might happen with other Bubble elements, but it’s still limited to the core rules of Search result fetching.)

4 Likes

Wow. Thank you for those comprehensive replies. It really helped a lot and a lot of free knowledge for us. Thank you. :smiley:

Gonna have to do some testing and I think it will take some time for me and o absorb some of it… :smiley:

@keith thanks for the details. Always eye opening.

Before I get testing on this, have you ever run a test to see if they are ordered in any way, such as the order in which they were created in the editor?

Something I found out recently about popups through a Bug report is that the first popup to be created can have a second popup created afterward appear on top (the expectation when you have one popup open and run a workflow to show another).

It was strange to me, but the explanation from support led me to copy the first created, then delete, then paste in a the issue was resolved (issue being that the second popup was getting shown behind the grey out of the first popup displayed, which was in fact the second popup created in the editor)

Because of that, and the way tabbing had been an issue for so long, I have a hunch that the first list shifter added to the page in the editor, would be the first Bubble would initialize and load the data for, all other things being equal.

Hey @boston85719: This is actually a really interesting question. But note that the answer (with respect to List Shifter) is complicated because of the fact that it loads Lists (any element that loads lists will also function similarly to what I describe below).

I did a little (actually a lot) of experimentation to determine the following:

So, first: When does a plugin element’s initialization function run? How is the order of this determined?

A plugin element’s initialization function starts when it first becomes visible on the page. (We know this from the plugin API docs and also by experience.)

But when that happens is determined by the element’s order in the DOM (the Document Object Model - the structure of that page that you can see in the Elements view in your dev tools).

It seems that the Bubble Editor pretty much structures the DOM in order of element creation. So, yeah, basically, “older” plugin elements will start initializing before newer ones.

Let’s say you put three List Shifters on your page, LS1, LS2, and LS3, in that order, at the same page level (for example, directly on the page), they will run their initialize functions in that order.

And, as a result, they will start their update functions in that order. But their update functions will not necessarily complete in that order.

How can we change these elements’ order in the DOM?

Now, here’s something interesting: Moving the elements around on the editor canvas doesn’t change their positions in the DOM. (Like, moving LS1 to the right of LS3 or moving it visually “above” LS3 doesn’t change LS1’s order in the DOM (its position on the canvas changes, but not where it lives in the DOM).

If we cut and paste one of those List Shifter elements, we essentially create a new element and that element’s DOM order will be lower than the ones we previously created and so it will initialize after them (e.g., we could cut LS1 and paste it and now the init order will be LS2, LS3, LS1). But of course, this will destroy any workflows using LS1 and cause other mayhem (should anything else refer to it).

However, there is a non-destructive way to change an element’s order in the DOM: If we do something like create a new group (so we have a newer element, which will have a lower position in the DOM), if we drag LS1 inside of that, now what will happen is the initialization functions will run in the order LS2, LS3, LS1 (and, indeed, if you inspect preview page using the Element view, you will see that the order of the elements in the DOM is LS2, LS3, and then LS1 further down inside of that new group). And doing this doesn’t break any references to LS1. Hooray!

So that all makes sense when you think about it, but it’s surprising when you actually first look into it!

Another thing we could do: We could completely control when a plugin element starts initializing and subsequently updating by setting it to NOT be visible on page load, but then setting it to visible based on some other event. (More on this at the end of this reply.)

QUICK EXPLAINER about initialize and update: In element plugins, we have a function initialize() that gets executed, as I said, just once, when the element becomes visible on the page. Once that is complete, Bubble calls the element’s update() function.

Initialize() is where we do things like, if the plugin is a visual element, we would set up our canvas (the visible part of the element), we might initialize certain variables, etc. Now, my plugins don’t usually have a visual element so they just have an empty canvas, but I also define all of the functions that my plugin needs in initialize and I also define all my defaults for things here and I put all this stuff on to the object called instance that Bubble functions can pass around. (Basically inside my initialize function there is a wrapper function, inside of which I define all my functions and stuff. Then, I run that function, which (handwave) basically compiles all my code.)

This initialize routine executes in a VERY short amount of time. (On my machine, the initialize function for List Shifter completes in somewhere between 0 and 1 ms.) Then Bubble calls the plugin’s update() function. The update function is where we read the fields defined in the main plugin interface. And, unless we do some clever scripting to prevent it (and there’s not much reason to do this, by the way), anytime that the field values change, the update() function gets called again.

What we do in the update function is read the values of the main interface fields and configure our plugin to do… well… whatever the plugin does. So, in my plugins, I read all your settings and (handwave) do whatever stuff needs to be done as a result.

At the very end of that function, I set a property called “initialized” to true, publish that to the plugin’s “Initialized” exposed state, and then trigger an “Initialized” event (sometimes I call that event “Initialized/Updated” because the event triggers on the first update as well as subsequent updates).

So, List Shifter’s Initialize/Updated event means that (1) not only has the initialize() function run, but (2) the update() function has also completed, and now we’re ready to fire Actions at the element. (This is true of all of my element plugins that have Actions.)

// END EXPLAINER

Now, you might think that, because JavaScript is single-threaded and the update() function is just a normal, run-of-the-mill synchronous JavaScript function, that our update routines will always complete in sequence.

However, there’s one complexity: If any of our fields are Lists or other types of objects (like Things) and we need to fetch them, Bubble will in most cases have to go to the database to get them (unless something else happens to have already fetched the same data). And what happens is that Bubble asynchronously begins fetching the requested data. This data comes down in chunks and Bubble will actually interrupt the update() function until all of our requested data is loaded. (So literally what happens is that our function gets called multiple times and subsequently interrupted, until the data loading is complete. If you’re interested, you can learn more about that in the Bubble docs here.

The net-net of this is that let’s say that LS1 is loading a big ol’ honkin’ list of data (like, a Search whose results are 1000 Things). If LS3 is just loading a tiny list, LS3 may easily complete its update() function well in advance of LS1, and maybe even in advance of LS2.

So: The elements’ initialize() functions are sequential and an element earlier (higher up) in the DOM will start and end that function before one later (lower down) in the DOM, but those events are not separated by very much time in most cases.

And while the update() functions will start executing in the same order, depending upon what the plugin does, those functions may finish in a different sequence.

So, we could imagine a case where we’ve got several (maybe many) List Shifters on our page and most of them are loading little things (maybe just a scalar value or a small list that you’re using to feed the headers in a table or something), but one of them is going to do a big-long search.

Let’s say that big-ass search is being done by LS1 (doesn’t matter where it lives in the DOM). We could it to be invisible on page load and then make it visible once all the other LS’s have gone to Initialized. (e.g., workflow trigger “LS2’s Initialized and LS3’s Initialized and LS4’s initialized” → show element LS1)

While I don’t think that this will result in LS1 becoming fully populated any sooner (in fact, that’s sort of impossible), it might make for a better UX as all of the things that depend on the “lightweight” List Shifters will be happy. You could test the difference between these scenarios using Debug Buddy to benchmark the time it takes for LS1 to reach the Initialized state in both scenarios.

4 Likes

Thank you again @keith for the further insight. That actually helps me out a lot in understanding more about the initialized state on the list shifter element, as in the past I had struggled a bit with dependencies between different list shifter element’s on the page.

Cool. BTW, chaining List Shifters together (or just using List Shifter to watch some other an expression for changes) is one of the most useful applications for List Shifter.

1 Like

Hi @keith , thank you for all your explanations. Your inputs really helped us a lot. :smiley:

I have a followup question. Here’s my scenario.

I have a Customer thing and a Booking Request thing. Both things have fields: first name, last name, birthday and another field that has ”fullname and bday STRING”

The flow is, customers can request for a Booking without actually making a customer record. But once the admin accepts the booking request, that is the only time that booking is converted to a customer thing. But before creating a customer record, the system checks for possible duplicate of that customer.

So I need to have duplicate checker (considering some typographical errors upon booking requests, or some users not inputting their whole/full name (i.e. inputting John Smith, but the full name is John Arnold Smith).

My initial plan is to use list shifter to get all the customers then sort them based on string similarity (https://www.npmjs.com/package/string-similarity#findbestmatchmainstring-targetstrings)

I’m not sure if this is the quickest and most efficient approach, but can I do that in list shifter?

You’re kind of on the wrong track and you don’t need to do what you’re proposing. I’ve seen your other posts in the forum.

You should just let anybody request a booking. They don’t need to be Users (capital U) of your app to request a booking.

Just let them request a booking.

If you insist on making (booking) users of your app have accounts, then just do that up front. Then you can uniquely identify them. (Because they will be uniquely identified by their email or login method.)

Yes, anybody can request a booking. Customer thing is not a User thing. It’s just a record of every client that already booked so we can track each client and check all bookings in client’s profile page. However, a customer record cannot have duplicates for our use case. That’s where duplicate checking comes in.

Hello! I’m having trouble implementing the plugin. My task is simple: move the elements in the repeating group up and down. In the figure, I depicted the algorithm that I am trying to implement. But I have a problem: it works every other time. What do you think is the best way to do this task?

So @rshatskiy the issue you’re having is that, in List Shifter, the “shift” operations like Shift Item are asynchronous. That’s by design, BTW. I’m mobile at the moment, so cannot recall if Shift Up/Down triggers a “shift complete” event (but I don’t think it does).

The design intent is NOT to support doing a database operation after any of the shift functions, but to do them in the page and then save/submit when the user is finished messing around (via a save/commit button press or similar).

If you want a synchronous version of this functionality, check out my new Floppy plugin, where these operations ARE synchronous. (By design.)

That last post reminds me: Avid users of List Shifter will likely be interested in my new commercial plugin, Floppy, which offers synchronous versions of List Shifter’s “shift” actions, and much more.

Forthcoming videos will explain and explore some of the key differences between Floppy and List Shifter. While they have some overlapping functionality, they have rather different design intents and different approaches to accomplishing some of the same things. Check it out here:

1 Like

@keith thank you very much for the answer! Now I’m trying to implement this task in your new floppy plugin. I figured out how to put the list I need in localstorage. Do I understand correctly that I should use the action SHIFT: Move RAM List Item Floppy? In order for me to shift the value down, do I need to specify the current index and index +1? The problem is that I can’t change the order of items in localstorage. Could you briefly describe the algorithm for solving the problem of moving list items in the new plugin?

Hey @rshatskiy! So, in short, just Set the RAM List values to be Floppy’s List (Storage). Then do whatever Shift functions you want on the RAM List. At the end of that, store the value in storage ( note the yes/no field at the bottom of all the Shift actions that let you take the updated RAM List value and immediately write it to storage… though you could also execute a Store operation separately. There’s no real big difference).

BYW, the general idea here is that you would assemble whatever list you want to Store in the RAM List and then Store it. That’s what it’s there for, but I haven’t demonstrated that as yet in a video. Thanks for using Floppy!