Forum Academy Marketplace Showcase Pricing Features

Iterative workflow - how to process a list of data? (SOLVED)

I am evaluating Bubble as a rapid development tool for building a specialized task management app for our organisation and could do with some help. I have undertaken the tutorial.

I’m looking to understand how in a workflow to iteratively work through a data-driven list taking action on each.

As it relates to my application, the requirement is to quickly deploy a predefined lists of tasks against a specific client.

I have modelled two separate data types, let’s call them Tasks and TemplateTasks. The reason they are separate is that they have quite a different set of fields. Tasks are assigned to individual users, have due dates and an audit trail etc, whereas TemplateTasks have only relative timings and roles.

I’m trying to create a new list of Tasks based on the selected TemplateTasks. This would be a 2 minute job in SQL, but I can’t see how to achieve it in Bubble. Copy list only allows you to copy items of the same type, and there is no “create multiple things” that I can see.

I can’t see an iterative workflow where I can work through a list and “create a thing” in each.

Iterative workflow will be important, because when we change a template I will have to iteratively work through all related tasks and decide whether to delete, update or leave alone.

1 Like

I have actually found a way to create iterative workflow but it’s ** unusably slow.**

There must be a better way, right?

  1. I create a conditional workflow on page load with a condition that there is anything in the iteration that needs to be done.
  2. If there is I select the first TemplateTask that needs to be copied and send it as data to another page.
  3. This other page loads all related data on page load and then immediately creates a new Task.
  4. It then redirects to the page in 1, which checks again if there is anything to do
  5. The page in 1 has a condition that if nothing does need to be done it forwards off to another page where the task list is displayed

For a production server creating maybe 50 tasks in a project that is too slow and very wasteful of workflow units. it would be 101 redirects to create all the tasks.

Is this a list that the user selects from ? In which case you could create the new things as you go.

Can I batch create a set of new records from a repeating group, for example - is that what you mean?

I can make the repeating group, but what would the workflow look like?

I thought of doing this but other postings in the forum suggested that you can’t create multiple records from a repeating group using a single button - you had to have a button in each record so multiple clicks, which is definitely too laborious for this task

I was just thinking about your use case.

Is this a situation where a user is presented with a list, and they select multiple elements they want (e.g. with a tick box) or a selection button ?

Or are they selecting the template from a list, and that then creates all the elements ?

The latter. The user selects a template and all the tasks are created. I could of course make it work the other way - show a list of the template tasks and have the user hit save. I would be willing to do this but I still can’t see a way to bulk create the new items. Do you know of a way?

I’m considering implementing some JavaScript to do this - should be fairly straightforward - but feels like I really shouldn’t need to. CORRECTION - doesn’t like it would be straightforward owing to the lack of native JavaScript support.

I do think I may be missing something.

If you are looking for something that does “Create an xxxxx for everything in yyyyyyy” then no.

Bubble doesn’t do that directly from a workflow. Josh has written a post about the current architecture, and as we have shared resources they are sensibly worried about a process like this running away with itself and affecting everyone.

You can do “iterations” from a bulk action however.

And I wonder if you could use the “Copy List of Things” and then “Make Changes to a List” to achieve this ? Although I appreciate there would be redundancy of data.

I am looking for Create an xxxx for everything in yyyy". I think that’s a serious weakness in the system - not so much from my perspective but for Bubble’s main purpose of building a fast, interactive app. It makes it impossible to do batch edits, e.g. where you present a Repeating Group populated with input fields and create/edit multiple records at once - so much faster as a form of UI.

No, it isn’t impossible at all.

It is quite simple to do batch edits, you just “Make Changes to List Of Things”. The list being the Repeating Group. Or some sort of temporary list that you create.

As to the original issue, this would be possible by combining your two data types, and having a “type” field with “template” and “task”. You can then copy the list and immediately do a “Make Changes to List Of Things” on that list to change it from a “Template” to a “Task”.

Whilst not ideal, it would get you a step further on.

Yes, some sort of create iteration in a workflow would be useful (we already have an admin function that works in that way) but it can’t be at the expense of being able to create a runaway data creation workflow that brings everyone down. I when I say useful I mean useful, not essential. If you can do it by being a little creative, albeit in a way that doesn’t seem all that “programmer-ish”, that is fine for most people.

That said, often when I say “you can’t do it” Bubble release something a few days later that does exactly that :slight_smile:

3 Likes

Well I read each post, that’s why:)

On that specific thing, what we may do is enable scheduling an API workflow on a list of things, not just one thing. That would let you run iterative stuff, but won’t be instantaneous (because then we can have some safety between each item so that it doesn’t take bubble down).

6 Likes

:smile:

So like the manual Bulk API but scheduled from a workflow on a list ?

I think that makes a lot of sense.

1 Like

So we just added this. See this

This can do a lot of things, but be careful with:

  1. it’s not instantaneous, it can take a few min/hours if the list is very large
  2. it counts against your workflow runs, for each entry of the list
  3. if you make something stupid a lot of workflows could run…
3 Likes

Whaooo ! Finally :grinning:

1 Like

haven’t you set any security on this, like requests/min or total requests/day/action ?

1 Like

Yes of course, the interval between each item depends on the size of the list. But in these are behind the scene stuff.

2 Likes

Thanks. I will check this out now for solving my original problem. It’s good to see you guys be so responsive. I think this was a significant missing part. And like you say you still have control of the overall load.

1 Like

OK, it works brilliantly for what I originally wanted to do.

So far performance has been good. It has created the things from the list batch of tasks for each project - about 16 in my current sample - before the next page load.

It will have several other applications in my project.

1 Like

@emmanuel. For “Schedule API Workflow on a list”, the list of things to run on seems to evaluated when the action is definied. Would it be possible to evaluate the list WHEN at the schedule date ?

For instance, I would plan to send a personnalized email to each participant of an event, one day before it starts. I would define this action when the event is created, so when there is no participant yet.

If the source list is from a Repeating page on the page, I understand the concern. But for a Thing’ List of things, simply defining the parent Thing would work.
This would able to handle more uses cases than right now.

It’s not a simple change, and more complex to implement.