Forum Academy Marketplace Showcase Pricing Features

Does one workflow need to finish before another will run?

hello,

Here’s a scenario I’m facing. I have a backend workflow that is recursively pulling in thousands of rows of data from Airtable and creating a new row in a bubble table. This is taking a while. Actually it’s coming in at a rate of 96 rows/hour. :grimacing:

I have a database trigger set up for a different table so that when a new record is created, or one updated, it will update/create a new record in Airtable. (Not the same Airtable table mentioned above.)

I’ve noticed that while the workflow mentioned above is running, the database trigger doesn’t fire. Does the first workflow need to finish before other ones will run? If yes, will the trigger(s) be held in que to run after the above workflow has completed?

Thanks,
George

A couple of questions;

  • When you say it doesn’t fire, is this according to Bubble logs or you just not seeing the records be added to Airtable?
  • What’s your app capacity looking like while this recursive backend workflow runs?

From my experience, when Bubble is under strain from a recursive backend workflow, other functions can often suffer and sometimes drop completely especially if your recursive backend workflow is taking hours.

Besides the fact that I wouldn’t recommend this kind of database duplication in the first place, will your app regularly be trying to pull in this many records or is this just kind of an initial pull to populate the Bubble db and once this is done, you’ll see less come through each day?

Hello @juicebox

Thanks for your response. That’s good confirmation of what I was thinking.

A couple of questions;

  • When you say it doesn’t fire, is this according to Bubble logs or you just not seeing the records be added to Airtable? I didn’t see the recorded added to Airtable when the new bubble record created, and I didn’t see it added when I edited the new record.
  • What’s your app capacity looking like while this recursive backend workflow runs?
    See below for my app capacity.

image

Besides the fact that I wouldn’t recommend this kind of database duplication in the first place, I agree 100%, I’ve avoided doing this for months, but the performance of trying to use strictly live Airtable data for each operation and form view is atrocious. I’ve given up on that strategy even though it’s what the customer wants. will your app regularly be trying to pull in this many records or is this just kind of an initial pull to populate the Bubble db and once this is done, you’ll see less come through each day? Yes, this is only an initial pull to get all the data into bubble. Once I’ve duplicated all the airtable records and the fields I actually need, I’ll run daily workflows to pull in and update bubble data that is new in AT, or updated since the last pull.

These airtable tables are designed very poorly and have dozens of calculated fields that really slow down the API performance.

SO then, basically, I need to wait until after thanksgiving to run any other Aoi workflows so that this one has the time to finish sometime after Thanksgiving LOL.

Thanks for your help. If you see a potential problem with what I stated please let me know. Maybe I’ll send you a Thanksgiving turkey. :slight_smile:

Thanks again for your help!
George

Okay, can you see the database trigger firing in your Bubble server logs? If not, then you know it’s a Bubble-side issue, but if it’s firing and sending the information to Airtable, then perhaps Airtable is rejecting the call because it’s too busy servicing requests from your recursive workflow. I find the latter unlikely but it’s one way of verifying whether the database trigger is firing or not.

Instead of the workflow graph, can you check the ‘Periods where the app hit its maximum capacity (%)’ graph? Also check how many minutes the app hit its maximum capacity usage. See below for an example.

I’m assuming the customer needs the data in Airtable and has no appetite to move it entirely to Bubble.

Okay cool, so this really is a temporary thing.

Is there any value in improving the database structure in Airtable so that your live calls from Bubble are more efficient?

Good morning,

After following your suggestion of looking at the server log I discovered I had a simple field type error that was causing the call to fail. Learned something new here…something I should have figured out long ago. THANKS!

Regarding the Airtable structure: No chance of that. Long story-short, the guy they have put in charge thinks Airtable is the bees knees, and isn’t even in favor of the app. He’d rather have everyone enter the data directly in Airtable. I sold them on me building an app, but the SOW document stated no data is to be stored outside of Airtable. I found that impossible considering how the AT data is structured. That’s why I’m going this route. It’s been approved unofficially because most of the “team” wants the app and nothing to do with Airtable. Politics. :frowning:

I am soon to post another question that will show the complication of this project.

Thanks again for your help!
George

This topic was automatically closed after 70 days. New replies are no longer allowed.