Staggering Workflow Runs to Avoid Duplicate Search Results

We have built a dialer for our team to call lead lists. The workflow is fairly simple - when a call needs to be triggered, we search and find the lead with the “most recent call attempt” with “descending no” as a sort, update that lead’s “most recent call attempt” field, and then schedule that lead to be called.

The issue is - when we have multiple users active, this workflow is often triggered multiple times simultaneously which results in the same lead being found in the first search step before the next step runs (which updates the leads “most recent call attempt” effectively putting that lead reference to the bottom of the list), and therefore the same lead is called twice at the same time

To combat this, we have placed a random number generator at the beginning of the workflow and then referenced that random number in a workflow delay before the steps to try to stagger the workflow from running simultaneously and ultimately reducing the amount of times the same lead reference is found in the search step. It’s not working…

Any creatives ideas to force this workflow to not run simultaneously and/or force the workflow to find a different lead reference in the search step?

1 Like

Hi @josh29, interesting problem!

It sounds like a variation of the race-condition from assigning sequential IDs to new records.

One key point; when creating or updating the DB, a workflow doesn’t know which client “got in first”, but it can rely on the database determining that records are updated in some sort of order, i.e. the created/updated dates won’t be identical.

Would you be able to add a field to Lead? example: assigned campaign

Step: Update lead, (search for lead, constraints: assigned campaign empty, sorted) first item, set assigned campaign;

I believe this is exactly what we are doing with our “most recent call attempt” field in the search.

Step 1: Search for lead (most recent call attempt, sorted descending no)
Step 2: Update search result of step 1’s most recent call attempt to current date/time

The issue is that step 2 must not be triggering quick enough (or if the workflow runs are triggered at an identical time), because the same lead is being found very often in that search.

We have added workflow delays on workflow delays, but maybe we haven’t set them up correctly or they don’t operate properly in backend workflows? What I’ve done for delays is generated a random number (1000 - 10000) then second step is workflow delay using the first step’s random number.

You would think that this would 100% solve the issue as it’s forcing the workflows to stagger, but it doesn’t effect it all. I’ve even added 2 workflow delays back to before doing the lead search and still no change…

Throwing in an idea …

Currently a search can return a list of available leads in date order.
Add to this a search which returns a list of available campaigns in date order.
Call a recursive backend workflow which searches and picks the oldest in each list and pairs them. Call itself if there are more available of both.

We found a solution - not the best solution but far better than nothing.

What we did was instead of searching for “first item” in our list search we are searching for “random item”.

Before indicating we want a random item however, we are using a filter which filters out any “bad” leads we don’t want to call at that time (for example, any leads marked as “do not call” or any leads who’s “Last retry attempt” is not long ago enough.

It’s not a perfect solution because it doesn’t exactly get us the perfect lead in the list that we want (the lead that was called the least recent ago), but it almost completely solves the double lead firing and with the filters on the lists being searched we are at least getting unique “callable” leads in each search.

1 Like

Sounds good!

If your app really needs a rock-solid solution, a third-party database is ideal, and can be cheap too for the tiny amount of storage needed.

Such a database can solve race-conditions by the use of any of these:

  • atomic transactions / locking
  • insert row, check for row position when order by date
  • auto incrementing columns (intended for unique primary keys)
1 Like