Forum Academy Marketplace Showcase Pricing Features

How to avoid duplicate data from API pull

Hi, appreciate any advice here. I am pulling in data from a parsed email via a webhook. The pulled data consists of multiple activities associated with a single event. For example, a single instance of a pull may have three items:

Activity1, Event1
Activity2, Event1
Activity3, Event1

The first step with the data is to create a new Event1 if it doesn’t already exist. To do that, I have ‘create a new Event’ set to only run when a search for Event is < 1. The search looks at two fields from the pulled Event that would be unique to make sure it’s not already in the system.

This works fine to ensure no duplicate Event1 is created, and I can then go ahead and create new Activity1, Activity2 etc that are each linked to Event1. However, when Event1 does not exist, the workflow proceeds to create an Event1 for each of the Activities. In other words, it seems like it’s not re-checking after creating Event1, so I end up with three Event1’s in the above example.

Is there a way to have it stop the workflow, or re-run the search, after a single Event1 is created?

Thanks for your help. Very new to this and I have tried to find the answer in the forums but couldn’t find this exact example.

EDIT: It sounds like the same problem found here, but can’t see if a solution was ever found. Annoying API bug when saving it creates duplicates the first time it runs - lag in the workflow!

I take the API data and send it into a backend workflow. In the backend workflows I use a plugin called ‘list popper and friends’ which allow you to take a list of things and remove the first item from the list. The data will be available for subsequent calls to utilize in two formats

  1. Popped ‘thing’
  2. Remaining List

I use these to perform the actions of making the data entries in my database during a series of recursive backend workflows.

If I am anticipating the user experience to be one in which the user will be needing the processed data during the same session I setup a data structure to allow me to be aware of when all the data has processed so I can allow the user to continue in their session with the processed data available.

1 Like