Custom state for backend workflow?

Hi,

My app has a number of integrations where users can click a button to sync data between the app and an external service. Is it possible to have a custom state for backend workflows? This way I can build a sync button that shows loading until the backend workflow finishes. Otherwise it looks like nothing is happening.

I downloaded the plugin list pusher by @keith , but honestly I feel a bit too stupid to figure this out based off the guidance I’ve seen. If anyone knows how to use this or has an alternative it will make my week!

Thanks

2 Likes

So, in this particular case, you don’t need a “custom state for a backend workflow” (there are cases where one needs one - and the simple “Flow State” SSA actions in my “List Popper & Friends” plugin are literally that).

But, in your case, all you’re trying to do is record the state of something being synced in a way that you can report that to the user and see that state in the page. This is what your database is for.

One very simple (but probably not very smart) way to record this state would be to write it on the user. (I wouldn’t do it this way, but just for the sake of explanation, I’ll continue…) So you’d go into the Data section of your app and add a field to your User object. It could be as simple as a single yes/no value, “Syncing”.

In this case, the first step in your backend workflow(s) would be to make a change to the Current User, setting the value of Syncing to “yes”. Oh, look, now the User is “Syncing”! The very last step in your backend workflow(s) would be to set that back to “no”. (If the entire backend process spans multiple workflows, we may have to pass the User along to any downstream workflows so we know what User we’re supposed to edit later.

In the page, now you can just look at Current User's Syncing. If it’s true, that User kicked off some sync operation and it’s still going. When it’s false, there is no sync going on.

While the above is simple to understand (and pretty simple to implement, right?), it’s probably not the smartest way implement this. But you can just go try that right now and see that you can do this. Then go and remove the “Syncing” field from the User datatype. Here’s what I’d actually do…

What I would be inclined to do instead is create some object that represents a Sync operation. So we might make a new type of Thing (a new custom datatype in your database) and it might be like this:

It’s a very simple Thing - in addition to the built-in fields (which - importantly - include “Creator”), we just added one field of type text for “Service”. This field might better be a Thing itself (e.g., a Thing of type “Service”), but the idea here is to be able to differentiate between which service for which syncing has started. (Your post notes that there are a number of services that might be being synced to.)

(One other aside: If you don’t need Users to be able to see Syncs created by other Users, you can set this Thing to be private when you create it. If you do this, Users will only be able to see Syncs that they created. It’s very handy if this is the case, BTW, as you’ll see below.)

It could have more fields on it. For example, the Service might not so much be associated with the User, but maybe with some other object like an Account of something, right? But at a minimum we want to know which Service.

Anyway, in this scenario what we would do is: When the user kicks off a “sync” operation, the first step in our workflow is to “Create a New Thing”. We create a new Sync and set its Service field to some standardized name. Like, if we’re syncing “Service A”, we’ll create a new Sync and set its Service field to “Service A”.

This new Sync thing will be created in the context of the User who kicked it off. So this Sync’s Creator will be whatever User is Current User (in the context of the page).

At the end of our backend workflow or workflows, we will then DELETE the Sync that was created in the first step. (In the case of multiple workflows, we may need to pass the Sync to downstream workflows so we can delete it, right?)

So now, instead of there being a boolean flag on the User that represents syncing, we have these Sync objects that only exist while a Service is being synced and then are destroyed when syncing is done.

In the page, we can know if the User is syncing some Service because there will be a Sync object that was Created By them. If Sync things are private, the only Sync things the User can ever see are Syncs that they created. That is, any page in our site can know that the current user has a Sync going. The expression:

Do a Search for Syncs

Will return a list of all Syncs that are in process for the current user. And when there are no Syncs going, that list will have no items in it.

If we want to know if there’s a Sync for some specific Service, rather than for any Service, we could instead for example: Do a Search for Syncs [with condition Service = "Service A"].

So now in our page our, “Start Sync for Service A” button can have a condition on it like, when Do a Search for Syncs (with condition Service = "Service A") first item is not empty, we can make the button unclickable and change its text to “Syncing…” (or whatever).

And when that Sync object is eventually destroyed by the backend workflow (at its end), that Search will magically go empty and our button will appear available again.

Of course, there are any number of other things we could do to alert the user that their sync has finished. Note also that we could get fancier with our Sync objects and they could have other fields on them (for example, a boolean [yes/no] field “Complete” that we set to yes when the sync is finished, instead of just deleting the Sync object right away).

9 Likes

Hey Keith,

Thanks so much for the lengthy reply. I really appreciate the effort in your responses and using your time to provide so much detail.

I replicated what you asked and I do remember I tried something like this before. What I didn’t actually make clear is that I’m using additional API’s on lists. So what I’m doing is integrating from mailchimp, getting a list of users, and then running another workflow on that list I’ve got from mailchimp.

The issue here is that it’s creating syncs in the DB for each list item, which is causing stop and starts of the sync animation. Is there a solution towards this or is it a case of accepting it in it’s current state?

Thanks!

Figured it out! All I did was create a list field and populated it with the list from the integration API. Then at the end of the list API, I did a search and removed the item that is being processed from the list within the sync item. That keeps the button animation syncing until the sync object list is completely empty.

@keith couldn’t have done it without your initial guidance, so really appreciate your help.

2 Likes

Just adding a comment here to say thank you for your elaborate answer that helped me today :pray: