Custom States Within WorkFlow Out of Order for TTS

[tahycoon]

Hello dear Bubblers,

I’m writing this at midnight after spending an entire day rebuilding a lot of my app logic.

In summary, I am building an AI assistant (just like Siri!) and so far so good, but one problem I am facing is my Text to Speech workflow is just running completely out of order and ruining everything.

Basically, I have a speaker button that when clicked runs an OpenAI TTS call, set a state of my bot main group to that audio file, saves the audio file, and then runs that audio file (I used to do it with autoplay in an html but discovered that it breaks in IOS since it blocks autoplay, so I figured a way to use an action to save that file and then run it in an action instead of an autoplay).

Things to know first:

1. Everything else is working fine, I pinpointed down to the TTS workflow.

2. I am using the AudioPlayer plugin to run the files received from OpenAI, the reason I am not using a normal HTML to autoplay this is that IOS blocks autoplay, and this plugin is the only way I found where I can manually play a file inside a workflow without autoplay or user interacting with it (per IOS rules).
   I don’t think I’m facing any issues with the plugin.
   (if you know a workaround for the autoplay for real-time dynamic files for IOS let me know!).

3. IMPORTANT: the workflow runs flawlessly in the Slow and Step by Step modes, but in normal mode, it completely flops. I searched this forum and found similar incomplete posts mentioning that it’s because setting states inside a workflow can get out of order and workflows get out of order as well.

4. Although I eventually want to have my bot respond automatically I think I would get that part working if I can get this issue fixed first --having an out-of-order workflow with states and API calls.
   (and I don’t want to make the post too long)

This is the button clicked for the bot to read the message:

This is the button workflow (the states here are just used to display a loading screen on my bot so don’t worry about it):

This is the workflow for my custom event I included on the previous button workflow to retrieve OpenAI TTS data, set the state of my bot tab to that audio file, set up my audio track plugin to that file, and then play the file.

(the reason I have Set Track Audio is because if I have my audio element receive the audio file from the custom state directly on the homepage without an action then I have to set it up as an autoplay which means it won’t work for IOS as I mentioned):

This’s my set state action:

And here is my audio element on my homepage (nothing to see here really)
This’s simillar to having an HTML that auto plays your audio file from your custom state set on the bot, but then it won’t work on IOS which is why I prefer to manually call and play the audio as I did.

And here is a video demonstrating how my workflow runs well in step by step/slow and when it doesn’t in case it was long to read

My TTS WorkFlow Issue

If you can provide me with any help that would mean the world to me because I really struggled today rebuilding all the workflows and stuff

[boston85719]

@tahycoon

I searched this forum and found similar incomplete posts mentioning that it’s because setting states inside a workflow can get out of order and workflows get out of order as well.

This is true…workflows do get triggered in order, but are not executed in order, which means step 2 might not have completed before step 3 gets triggered.

Ways to ensure the workflows execute in the order you need them to include using the result of previous step as a datasource for subsequent steps that require the completion of the previous step OR to use a custom trigger and put the actions that require completion inside of the custom trigger event, then use that custom trigger event in the main workflow action (so if step 3 relies on step 2 being complete, put step 2 into a custom workflow trigger and in the main workflow trigger that custom workflow trigger as step 2 so that step 3 will have the necessary data available as the actions from custom workflow trigger will complete prior to the step 3 in main workflow series getting triggered).

You can also leverage conditional workflow triggers to run the actions that require the values from previous steps, so might be something like ‘when custom state value is not empty’ if you have workflow actions that require the custom state value to be set for them to run properly.

Looks like step 3 of setting the track of the audioplayer is the issue. If it were me, I’d setup the conditional trigger that ‘when custom state of Group AI tab’ (ie: your custom state from step 2) is not empty and in that conditional trigger I’d set the track of the audio player and toggle play/pause audio player.