Finally, my API data refreshes without page reload- How I got it to work

Refreshing a repeating group has been covered so many times over the past few years… One area that I’ve recently had major issues with is refreshing a repeating group from API data. The data would always be the same - Only a hard page reload would give me fresh data.

My case: My app grabs tasks directly from our Wrike Ticketing System that are a particular status via the API. It does this in real-time, with no Bubble Database needed. When my app loads, it goes and grabs all of the current tasks with status “Escalate” in Wrike and slaps it in a Repeating Group. Standard stuff.

Now, since I’m not using a Bubble Database and only grabbing “real-time” data from Wrike, if a status changed in Wrike, my Bubble app wouldn’t know… Unless the user did a hard page reload. Not ideal.

I went through the full gambit of “refreshing a repeating group”… I tried placing the repeating group in a group, resetting the group above the repeating group, blah blah blah, on and on it went with trying all of the different solutions in the forum. Nothing worked.

The solution: As in the post above, I restructured the data to come from a Custom State vs. a Data Source on my Repeating Group. Then, when I wanted to refresh the data, I cleared the Custom State, then replaced the Data in the Custom State… A bit confusing. Here are some pictures:

First, I placed my Wrike API data into a Custom State on Page Load.

Note: When this data is pulled from Wrike on page load, it is cached, meaning it does not refresh unless there is a fresh page reload - A way Bubble can speed things up by reducing the amount of data traffic. So, I had to find a way to “bust the cache” as @mishav mentioned in the post above.

For testing purposes, in my app, I created a manual REFRESH button. When clicked, I wanted it to grab fresh Wrike data without having to do a page reload. This would then pick up the fact that if a task no longer existed in Wrike, it would be removed from my app immediately!

To do this, I plopped down a Refresh button, and then the following workflows on said button:

First workflow step on Refresh button: CLEAR the Custom State that used to have my Wrike API data in it.

Second workflow step: Exact same as the page load. See screenshots below.

By “clearing” the custom state before setting my Custom State again, it was finally able to refresh the data without requiring a page reload. I can go to Wrike, change the status of my task, click “Refresh” in my Bubble app, and it works!

Hopefully this helps anyone.

20 Likes

Update to this post - There’s a MUCH easier way to approach “busting the cache” that doesn’t even involve custom states.

Backstory: The method above was no longer working with new workflows I was creating… So, I sent in a Bug ticket and got great help from Bubble.

The new solution to refresh data from a 3rd party API without a page refresh: Add a “Date” header to your API call as seen in the picture below - Set the current date time w/ Bubble. Notice I did the simplified ISO. That’s so it gets down to the second vs. the minute.

Adding “Date” header now tells the API to grab data for that very second - Which means I get a fresh crop of data!

So, the above solution with the custom states and all of that is overkill - I wouldn’t recommend it anymore. I am now displaying the list of data right in my Repeating Group. Much much easier.

Enjoy!

OLD WAY

NEW WAY

29 Likes

Hey @w.fly,

Thanks for the insight into achieving a more native-esque interface. There’s just a few considerations to make with this approach that may help teams decide if it’s the appropriate one for them:

  • Iterations: Bubble watches dynamic properties for changes, and opts to re-evaluate data that relies on those properties. Since time is always changing by the millisecond (in UNIX terms), Bubble is having to re-evaluate values thus making this approach possible. As Bubble watches for values changing rather than significance, you’re responsible for controlling the interval.

  • Bounds. Since time is going to infinity, the re-evaluations will not cease unless a condition or scenario steps in to make it so. By default, this function will keep executing forever without bounds. An endlessly-updating function may or may not be the desired outcome for some cases.

  • Performance. Making networks requests of any kind, be it for a native Bubble function or third-party service, will be run through Bubble’s server. Naturally, Bubble also handles the parsing of data returned by services to make it workable within the Editor. Each request uses resources that do count against your available capacity and may effect user experience, which may not be ideal.

  • Rate Limiting. In order to protect resources and prevent crashes, modern API architectures typically implement series of measures and actions. Rate limiting is one of the most popular methods of ensuring system stability, and are usually stated clearly in documentations. Not having enough bandwidth to deliver data requested by users may require an altered approach.

    Example: Wrike states a rate limit of “100 requests per minute”. Assuming Wrike’s limit is account-based, just two users being present on your page would exceed your limit in 50 seconds; this doesn’t take into account any calls your own application may be making to keep things running.

Just knowing how Bubble re-evaluates (as you shared) can change how you build great interfaces. :sunglasses:

9 Likes

Excellent things to consider. Thank you! It’s fun for me to compare Bubble with the “traditional” programming world - And how similar some of the issues are. Items like rate limiting in 3rd party APIs exist as a crutch to both Bubble and “traditional”.

Luckily in my case, only 1 person is using this app. Thank goodness… :slight_smile:

Can’t you just use a webhook, rather than polling Wrike constantly? https://developers.wrike.com/documentation/api/webhooks/overview

4 Likes

Yea I’m sure I could - Just haven’t invested the time. For my app, I only poll the API once per page load - Refreshing the data from the API only happens rarely. And honestly, if I wouldn’t have found out how to refresh the data manually, a page reload is a nice and quick “ctrl + r”… :slight_smile:

1 Like

Gotcha. Previous posts made it sound like u were overdoing it on the polling thing. Clear now that you are not. Doing things entirely in the browser needn’t be scary.

Hi @w.fly, @keith

I got stuck at some point with my app which I am working.

I am using SQL connector and fetching some data from MySQL and performing CRUD operations (Create, Read, Update and Delete) .

When I run any of the query, I am not seeing fresh data until I refresh my browser. Can you please help me with this ?

Thanks.

PM: On above you mentioned that pulling data using ‘date’ will work, as we can’t use headers in SQL connector plugin how can we fetch the data without refreshing the page?

1 Like

Hey Sam!

Well, I’m not sure how the SQL connector works, but maybe showing you all of the pieces of what I did will help get you to the next step:

  1. In the API Plugin, I added in a “Date” header into my API call. Make sure to uncheck Private - You want to be able to access this in the Bubble editor.

  2. Next, when I made my call to the API again, you’ll notice that there is a (header) Date field… Which I then filled with the Current date and time. See screenshot.

  3. Finally, take a look at the formatting - I formatted the Current date and time to be ISO. See screenshot.

Sorry I can’t get deeper into specifics into your individual case with the SQL Connector plugin - But hopefully this helps. The methodology above has worked for me for the past few months.

Hi @w.fly, thank you for your investigations, it saves me a lot of time !

As the 2nd option can be very heavy on Bubble and my API performances, I would prefer to trigger a workflow action every time I need my repeating group data to refresh.

So would you say that your option 1 using states as data source is the only way to achieve this ?
Until now I was also refreshing the whole page, which is not very UX friendly.

Thanks !

The 2nd option is actually only triggered from a workflow when I need the repeating group to refresh - And it works wonderfully. So it only runs if this workflow runs (it doesn’t run every second - Which, you’re correct, would be very bad for performance!!). See attached image for reference :slight_smile: I don’t believe the state method works anymore… I’m not quite sure.

Hope that helps!!

1 Like

Ok I see thank you.
Correct me if I’m wrong : I need to leave the repeating group data source blank, but only pushing my API data with a date header and using a worklow when needed ?

I don’t - When the page loads, the repeating group has the data source in it. I know it’s odd because all through the forum are posts saying the data source has to be blank and that the “Display list” doesn’t refresh. In my case, keeping the data source on load worked just fine. Maybe the “Date” field being dynamically different each time a call is made has something to do with it?

Thanks @w.fly
For me, it finally turned out that I just have to re-display the list in a workflow, with or withoud a date header in the source API apparently.
I was pretty sure I havetested this before but apprently not…

2 Likes

Hey @sam, did you get an answer?

I’m currently developing an app that heavily relies on the SQL Connector plugin and encountered the same problem then you. I found my answer on the forum; it’s quite simple.

In the SQL query you want to be able to refresh, simply add an extra parameter (that won’t be part of the SQL statement). I named mine “dummy” and the type is number.
Now, whenever you need to reference this query, in addition to your necessary parameters, you include the dummy one with the value Current date/time:extract UNIX. As mentioned above, this will make Bubble repeat the query instead of just sending you back the cached result.

3 Likes

Hi @julienallard1

Thank you so much for your response.

Yes, I got it worked.

After couple of hard working days later I realized that time is the only parameter we have which keeps changes.

Anyways again thank you so much for the response you gave. If we have people around us like this to give helping hand, We can keep going with bubble.

1 Like

Is this still working because I can’t seam to make my api calls refresh

The SQL method with a dummy parameter based on a random value like todays date in ISO format worked a charm for me today. Thank you so much.

1 Like

Yea, it still works. Make sure the format of the date is correct and that it’s in the header. Here’s what the call looks like on my end. Hope this helps.

Screen Shot 2020-07-31 at 9.04.05 AM

Screen Shot 2020-07-31 at 9.04.13 AM

1 Like

@w.fly

Hello!
Just see your screenshot and I’m wondering what’s your fomule/pattern to get the date as an API compatible ISO ?
I’m currently using a paid plug-in just to do this conversion!
Moreover, are you able to do this the other way around? I mean getting a date from this ISO format to Bubble date? Because I’m currently unable to save data from API call as date!

Thank you, this will help me a lot!
Have a good evening :slight_smile: