Is this done by using Workflow repeatedly forever? (get RSS feed content whenever they published and storing them in the DB)

Question
1

Hi everyone,

I want to get RSS feed contents and store each one in the DB when it have published , so is this done by using Workflow (that make a call for the RSS to check if there is a new content) executed repeatedly every minute or every 5 minute forever? is there another way that trigger a workflow whenever a new content published?

2

If the website you’re trying to track doesn’t have any webhooks or real time notifications, you may need to result to periodic scraping.

3

@ntabs
How can I know If there are webhooks or real time notifications?
I am trying to get the rss feeds content using a url like this:
https://wakingup.libsyn.com/rss

4

https://www.make.com/en/integrations/gateway/rss

No clue if these will work out cheaper than the WU of using Bubble.

5

@georgecollier
I am trying to avoid using tools like Zapier and Make, I am trying to figure out the original way that they use to provide there webhooks, that why I asking if this done by using Workflow executed repeatedly every minute forever

6

I added a functionality similar to RSS for an app a few months ago. Basically we do a call to twitter api to get the relevant tweets and we do this based on the maximum realistic tweets per period (3 hrs) based on the history of the clients account. It was a good work around given that twitter doesn’t have a webhook for the type of tweets we want to watch (not our own tweets).

With RSS you will most likely find that you can use ACID with make or zapier to hold the position in the list so that when you repoll the list it will only return the new entries. Otherwise in bubble you would need to add a filter to ensure only new entries = new data.

Doing it every minute in bubble will blow up your app workload units so it will likely be much cheaper using make to send the data to bubble and running the check every few hours or daily.

7

@mitchbaylis
So, if I understand it, the repeating workflow that check if there is new entry is the original way for this, and this is the way that used by Make and Zapier to provide their webhooks, but since this will cost a lot in Bubble you suggest using Make and Zapier because they do it cheaper, right?