I need some help figuring out how to update a field with the difference in price from the day before.
I have a web scraper running daily on an external server and it updates daily. I have it add the scraped data, which includes price, into an “archive” data type daily.
How can I have it update another field in the data type called “price change” where it looks at the price difference from the previous day compared to todays new updated price?
I am thinking I should include the update in my backend workflow that receives the daily update of scraped data. But, I just can’t figure out the structure or actions needed to make it loop through the users paired with their specific url, filtered by date, and then look at the difference between the 1st and 2nd data entries.
I don’t think you have to even “loop” through these values to “pre-compute” the values. You can compute these values when the user loads the page.
To do this you do the steps you said above:
Have text on the page that does a search for prices with constraints to be the specific price the user is viewing. And we want it sorted by date so we can find the most 2nd most recent one minus the most recent one. [Price]...Item # 2 - [price]...:first item is basically what it should look like.
So I’m actually setting this up so I can send an email to the user whose account this price change occurred under so notify them of the price change occurring.
I don’t think I would be able to set it up this way because if the email and not being displayed on the page and need it to happen if the user logs in or not.
So you’re right, my first thought would be a backend workflow if you want to do that. (And you said this in the first message, so I just didn’t read it…Anyways yeah here’s what I’d do.)
But instead of a backend workflow I’d add a step when saving the data from your scraper. I’d add a new data in the Price field that includes the change in price.
How are you saving the data to bubble? If you’re doing that fairly easy then adding this step should be easy as well. What issues are you running into?
That actually just made me remember. I have the data coming in one set at a time per url. So, I can easily do what I was originally thinking, a “search for” url and user ID. Then sort by date. Then find the difference of the first 2 entries and add that as the “price change” field.
Appreciate the help virtually talking through this!
