Advice on Building a Web App Using API to Fetch Car Dealer Data and Update Every 24 Hours

petarmx5 · 2024-11-10T17:11:57.047Z

Hi everyone,

I’m building a web app that pulls car listings from a car dealer API. Users input a dealer’s business name, and the app fetches all vehicles they have for sale starting from that day. The API updates every 24 hours to check for sold cars or newly added listings. The initial data retrieval and display are working fine, but I’m facing an issue with how to handle updates and compare new data. to avoid creating duplicate records in the database.

Here’s the situation:

Each day, the API fetches a list of cars from a dealer and I need to check if each car is already in the database.
If a car is no longer available or has been sold, I need to mark it as sold or remove it from the active list.
If new cars are added, I need to insert them into the database.

My question is: Should I create duplicate records for cars each time the API updates (e.g., creating a new record for every update) or should I do it differently?What’s the best way to do it efficiently in Bubble?

Some additional questions:

How do I compare the new data with existing listings (e.g., comparing by unique identifiers or other fields)?
What’s the best way to flag or remove listings that are no longer available?
Is it better to keep a historical record of cars (e.g., archived cars) or just update the active listings?
Any tips on handling this efficiently, especially when the dataset grows over time?

Looking forward to your suggestions!

mac2 · 2024-11-10T17:17:38.974Z

petarmx5:

How do I compare the new data with existing listings (e.g., comparing by unique identifiers or other fields)?

VIN makes the most sense if you can access it from the API.

petarmx5:

My question is: Should I create duplicate records for cars each time the API updates (e.g., creating a new record for every update)

No, don’t create duplicates. When cars are ingested from the endpoint, search by VIN (or a unique ID) and modify the vehicle if found or create a new car record if not.

chris.williamson1996 · 2024-11-10T17:29:49.405Z

petarmx5:

My question is: Should I create duplicate records for cars each time the API updates

Never, you’ll destroy SEO traffic this way.

The dealer website will provide a listing ID especially if using one of the big providers like anything made by cox, dealeron, CDK, dealer inspire, etc compare that listing id using do only when search = 0 if not 0 then create a new listing. After this process you’ll send it through a flow that deletes any car from your DB that exists in yours but not theirs.

Bare in mind if you’re going for mass adoption most of these car websites and inventory management tools use excel sheets hosted on FTP servers, kinda a pain, but if you setup that flow there won’t be any customer you can’t help in the dealer space.

I’ve spent a lot of years in that niche both developing and marketing, if you need help send me a message.