Suggestions for "merging" api data with database data

I’d like to display both API data and database data in a repeating group. My use-case is that I have live data and cached data that I want to appear together in a list.

(If my two datasets both were coming from the same local DataType, or the same API DataType, obviously I could use merged with, but that’s not the case here.)

Looking for suggestions - I’m sure there is no simple solution, but I’m sure pretty sure that someone has already come up with a fairly elegant strategy for handling this situation.

The data shown in a RG cell doesn’t all have to come directly from the RG’s data source.

Extending this point, the RG data source should be a list which contains enough rows to match both your datasets, and it should provide a way to link the two datasets for each cell.

In general, its made trickier by not having a function to find the position of a matching item.

Some examples:

Data source: List of numbers 1 to 100 (or maximum of each dataset’s count), then elements and workflows in each cell can use the cell index to reference other lists: datasetA :item # cellnumber 's fieldX . This relies on both datasets having items related by position.

Data source: List of identifiers, each cell can then contain a search on the dataset for matching identifier.

Data source: List of JSON texts, if using Javascript to do matching/sorting/etc.

1 Like

thanks for that! hmmm… i’ll sleep on it and see what i can i do tomorrow.

any suggestions if i have to sort by date?

I am not being helpful here, BUT: @Bubble folks. Please just fix the API Connector (or more specifically, just FINISH this aspect of it). That arrays can come from the API Connector and the LIST is stored and accessible, but the individual objects within the list have no type is a HUGE problem.

We would have workarounds if those types were simply created and were thus were then searchable/destroyable/creatable. It’s like someone took a coffee break and didn’t remember to finish what they had started vis-a-vis lists.

For excruciating detail on this problem, please refer to any post I’ve made referencing the “API ghetto”.

@keith can you share an example of a response that the API Connector is not handling well

Hi David,

Anything that returns an array of anything (or a subarray pretty much) is very problematic, for no good reason (except for Bubble’s allergic reaction to arrays I guess).

I need to make a video for you I suppose.

this is so stoopid, i don’t get how this bubble thing got where it is with such a bad implementation of how this API stuff works, i mean, then i can basically use any other CMS like webflow. The API calls are the only things that make bubble unique.

yeah basically everything, i mean what API can you use that always has the same positions for the data? What API can you use that does not work with arrays? And you can’t even call arrays in the API itself, i would say 90% of the APIs are unusable because of this and i don#t even know jack shit about this stuff so don#t take my word lol

The examples were for workarounds to use a RG differently than usual, you don’t have to try them.

Expanding on this point: every plugin is a workaround for some Bubble limitation. Even using Bubble is a workaround to avoid writing code and doing the hard things.

Instead of rubbishing people’s ideas, you could be constructive and offer solutions of your own.

3 Likes

Hi, @VanV. Let’s keep it constructive, OK? Limitations in various software packages can be frustrating. And, indeed, the API Connector has some rather odd limitations that some of us have been working around for some time. (Too long, for sure.)

However, while we may have challenges and even disagreements here from time to time, the theme of this forum is helping each other (whether that’s helping each other with our apps, or helping Bubble to improve in ways that let us accomplish our goals). I, too, am frustrated with certain limitations of the Bubble API Connector. But, let’s keep it cool.

One thing you could do: Make a video showing and explaining the challenges you’re facing with using the API Connector with array data and share that here.

It’s OK to express frustration, but let’s try to do so in a positive way. (I too get really cranky with Bubble at times, but it’s an amazing tool. Share details rather than throwing shade, please.)

@DavidS is trying to help and we can help him help us by providing detailed examples of how the API Connector fails to meet our expectations.

-K-

yeah sorry for using the word “stoopid”
am very sorry when i offended any snowflakes

expressing frustration in a positive way, lol

thanks guys

when you want details just look up my problem, oh wait, all the mod did in there was expressing his frustration: API JSON parsing/format into bubble database
but he did not used “stoopid” so i guess he is fine…

Dude, I’m that guy. I’m too am frustrated by limitations of the API Connector. And, indeed, there are certain limitations there that cannot be bypassed by ANY workaround at present. (And I’ve written reams about that here.)

What I’m saying is that what would be helpful is to document the specific challenges you’re facing. (For my part on this issue, I’ve a new example that is maybe better and more easily understandable than anything I’ve found before, but I am at the moment encouraging you to chill a bit and watching Game 3 of the World Series, so I can’t ATM create one of my amusing videos.)

Please note: I often use foul language (in fact, I pretty much always do) and it’s not your use of “stoopid” that I’m correcting here. I’m trying to enlist your help in explaining to the Bubble team what’s problematic about the API Connector. I feel like I’ve communicated about that a huge amount and it’s not yet been understood, so it can only be helpful to add other information to that.

I am also critical of Bubble when I need to be and make no apology for that. But I eventually share details that are sometimes helpful. (This is how you express frustration in a positive way. That’s not an oxymoron.)

We actually need more engaged and passionate users here who want to see the platform improve (and not have this forum be just an endless stream of brain-damaging “how do I…?” questions) and I’m trying to encourage you to engage in that spirit. (Note that there are other mods who may choose to edit you. I’m NOT that guy.)

1 Like

well i get where you coming from but i really don’t know how to be constructive when bubble seems to ignore how APIs work. I have no clue how i can be positive about this, nor do i have any clue how to work around this horrible flaw, there is no way i can properly work with an API here, there is no way i can properly select an array or anything in an API really, all i get is stupid lists of meaningless data scrambled together that make zero sense.

Saying that bubble does not understand this problem, is imo an offence to their intelligence. They can’t possible build a whole app around the feature of using APIs and managing data and then forget half of how APIs work, sorry i don’t buy that.

I can’t share any details that are helpful, i can only link again to my thread where i explain my problem in detail, where you commented on. Am stuck and tried to figure out how to work with this bubble thing for the alst 2 days. I just thought am stupid and that it would be easy in the end to sort the data of an API, but it is not and that is a major flaw, relly no clue how i should be positive about this, nor did i find ANY helpful data on this in the forum. All i find is people complaining about this very nature, telling that you need to make it a text data and then send it to google sheets or charts but there are only payed plugins for this. Can’t be possible that i need to buy a third party plugin for a third party service just to be able to use the API data in a meaningful way. When you know how to, pls tell me, but it seems you all don’t know how to handle API data gracefully. Istopnowwiththewalloftextbye

You can create a backend workflow to save the api data to the database where your local information is stored, and then display that local database in the repeating group. I have got this to work, but my issue is the data im importing is much larger than the data stored in my local database, so the saving of all the api data into the database creates a longer loading time.