Is there a web service to import large dataset that Bubble can import through API?

I have a potential customer that exports a large dataset to excel/csv, >10,000 rows, each week. The data from this “dump” needs to be managed. Current methods don’t work for several reasons, plus it needs to be managed “in the field.”

Since Bubble has a low limit on the number of rows that can be imported at a time, and breaking up the data into smaller chunks would be inefficient, error-prone, and a deal-breaker, I’m wondering if there’s an effective work-around to this problem.

Does anyone know if there’s another service that the data could be imported into, then linked/uploaded to Bubble through API, etc? What about to a SQL database and use the SQL connector? If that would work I have no idea how to set up the SQL table, or whatever it’s called.

Any suggestions?


Hi @gnelson

At the moment Bubble import limits are not ideal, very slow and limited to 4 minutes before ‘time out’. Currently, the process of using an API is also not ideal, because the field-time process is too long. For 10,000 records it might take hours. We look forward to an effective solution to this situation. I suggest to mount another server with at the limit a visual tool like ‘WinAutomation’ and support downloading and processing the file, do keep the essentials. Not an easy task unfortunately.


Thanks for your feedback. So it wouldn’t work to upload to SQL database and connect to it using the SQL connector? All records don’t need to be retrieved at once. Generally 10-50 with occasionally maybe twice that.

As far as I know you cannot import from sql into your bubble db directly. You could build a hybrid solution where most of the data resides in bubble db and refresh data resides in a sql db which is easily refreshed. The winautomation idea is probably the best plan if you need to keep everything in bubble db and can be surprisingly fast if built properly.

1 Like

Thanks again for the feedback. I’m confused though. What then is the function of the SQL Connector plugin? I had plans to utilize that in the future for an app that I already have in service.

How does WinAutomation get the data into Bubble?

The sql connector allows you to talk to a sql db. Once connected it’s sql as usual provided you can write sql code.
Winautomation is a bot. You code it to do an task and churns away happily. Ie take the 10k list and upload it in batches of 200 without making any mistakes.

1 Like

1 Like

Thanks @JohnMark for that detailed example. I’m still fascinated by the level of support members of the Bubble community are willing to provide to others.

I’ll be trying this tomorrow. Can’t wait.


@gnelson any luck with this solution? have a similar use case as what you describe so chasing down some potential solutions

Hi @eric3

Unfortunately I’ve been swamped on my day job so I haven’t had the chance to play around with it yet. I downloaded it but that’s as far as I’ve gotten.