I’m currently working on a payment processing platform project for dentists, and could use some advice from experienced Bubble developers regarding handling high payload volumes.
Here’s a breakdown of the situation:
Project Overview: I’m developing a payments platform using Bubble, aimed at streamlining payment processing for dental offices.
Integration with Third-party Services: As part of our payment processing workflow, transactions are initially approved through an external API (PCI compliant payment gateway similar to Stripe or Auth.net). However, we need to integrate webhooks from this service to receive real-time settlement notifications for each transaction that is approved.
Concerns: With the potential growth of the platform and increasing transaction volumes, I’m concerned about Bubble’s ability to handle large payloads efficiently, especially when dealing with real-time webhook notifications. It won’t take long before there are thousands of transactions that will need to be settled on a nightly basis - and in a pretty close timeframe.
Specific Questions:
Has anyone worked on projects involving high transaction volumes and webhook integrations in Bubble?
What strategies or best practices would you recommend for optimizing Bubble applications to handle large payload volumes, especially with real-time webhook notifications?
Are there any known limitations or challenges with Bubble when it comes to handling high data volumes or processing intensive tasks like webhook notifications?
Request for Advice: I’m seeking insights, tips, or experiences from fellow Bubble developers who have dealt with similar challenges. Your advice will be invaluable as we strive to ensure the scalability and performance of our payments platform.
Thank you all for your time and assistance - I’m eager to learn from your experiences and expertise.
I can’t speak to large payloads within individual requests, but find that Bubble is quite “expensive” at processing inbound webhooks in high volume. Smathering of thoughts below, but I do think that if you’re processing high amounts of data, Bubble may not be the most scalable solution.
My app syncs 100-200 JSON objects every day (syncing calendar data), using a hacky webhook relay service I built. This ends up costing about 1k-1.5k Workload Units each day, which is about $0.30 to $0.45 once above the plan’s monthly limit.
If you’re running very high volume but only need simple confirmations, I would consider whether you can store that data in a traditional SQL database and use the SQL Query plugin to access the data as needed. If the transactions can be settled in bulk outside of bubble, you also may be able to construct a CSV of that data and perform bulk uploads ([New Feature] Uploading as CSV in run mode - #9 by vascolucci), though I don’t know how this would compare in terms of WU consumption.
I would generally also recommend you consider whether you need to directly store each of these records or whether it is something that could be fetched at runtime as needed, or whether you might be able to build a workaround for the “inverse case” – would you be able to assume (within bubble) that the transaction was successful after a certain amount of time unless you receive a failure notice? This would dramatically decrease the load on bubble, even if just as a holdover.
I have done something similar when creating my own API.
If the action after receiving the web hook is heavy. Consider creating a queue. That way, you can receive the data and process the heavy workflows one at a time and space them out if necessary.
Thanks for your response! Your suggestion about creating a queue makes a lot of sense, and it’s definitely something I’ll explore further. Breaking down the processing of heavy workflows into manageable chunks seems like a smart way to handle the workload more efficiently.
I’ll dive into how I can implement a queue system within my app to ensure that I can handle webhook-triggered actions without overwhelming the system.
Thanks again for your input – it’s much appreciated!
Thank you for sharing your insights and experiences.
Your insights have given me some valuable ideas to consider as we fine-tune our platform’s architecture to ensure scalability and performance. I’ll definitely take a closer look at these options and see how they align with our requirements.
Thanks again for sharing your thoughts and experiences!
