I’m trying to build a personal finance app using bubble and wondering what’s the best way to structure my application. Basically, I need a user to authenticate their bank account (Plaid) and to show them aggregations of their transactions data (not the raw data) . So here is the problem, where/how can I do complex data transformations?
My intuition says to build an external web app that exposes an API that can fetch data from Plaid and send aggregated data bubble for displaying it to the user.
What are other ways to do ETLs/Data transformations with bubble apps?
This is what I’m thinking of, but if it’s possible to do this in a more simple way, I’d love to hear it.
Yes, that get me 50% of the way there. It serves back line by line transactions from the Plaid API. From there I want to do complex data aggregations and transformations, like say calculate the money spent on Uber or Amazon over a time interval. I don’t think bubble is great for these ETL jobs, but I might be wrong. This is why in exploring building a web API that can pull/store/transform data to be served in a clean way to bubble… Thoughts?
I am not sure if you have seen this thread yet or not:
But you could potentially group data using this feature. I am not sure, whether you are currently saving the information after fetching in the Bubble DB or trying to process it directly. You might need to see how best you could implement it, but just want to highlight this capability.
Otherwise, ofcourse, you could always do all this processing somewhere else and then just fetch the processed data into Bubble.
Thanks, I’ve seen that one too. It’s more than just grouping by - the calculations include a full scope of SQL functions - multiple joins, reg_ex, partitions, etc.
We heard that Bubble is coming soon with better speed (maybe on demand?), that will solve your problem to go directly (Plaid and Bubble). Question is: when?
As your data will be downloaded to Bubble, then a Do Search with constraint Description contain UBER (in a repeating group), then sum (outside the RG) the fields with :sum
Okay, this is good to know. I hadn’t thought about it this way. Thank you John!
I always figured I would do calculations in some backend, serve the frontend a json/csv file and use a javascript graphing library to display the information. I think I would prefer to do it in this way, rather than being limited to what front-end calculations are available to bubble.
@stanleychris2 check out https://parabola.io. I was recently introduced to it and was really impressed with their data transformation functions. Seems like that’s their thing and you can import data from API endpoints (e.g. Bubble Data API or some other External API, hopefully Plaid for you) and export results to an API Endpoint (e.g. Bubble Workflow API).
I’m going to do a tutorial on using it with Bubble soon, but your post really hit the use case - might be exactly what you’re looking for.
Edit: The one thing I hope they add is real-time runs of the workflows like webhooks. At best, looks like you can schedule the flows to run every 10 min.
Hey! Thanks for this. It look’s pretty good and would solve a lot of problems. One thing I wished they had was a way to do transformations via code or SQL.
I think I found a solution that requires a bit less programming than say a Flask app, but more powerful than parabola - take a look at this: https://www.dreamfactory.com/
thanks Gaby - Parabola looks awesome and applicable to some spreadsheet style modifications that I need for some of my reporting. Thanks for posting the video, I signed up with Parabola as a result
I’m glad to see posts addressing the challenge I’m trying to solve. I’m curious about the outcomes of your endeavors. Fast-forward to 2024, where many no-code tools are available and AI technologies simplify finding solutions. Could you share any lessons learned from your experiences? I would really appreciate it.
It will give you api out of the box, free plan, low prices for managed installation, based upon a powerful open source db that can scale called Postgres.
Not familiar with writing postgresql? Learn how to use ChatGPT and Claude.ai and before you know it you will write complex and powerful Postgres functions that will power your business
I hope it gets you what you need!