Forum Academy Marketplace Showcase Pricing Features

Building Data Intensive Apps

Hi there!

I’m trying to build a personal finance app using bubble and wondering what’s the best way to structure my application. Basically, I need a user to authenticate their bank account (Plaid) and to show them aggregations of their transactions data (not the raw data) . So here is the problem, where/how can I do complex data transformations?

My intuition says to build an external web app that exposes an API that can fetch data from Plaid and send aggregated data bubble for displaying it to the user.

What are other ways to do ETLs/Data transformations with bubble apps?

This is what I’m thinking of, but if it’s possible to do this in a more simple way, I’d love to hear it.

Thanks,
Chris

1 Like

Have you already checked out Copilots Plaid plugin?

1 Like

Yes, that get me 50% of the way there. It serves back line by line transactions from the Plaid API. From there I want to do complex data aggregations and transformations, like say calculate the money spent on Uber or Amazon over a time interval. I don’t think bubble is great for these ETL jobs, but I might be wrong. This is why in exploring building a web API that can pull/store/transform data to be served in a clean way to bubble… Thoughts?

I am not sure if you have seen this thread yet or not:

But you could potentially group data using this feature. I am not sure, whether you are currently saving the information after fetching in the Bubble DB or trying to process it directly. You might need to see how best you could implement it, but just want to highlight this capability.

Otherwise, ofcourse, you could always do all this processing somewhere else and then just fetch the processed data into Bubble.

Thanks, I’ve seen that one too. It’s more than just grouping by - the calculations include a full scope of SQL functions - multiple joins, reg_ex, partitions, etc.

We heard that Bubble is coming soon with better speed (maybe on demand?), that will solve your problem to go directly (Plaid and Bubble). Question is: when?

It’s not even really about the speed, just about the transformations required - say I get a list of raw transactions that look like this:

2018-01-01 | UBER TRANSPORTATION | 4.99
2018-01-01 | TARGET | 22.99
2018-01-01 | UBER TECH | 8.99
2018-01-02 | UBER TRANSPORTATION | 3.99

How would write a function in bubble to select only the ones that string match “uber” and sum them over a daily basis?

As your data will be downloaded to Bubble, then a Do Search with constraint Description contain UBER (in a repeating group), then sum (outside the RG) the fields with :sum

08%20AM


2 Likes

Okay, this is good to know. I hadn’t thought about it this way. Thank you John!

I always figured I would do calculations in some backend, serve the frontend a json/csv file and use a javascript graphing library to display the information. I think I would prefer to do it in this way, rather than being limited to what front-end calculations are available to bubble.

@stanleychris2 check out https://parabola.io. I was recently introduced to it and was really impressed with their data transformation functions. Seems like that’s their thing and you can import data from API endpoints (e.g. Bubble Data API or some other External API, hopefully Plaid for you) and export results to an API Endpoint (e.g. Bubble Workflow API).

I’m going to do a tutorial on using it with Bubble soon, but your post really hit the use case - might be exactly what you’re looking for.

Edit: The one thing I hope they add is real-time runs of the workflows like webhooks. At best, looks like you can schedule the flows to run every 10 min.


Gaby at Coaching No Code Apps (formerly Coaching Bubble)

Courses & Products, Tutorials, Private Coaching, and High-level Development

Start Learning Today :mortar_board:

4 Likes

Hey! Thanks for this. It look’s pretty good and would solve a lot of problems. One thing I wished they had was a way to do transformations via code or SQL.

I think I found a solution that requires a bit less programming than say a Flask app, but more powerful than parabola - take a look at this: https://www.dreamfactory.com/

4 Likes

Thanks so much for sharing! Adding it to my list of tools to explore :smile: I hope it gets you what you need!

In the end, I decided to use flex.io.

Give it a look, it solves my problem perfectly and less configuration than dreamfactory.com.

4 Likes

@romanmg - Hey Gaby,

Have you had a chance to do that Parabola tutorial?

@pflu0008

https://youtu.be/QjQfWPvRVc8

3 Likes

thanks Gaby - Parabola looks awesome and applicable to some spreadsheet style modifications that I need for some of my reporting. Thanks for posting the video, I signed up with Parabola as a result

1 Like

Hi Stanley, @stanleychris2
(please let me know if this @ was not necessary - I’m still new to forums)

I am just considering Bubble, but like you need some ability to manipulate data.
I’m getting lost down rabbit holes as a solo non-tech bootstrapper.

How did you go? Can I see your website? What were the reasons you chose flex? And are you happy with how it worked out (so far)?

Cheers,
Baz