Forum Academy Marketplace Showcase Pricing Features

Large List Aggregation (Group By) for metrics issues

Looking to see if anyone has handled a similar issue. I am attempting to aggregate a large set of data ~ 1.5 MM records but it continues to timeout. We need a way to calculate metrics on our dataset once a day or so. In a relational database I would do a group by and a count() to accomplish this. I am attempting to accomplish this task in bubble using the following workflow feature.

This is the specific piece of the workflow we use to update a record in our database to hold the result.

The group by looks like this:

I have thought about syncing the data to another data repository like Snowflake to do some of these more costly analytics and calcs but I am not sure it is required yet.

Thanks in advance.