Bringing In Significant Number of Records

Hi, community! We have to show and process in “real time” information from an external app. We’re bringing this data via API, specifically from a table that will get populated from 0 records to 400K in a 9 hours timeframe. Basically, we have to do some calculations using filters and show it on our app’s dashboard.
Can someone throw us some ideas on how to accomplish this without failing big time? Thanks in advance!

Personally I’d use the data API the create the basic data as it’s far faster than workflows then adjust the data from there with backend workflows. However it depends on what your data is and what you’re trying to do with it.

@jared.gibb is a great resource to help with this on a consultant basis