API response too large

[robbin]

Hi

I am getting an error API response too large when setting api as action and trying to get data in text format because it doesn’t pick up JSON. The actual dataset is a JSON with 2 arrays with one array have 149 data points and other array holding 85k data points. This data is being pulled from public endpoint- https://api.raydium.io/v2/sdk/liquidity/mainnet.json and gets updated real time.

My use case is to pull data from the public endpoint every 30 secs into bubble. I tried running a small script locally that had a file size of 127 mb for 85k data set in JSON and the api response returns this each time, then fetches new data points to save. How can this happen in bubble ?

[cmarchan]

50mb hard limit

Hard limits

[pyli]

how can i get smaller chunks of data / cut the response up to be smaller?

Pulling an API request endpoint whats just got tons of data, and i tried using a limit, but seems to ignore that completely:
https://api.servicem8.com/api_1.0/formresponse.json?limit=2

[tonare]

Hey @pyli!

You should check the API documentation where you are retrieving this data to understand how they implement data pagination and how you should adjust your request to return a smaller amount of data.

Typically, this type of information is explicitly provided in API documentation.

[pyli]

hmm, i guess i didn’t see it, but i’ll check for “pagination” thanks!