Data API URL only pulling 100 rows

Hello,

I’m able to pull only 100 rows from my data api url. How do I get more than 100 rows?

Thanks,
George

Hi George,

Simply loop on it (pagination).

Hi John,

I’m currently on the professional plan already. What plan is needed?

Thanks,
George

This is covered in the reference. It’s called pagination.

2 Likes

You should change this. No one understands it.

Azure literally has a switch - pagination on or off. If on then everything is automatically handled. No cursors or other nonsense.

Azure also fetches 1000 items before pagination is required - not 100. 100 is way too small a number.

Finally you can also set a limit on the number of number of records to be retrieved- e.g. stop pagination after 10,000 items, or 5,000. Etc.

You should consider making your pagination this easy.

7 Likes

I second this!

1 Like

Hi @emmanuel,

Is it possible for Bubble to implement something similar to what @scriptschool describes above? More rows

Thanks,
George

Not this second, but we’ll look into where it fits in the roadmap. I can’t give a timeline now though.

2 Likes

:unamused: Sure is a greatly limiting factor.

Thanks,
George

Hi, as follow up I am for the first time using an API endpoint/wf to ‘return data from API’.

I am getting only 50 rows now from these API calls when i know there are more on this search.

Is the ‘return data from API’ limited to 50?

Update: After checking the manual… yes, the limit is 50.

Guys - 50 is such a small number. Please, change this.

These are the keys and data types of the parameters that are returned. When the name and type of data are set, define the data you want to return. In the case of the Workflow API, we limit the size of lists to 50 entries. For a full list of data, please use the Data API that handles pagination.

1 Like

Hi Emannuel,

Almost a year later and we’ve also hit a roadblock with the pagination. Would be extreme useful to disable pagination for apps on a certain plan.

Thanks,

Mido

1 Like

Hi @emmanuel,
Is there any update regarding this? It’s a huge roadblock for some of us.

Thanks,
George

1 Like

Yes, we managed to solve this with custom code. Which API are you using? Feel free to message me.

I would love to get your custom code to be able to get more than 100 items per call. Please msg me ASAP. Thanks

1 Like

Hi,

I’m importing data from an external API and can’t find a way to get around the pagination. The data comes in pages of 500 with a next parameter that is passed into the url to collect the next 500. Would love to have a look at the code you have written if possible, this issue has been bothering me for 2 weeks and I’ve tried all sorts of solution. Thanks

(solved, see below post)

Why’d you recall? I’d like to know your solution as well. I have no issue with paying.

It takes a custom js approach and a custom api.

Let’s chat! I can set this up fairly easily

I’m on vacation atm but next week I’m back full force!

If your having the same issue with pagination, it is possible to loop through the pages on the backend using API workflows.

Simply put, you call a first backend workflow that calls the data, and then sends it through another api workflow with a count that stores that data. I will attach screenshots of the loop to help out.



Hope this helps.

2 Likes

nothing new on this issue @emmanuel ? It is very complicated to work with 1000s of data with this limitation (impossible to get our dashboards on PowerBi for example…)

Happy to code out a solution for you at a reasonable offer. This will allow you to pull in as much data as you need from the api.

Send me a private message if interested.