Best way to return 200+ items through the data API?

In my app, I have a datatype called “fruits”

I want an external website to fetch on a daily basis a list of fruits. This list would always be 200+ items.

At first, I built a backend API workflow, using “return data from API”, and “do a search for”, but I’ve noticed that the response is limited to 50 items.

I’ve then explored the data API with pagination, but since my datatype “fruits” needs to be fully public in my app, I’ve noticed anyone can call my endpoint and get the data, which is fine, but I am concerned about attacks ? How can I prevent abusers from calling this endpoint 1000x per day, making my WUs costs skyrocket ?

What is the best approach to make this work ?

Thank you for your help :folded_hands:

API Tokens

But I have noticed that anyone can call my data API endpoint, Even without an authorization header.

Yes, without an authorization header, anybody would be able to access. If you use an authorization header, only calls with correct authorization headers would have access.

Not true

Data API is accessible to anyone and they can access all data their privacy rules permit.

@thibautranger can’t you use return data from a backend workflow, but instead of returning the actual Bubble things, return JSON and format the things appropriately? I’d be surprised if there’s a limit there…

The cheapest and safest method is to handle your request through middleware like Cloudflare workers. Firstly you can add bearer tokens for auth from the external site. Secondly any calls to a Cloudflare worker can be routed through a CF domain proxy that has DDOS protection. On top of that you can set up rate limits.

It’s an extra step but if you are concerned about security then Cloudflare is your best bet. If your traffic isn’t high the free Worker plan will more than suffice. You’ll need to pony up for a CF managed domain but you can buy some cheap domain just for this.

1 Like

Would you mind explaining in a bit more details the solution you’re imagining ?

I am currently doing this, returning data from API and setting content-type as structured JSON.

With this setup, list is limited to 50.

Set the content-type to custom and use application/json

So authorization headers or api tokens in bubble api do nothing?

In the data API, it affects what data you can access, not whether you can call the endpoint.

In the workflow API/backend workflows, it does the same if ignore privacy rules is unchecked. If the barack end workflow requires authentication, it means the user must be logged in, or an admin API token is required.

Okay, so the Admin API Token is needed in the Authorization Header I believe based on my testing and what I am seeing, but I may just not understand the security aspect very well.

If I have an API call set to take data from my apps database, in the settings I have each data type in my database available to be checked on or off basically, so as to make them accessible or not. When I try to initialize the API call with the data type unchecked, so as to not be available, whether or not I send in an Authorization Header with a Admin API token, I get the following error from Bubble.

But if I do check the box for that data type, so as to make it accessible, and I try to initialize my api call without an authorization header present, I get the following successful initialization.

But if I do check the box for that data type, so as to make it accessible, and I try to initialize my api call with an authorization header present, but not a valid Admin API Token, I get the following error message from Bubble.

In both situations above with the data type api endpoint enabled, I did not have Privacy Rules on the data type. So once I add Privacy Rules that allow if the User Type is Admin to find in searches but everybody else can not, then I get the following.

Without Authorization Header Present - Successful call with no response of data fields values

With authorization Header Present but wrong Admin Token - Failed Call

With authorization header present and correct Admin Token - successful call

So what does this all mean?

This is true that only calls with correct authorization header would have access so long as there are privacy rules that do not allow ‘everybody else’ to find in searches. But it is not true that without an Authorization header present anybody would be able to access ONLY if there are Privacy Rules that restrict access to ‘everybody else’.

This is true ONLY if there are no Admin API Tokens. But if the developer sets up Admin Tokens and the Authorization Header is not present, it doesn’t matter what Privacy Rules are in place, without an correct Admin Token in the Authorization header, the data is only available to those whose privacy rules permit and whose API Calls had a valid Authorization Header present.

This is not true. The Authorization Header and correct API Token affect only whether or not you can access the data via the api endpoint, and it is the Privacy Rules that affect what data you can access.

This is similar to why I suggest for the new feature of Turn off File Upload API endpoint, that a better, more robust, actual solution is to set the File Upload API endpoint to have an authorization header that uses an Admin Token. Because the way that is setup now, is that it is simply a checkbox, of turn on or off, which is JUST LIKE the checkbox to turn on or off access to a specific data types api endpoint. When it is not checked, it is not accessible to anybody whether there is a valid authorization header with valid API Token or not. And once it is checked to be accessible, we then need to have a valid authorization header with valid API Token.

I don’t know, I might just not fully understand it well enough to know if there are security implications or not.