Working with APIs - Sports Related App

[martin.reboux]

Hi,

I am new to Bubble and try to understand some concepts before jumping into building a sports related app.

The project I have in mind would massively rely on external API data with some very various degrees of updates. I could go from a yearly update (season schedule, teams) to an almost “real-time” update (for scores, games stats, etc.) with in-between updates for players status, trades, etc.

The main question I have is if Bubble applies limitations to API calls made from an app. I am speaking of call amounts, minimum interval between calls, amount of data fetched, etc.

Then, I would like to get insight for the following:

• Can I store data from an API call within a database ?
  (eg. getting all games of a season within a “games” database)
• Can the same API call populate different databases ?
  (eg. one call gets players and teams and I need to populate one “players” collection and one “teams” collection)
• And on the other hand, can one database be populated with several API calls ? Can we add condition upon database filling ?
  (eg. got a “team” collection that gets data from 2 different APIs, first to get team names, second to add team logo upon matching team name)
• Is there a way to handle the data coming from the API call before using it in the app ?
  (eg. adding capital letter, trimming strings, changing date format, etc.)

Finally, can the calls be scheduled natively in Bubble (each minute, each day at specified time) ?

Thanks

[Jici]

A) you can store data from API if you have set the call type to be “Data” and not “Action”
B) Yes. If you have lists inside the API call, there will be a type for each of them
C) Yes.
D) Not really, but you could consider just doing all of theses stuff in the front end when it’s time to display data.
E) Using Scheduling WF yes

[faisalkarimstubapp]

Hi @martin.reboux .

I am always interested in a sports related app because previously I ran a sports related tech startup.

I don’t have much experience on Bubble (1+ year) but looking at all four of your requirement from API calls and updating Bubble database, it is possible to be done Bubble. From 1 API call to update several databases to several API call to update 1 or more databases. You also can trim and extract data coming from API because displaying or saving to the databases.

[martin.reboux]

Thanks !
I think I just need to get my hands on and try to add data to a database to see what and how I can archive these goals (…)

What about the potential limitations ? There are always limitations on API calls either to save overall performance or get some additional money from the app owner. I assume Bubble isn’t different from other platforms but can’t find the informations anywhere…

[martin.reboux]

@Jici

Got an additional question for you regarding potential API limitations

Can I use a database as a “cache” to deal with “real-time” data server-side in order to save API calls ?

Let’s say I have 10000 users that want to get refreshed scores every minutes. If I allow then to trigger API calls client-side (using a button), I would be charged 10000 calls maybe more if users are impatients and hit the “refresh” button many times. Using a cache database would only cost my app one call per minute saving tons of API calls.

This feature would require scheduling and the ability to update a database, not just erase or refill features…

Bonus question: is there a way to create “cache database” that would be created for a specific event, updated during this event and then be deleted or transferred into a permanent database ?

[Jici]

I’m not sure what you mean here by Cache. What I think is that you shouldn’t have a refresh button. Get the Data call every min and the display will automatically update. If you use backend WF and DB, 10 000 user will always only be 1 api call /min.

You can handle your DB setting the way you want. You just need to think about how you want to handle Data, how many time to refresh it and set your process according to that

[martin.reboux]

You get the “cache” idea correctly. The idea is to get data service side, not client side to limit API calls.

Do you now if there are limitations and were I can find them ?

[Jici]

There’s no limitation from what I know except to be able to schedule API WF according to Bubble plan. You don’t need recurring WF because you can schedule it in loop in your case.

Main limitation will come from the API you call

[martin.reboux]

The API I call is open and doesn’t have any limitations. It’s actually not really an API but open endpoints on a website that gives many great and official data.

I started to play around with the API connector and managed to use data from the API in a basic repeating group. Now starts the tricky part, getting the data I need, were I need it.

As far as I understood, the API connector offers 2 ways to use fetched data, either as data or actions. I assume “data” gives access to API data directly in the app (like opening access to a remote database) but the “action” option remains unclear to me…

my questions:

• What is “action” mode used for ?
• When in “data” mode, is the called performed once, on page load ?
• How can I store data within a bubble database using an API call ? Shall I build an “admin” dashboard that would include button to trigger API calls and then populate a database ?

[Jici]

Action will be available as an action in WF. Most of the time, Action should be use with a call of type “POST, PUT, PATH, DELETE” that will create, update or delete data into the API. Using Action will NOT let you store the result directly in DB and may be more complex to manage for caching.

Data are available most of the time with a GET request (in some case a POST when you use a search for example). Data are need exactly what the title say: DATA. This is to get data and display (or store) them, not to send any data to API. This is what you need to use if you want to store data in DB because when you select DATA, bubble create “Data thing” like if you create a new thing (DB) in backend end.

Data can be called in different way. On page or in backend, this is not important. This really depend on each use case.

Yes you are better to create an admin page for the initial trigger to get data and store it in DB.

You will need to create a DB Thing that will store the data from API. You basically need to manually create one thing in it, and always update this item every min using backend WF. In this thing, you will need one thing that will be of the Data type of the API Call. So you will first need to initialize the call, create a new DB, create a new field, and in the dropdown for the type of data, select the API Data type you have created from API Connector.

In the Backend WF, you will update the item with this field using Get Data from API Connector.

[martin.reboux]

Well, I would only do GET requests since the API is only a source of data. I need that data as the foundation of my app, sending back data will never occur.

I didn’t quite get you point on how to add this data to the database. Is there a native way to "import data from an API to a database " in Bubble ? (like in Adalo for instance).
If not, can you provide a tutorials ?

I have tried to build something, in this case a backend workflow.
I have created a “Player” database and set some fields (FirstName, LastName). And then, I set this workflow.

Now, how can I either trigger this backend workflow manually or automatically ?

[Jici]

The goal is not to create a thing for each data (players) but mostly to store the data payload.
When you set an API Call as “Data”, this is also created as a “Data type”. So when you create a field, you will have the API Call available in the dropdown of “Field type”

The WF should be a Update a Thing (that you will create manually in a specific DB). And in this call, you will use the Get Call Api call to update it. At the end, schedule this same API Endpoint in 1 min later.

you only need an initial trigger that can be set on admin dashboard. You can cancel this at anytime in the Logs Scheduler if needed

Here’s few screenshot that I think will help

[martin.reboux]

I am not sure to understand. My goal us actually to put in the app database some informations coming from an API.

Here is the endpoint I use: http://data.nba.net/prod/v1/2020/players.json
It gives all the players in the NBA for the 2020 season.
You can see an example of results I get below. Please notice I only added the first player, but the structure is like that:

{
"_internal": {
    "pubDateTime": "2021-02-15 11:39:39.022 EST",
    "igorPath": "S3,1613407171068,1613407175097|router,1613407175097,1613407175101|domUpdater,1613407175214,1613407178577|feedProducer,1613407178725,1613407181808",
    "xslt": "NBA/xsl/league/roster/marty_active_players.xsl",
    "xsltForceRecompile": "true",
    "xsltInCache": "false",
    "xsltCompileTimeMillis": "113",
    "xsltTransformTimeMillis": "2785",
    "consolidatedDomKey": "prod__transform__marty_active_players__1277780735455",
    "endToEndTimeMillis": "10740"
},
"league": {
    "standard": [
        {
            "firstName": "Precious",
            "lastName": "Achiuwa",
            "temporaryDisplayName": "Achiuwa, Precious",
            "personId": "1630173",
            "teamId": "1610612748",
            "jersey": "5",
            "isActive": true,
            "pos": "F",
            "heightFeet": "6",
            "heightInches": "8",
            "heightMeters": "2.03",
            "weightPounds": "225",
            "weightKilograms": "102.1",
            "dateOfBirthUTC": "1999-09-19",
            "teamSitesOnly": {
                "playerCode": "precious_achiuwa",
                "posFull": "Forward",
                "displayAffiliation": "Memphis/Nigeria",
                "freeAgentCode": ""
            },
            "teams": [
                {
                    "teamId": "1610612748",
                    "seasonStart": "2020",
                    "seasonEnd": "2020"
                }
            ],
            "draft": {
                "teamId": "1610612748",
                "pickNum": "20",
                "roundNum": "1",
                "seasonYear": "2020"
            },
            "nbaDebutYear": "2020",
            "yearsPro": "0",
            "collegeName": "Memphis",
            "lastAffiliation": "Memphis/Nigeria",
            "country": "Nigeria"
        },

My goal is to put all this data within the database “Player” in my app to then use this data across the whole app. Some of this data is “stand-alone” such as the "“firstName” but some others are referring to another database, such as the “teamID”.

I followed you workflow, setting the API call to be data and then trying to create fields in a database that would match the field type. But sadly, I only get what is looking like the first level of data called league standards’s

When I try to use this data straight on the app, using a repetitive group, I can go deeper the the fetched data and actually select each entry (firstName, lastName, etc.)

Another stuff that I’ve tried, adding the data in the database using a button.
I’ve set the following workflow on a button

I actually put data in the database but don’t create one entry per player…I just get one entry with all the data, correctly added in each mapped field but all gathered in the same field (got all players name in the same name field, etc.

I assume it’s either the API structure or my way to handle it that is not correct but can’t figure it out.

In addition, backend WF looks great but require a paid plan. Would rather use a manual way to update data, once a day or even trigger that data to be updated using Zapier or Integromat (if possible).

[Jici]

If you do that, you will take a large amount of app capacity and it will slow down your app a lot if you refresh this often. The way I tell you is a fast and easy way to do that. You don’t need to reproduce the DB structure of the json into your own DB. You just store the JSON payload and Bubble will be able to use it like it was a thing with is own field.

If you really want to go that wya you will need to use Schedule API WF on a list a provide all fields needed in your Backend WF settings. Or store it in DB first using the way I tell you and Loop each item one by one using the same WF until all item have been processed. It will take less capacity.

If you follow my way, you only create one item in DB. And set your field to “Players league " or “PLayer League Standard”. And in this field you will do a Get Data from API’s player league (or player league standard”. Don’t forget that if you use player league standard, you need to set it to “is a list”

[martin.reboux]

So, put in a more “no-code” way, you suggest to put all the data fetched by the API call on one field of a DB to have the data within the app and then use this data without trying to organize it properly in a DB. Is that correct ?

To update it, you suggest I can either trigger scheduled WF to make it automated or do it manually using a button. Correct ?

Now let’s say I get games scores from the API and need to determine if a user as chosen the winner (bet-like app). User’s choice is stored in a DB in the app but game scores can be fetched dynamically using current date and an API call. Will it be possible to work with data data is not actually stored in the app DB ?

[martin.reboux]

To go even further with some advanced data handling…

How can I display some information from an API call that is enhanced or linked with data from another data call ?

Let’s say I want to make a player bio screen and show among other stuff his current team.

• each player has a personID and an actualteamID coming from the “Player” API call.
• each team as a teamID and teamName coming from the “Team” API call.

With the “Player” API call, I get the actualteamID parameter value of 12121212. For obvious reasons, the users don’t care about that value but want to get the real team name.

That real name is available with the second API call, the “Team” call.

How can I make the link between 2 API calls and say to Bubble "display the teamName value for the Current Page Player actualteamID" by actually looking for teamID value in the “Team” call that match the “Players” call’s actualteamID value.

[Jici]

There’s nothing different. You can access all related API information if needed.
You can either call the API or if you store data into DB, call your DB to find the correct information.

I have created a simple demo for you here: Sandboxjc | Bubble Editor
I don’t know all calls from this API so maybe it’s not the best way. But it should give you a few ideas on how you can set and work with API using both DB “caching” and Direct API Call.

[martin.reboux]

That is a great way to understand, thanks a lot !

I just have issues to get want you did there:

Here, I don’t get the “formula” used to search for the proper team name. Why the filtered function? Why the first item function ?

Here, you are asking the app to run an update to the database right ?

On this step, that is the actual updating workflow that put up to date data to the database. You chose first item because you know there is only one item in the DB ?

[Jici]

Exactly. I’ve manually created one item the DB. It’s only to store data in it. So I don’t need to create a new item each time I update data. Just need to reuse the same item everytime.

[Jici]

For the filtered function, the reason is that actually I’m loading the Teams list, not just 1 team. I didn’t find a call where I can “Get Team” and put the Current cell Team ID in the request to get this specific information. So I use the “Get Teams” that is a list. Because of the structure of the payload and API Connector, I need to load the list and use filtering.

But if you have a call like /team/[teamid] … you should use it by dynamically use the Current cell team id