Backend Workflow has stopped writing to DB

[kevin.hunt]

Hi Bubblers,
I am hoping that one of you help me figure out what is causing my issue.
Up until a day or two ago, I had the Google Drive API happily feeding a backend workflow that writes the drive records to the database. I was developing the backend recursive workflow to iterate through the pages (using nextPageTokens) and either I did something that made it stop working or something in Bubble has changed (i have seen a couple of posts in the last 48 hours that make me wonder if it oculd be the latter.

I have stopped trying to build the nextPageToken process for the moment and built a simple front end call with just one field to keep things simple for debugging. I also added a step just before the list is fed ot the back end in which I capture the raw body data. What I found surprizing is that the raw body data looks very different from what is getting fed into the backend.

Raw Body

Backend Workflow List

I am using v3 of the Google Drive API which allows me to specify which fields I pull in from the API. This is useful as it allows me to build a “flat” Json file (that Bubble can handle. The backend workflow seems to be fed or has a map to all of these. At one point I was able to dynamically pick the fields and it worked but I might have changed something to make that problematic, Can anyone shed some light on this for me.

You cannot see the whole window for the fields parameter values that I used so I am including them here.

fields : nextPageToken, incompleteSearch, “files(id,kind,mimeType,starred,description,trashed,explicitlyTrashed,name,trashedTime,viewedByMe,thumbnailVersion,parents,iconLink,spaces,viewedByMeTime,version,hasThumbnail,webViewLink,createdTime,modifiedTime,sharedWithMeTime,teamDriveId,folderColorRgb,driveId,shared,modifiedByMeTime,modifiedByMe,fullFileExtension,originalFilename,fileExtension,quotaBytesUsed,md5Checksum,contentHints,headRevisionId,size)”

As one of the changes I made after I had it working was to add nextPageToken and incompleteSearch. So I tried reinitializing with just the fields : files(id, description…)
but it did not make any difference.

Here are some more screen shots to help get a better picture of what I am doing in the cut down debug page and set of workflows. Thanks in advance for your help with this.

Please pick a subcategory to help organize the information. Thanks!

Also, the more context you provide with your questions (screenshots, link to an open app, etc.), the more likely you’ll get good help quickly

[boston85719]

@kevin.hunt

What I found surprizing is that the raw body data looks very different from what is getting fed into the backend.

This looks like the Raw Body and the Backend Workflow are exactly the same…are you saying that what is different is what gets into your Database? If so, provide a screen shot of that so others can see the difference.

@kevin.hunt

Up until a day or two ago, I had the Google Drive API happily feeding a backend workflow that writes the drive records to the database. I was developing the backend recursive workflow to iterate through the pages (using nextPageTokens) and either I did something that made it stop working or something in Bubble has changed (i have seen a couple of posts in the last 48 hours that make me wonder if it oculd be the latter.

Try to put it back how you had it, without the pagination and see if it still works as it had, and if it does, then Bubble didn’t change anything (as much as I blame Bubble bugs for my issues, I do not find it likely that this is caused by a change by Bubble).

@kevin.hunt

As one of the changes I made after I had it working was to add nextPageToken and incompleteSearch. So I tried reinitializing with just the fields : files(id, description…)
but it did not make any difference.

Create a duplicate of the API call. Remove the nextPageToken and incompleteSearch and get the call initialized as you had it working previously. Then you can maintain a working version without pagination and then have a copy of the call where you try to add in pagination.

I do not believe the documentation of the API provider indicates that you should be requesting the fields of nextPageToken…those are not fields on the ‘data type’…think about what you likely know about Bubble database to try and apply it to other database systems.

In Bubble you have a data type, say ‘files’ and you have data fields like ‘name’ or ‘size’ that may indicate some properties of the file. Then in Bubble when you want to show a list of ‘files’ you would display that via a repeating group. If you want to do pagination through the repeating group, you have to limit the number of files to display and then use actions like go to next page. Nowhere in your Bubble database would there be a field on the datatype of ‘files’ be a ‘next page’ field to indicate for a specific user session the next page for that user.

In the API, the nextPageToken is returned automatically if the number of ‘files’ returned exceeds your PageSize, so you can not restrict if it is sent or not, nor do you need to request it…if you have not poured over the API documentation of the provider regarding how pagination works in their API setup, I recommend you do…likely what you have to do is know that the nextPageToken that gets returned on a single call is needed to be added to the nextPageToken parameter of a subsequent call.

So on the parameter of nextPageToken check the box of optional because it should be since your first call will not have that value since it is the first page of the pagination. Then to initialize with the value and see how pagination works, you should copy the nextPageToken from the first call and add it to your nextPageToken parameter and initialize again.

Also play around with changing the pageSize value to get a better grip on pagination.

[tylerboodman]

Also all your files (up to 1000 at a time) can be instantly created using the Data API Bulk Create endpoint, instead of doing the “Schedule on a list”

[kevin.hunt]

Hey @boston85719, thanks for the feedback. I think you must be right. I thought I had removed the nextPageToken field but I may not have properly reinitialized. I cannot remember why I added nextPageToken as a parameter - trying to follow the Google API documentation is a nightmare so must have got my wires crossed there. My first attempt at iterating on the backend in Bubble in a long while went in the completely wrong direction so I have mucked around too much. Will try your advice! Cheers KH

[kevin.hunt]

Hi @tylerboodman, thanks for the tip. I will read up on the Data API Bulk Create endpoint and try it out. Cheers, KH

[kevin.hunt]

Hi @boston85719 and @tylerboodman ,

I just realized that I sent the wrong screenshot. The Raw data Versus what is going into the Backend workflow is this.

Does that look normal?

[boston85719]

The logs are not making the issue at this stage in my opinion. It is the improper API call with use the nextPageToken as part of the fields to request (I could be wrong), but in my experience with Google API pagination and other API pagination you would not be adding the pagination values into the fields to request parameter.

[kevin.hunt]

Hi @boston85719 @tylerboodman,
Thanks again for your suggestions last night. Tyler I am just about to start work on building the same process using the bulk load approach that you recommended and am desparate that it will work - because I am still having the same problem as last night despite a lot of effort to prove that the error is mine.
Per Boston85719’s recommendation, I built a new much simpler version of the Google Drive API call (using same authentication as I doubt that could be the problem). This one just asks for file Id and does so with the parameters
fields : files(id)
pageSize : 3
mimeType : mimeType contains ‘video/’
It initializes perfectly. I also created a new version of the datatype and I created a new version of the backend workflow and … I am still getting the same error.

From the same api call that fills a RG perfectly, I get

Raw Data captured =

{“files”:[{“id”:“1xQ-qXxCN7amU208adekCAfvFA6ZyHUKH”},{“id”:“1of_FJqNNXaOqFtALsDFHR0nAlf6BVrzm”},{“id”:“1c9Guwm6b6CvPGCNppi_AnH-f3f-4Jdtn”}]}

What I see being fed into the Backend workflow is :

Run a list on : [{"_class":“StaticObject”,“data”:{"_api_c2_id":“1gMNNoMhC7w5R4OYvBnEhuJMSp4LMgV4G”,“body_raw_text”:"","_id":“01d81853a5643bdf7900532c62976a4e”},“value_id”:“api.apiconnector2.bTHlz.bTImB.files”},{"_class":“StaticObject”,“data”:{"_api_c2_id":“1bSbNzs7siNqI5JkEpei3n7r1kX1YEpki”,“body_raw_text”:"","_id":“2e7bf3bc9c3bb1112c92da2c09208570”},“value_id”:“api.apiconnector2.bTHlz.bTImB.files”},{"_class":“StaticObject”,“data”:{"_api_c2_id":“1Asp_tcXNGLNgwkNZMqJiLGiiafyKE6HM”,“body_raw_text”:"","_id":“87b3718d6795b0a78e17cb950cd902e1”},“value_id”:“api.apiconnector2.bTHlz.bTImB.files”}]

Do either of you have any suggestions on what I could be doing wrong? I have logged a bug but am still open to being told that I have done something stupid.

[boston85719]

The values you see on the Run a List on is Bubble ‘code’ I believe…you see the same type of stuff when you do list operations on backend workflows using entries from the database as well.

Try to run the workflow on a page instead of the backend workflow. Have it run for a single item to see if it works. If it doesn’t, them something might be wrong with the workflow setup. If it works, then try to not send the ID of the file and instead send to the backend workflow the actual file. Looks like your backend workflow parameter of ID is a text which is not a file, so all your backend workflow looks to be doing is saving a text value equal to the file ID, which is not a file, but instead just simply an ID.