Need to handle GZIP (Accept-encoding... GZIP) in API

Need help
1

Hi,

I have an API that requires the request to accept gzipped encoding.

I played around with headers (such as accept-encoding: gzip, deflate; etc.) - no luck (just really long hang times).

Any way to do this?

Thanks,
P

1 Like
2

Bumpā€¦

3

Did you get things to work in postman?

1 Like
4

Yes.

.

5

So can you share a screenshot and your settings in Bubble?.. Thatā€™s how to do it, you replicated in the API connector what you have in Postman.

6

Postman handles GZIP encoding, Bubble doesnā€™t or doesnā€™t seem to - this is what I want to know from youā€¦ Does Bubble handle GZIP encoding or not? If not, can you enable it? If not, is there a workaround?

No ability to post a screenshot just now but no point - all I get from Bubble is error message saying data is Gzipped. Can follow up later with a screenshot if you still think itā€™s important.

P

7

Do you have a link to the doc of the API?

8

The documentation is in my account (not documented outside the account), so Iā€™ll just share what it says, itā€™s extremely simple.

It reads:

"CSV or JSON from the last run

Use the following RESTful API to download data from your last successful Extractor run in CSV or JSON format. Please do not share your API key with anyone."

Thatā€™s it - you get an API address (https://data.import.io/extractor/[long code for this particular extractor]/json/latest?_apikey=[enter key] and youā€™re done. [Note: if you want a CSV download instead of JSON data you just change the ā€˜jsonā€™ in the API to ā€˜csvā€™. Iā€™d like the JSON]

Iā€™ve successfully used this in Postman and another platform (Built.io). Both Postman and Built.io have Accept-encoding: gzip enabled so it was easy to get it working. I have not been able to get it working in Bubble.

Anyway, hope this helps? Let me know if you need anything else to investigate this.

Thanks,
P

9

And can you share a link to your app with some credentials to test? Itā€™s likely a feature request, not a bug, so we canā€™t guarantee we can add it.

10

@emmanuel seems like you never got a link on this question. I have the same problem with import.io. It runs in Postman but bubble gives me error.

Here is a link:

Any suggestions?

11

I solved it by using different product: Import.ioā€™s ā€œMagicā€ which doesnā€™t seem to require gzip or at least it worked there.

12

just wanted to check if this was resolved?

I am using API calls via a plug-in in Bubble to connect to our own system. The integration broke a few days ago and I have traced it to a change in the way the API sends responses. A change was made to the API where the response JSON is gzip compressed. Now in Bubble I get an error like this:

image

What caused me a bit of confusion initially was that the first thing I did was check Postman, but it turns out that Postman transparently unzips the compressed response.

Is there anything I can do or set in Bubble so that the API calls can work with the compressed response?

13

further to my last post here are the screenshots from Postman, showing a nicely formatted JSON response and the headers of the response indicating the gzip compression.

image
image2215Ɨ848 115 KB

image
image2244Ɨ878 58 KB

14

Same issue here and with another recent poster API response in gzip format

I have an API which is encoded in gzip which Bubble does not support. Postman will show the content just fine with Content-Ending as gzip.

@emmanuel Can we request gzip support in the API Connector data types

15

If anybody, is still looking for the possible solution to bubble limitation. I will share a solution for this.

  1. Instead of creating API call in a plugin, an action inside the plugin can be created

34%20PM
34%20PM1275Ɨ694 62.5 KB

  1. Make sure that you set the Action Type as ā€œServer sideā€

  2. Add the following code inside the code section

    function(properties, context) {

   const options = {
     uri: ā€˜ā€™,
     gzip: true, // !! This is important needed for unzipping !!
     method: ā€˜GETā€™, // or POST, depending on the needs
     headers: {
         ā€¦},
     body: ā€˜ā€™ // if POST is used
   }

   response = context.request(options)

   if (response.statusCode == 200) {
     json = JSON.parse(response.body)

     return {
         your_property1: json.<fieldName1> , your_property2: json.<fieldName2>
     };
   }

   // if you want to return http error code and message as property values. For this you need to define this properties in return values section
   return {http_status_code: response.statusCode, http_error_message: response.body}
 }

  3. You include the plugin in your application. Then you can add the action anywhere with plugins->

  4. The result of the action with the all the result properties <your_property1>,<your_property2>, etc is available for the next actions with ā€œResult of step nā€.

Forgot to say that you can get and use the API call parameters from the UI when you are describing the properties in the ā€œFieldsā€ part of ā€œPlugin actionā€

P.S. I think that this approach is pretty much similar what bubble does with API Connections in plugin builder. If it is true, I wonder why bubble does not support gzipped results.

16

Hey Serg,

Thank you for sharing your solution to this gzip decompressing issue. I really hope @emmanuel and the Bubble team can quickly create a solution for this (seems like a relatively straightforward update for them to add gzip support to their api connection plugin, but I could be wrong).

I tried using your code to create a gzip solution of my own. Iā€™m still getting an error thrown however.

Would you mind taking a look to see if thereā€™s any error in my code. It looks fine to me as far as what I can tell.

function (properties, context) {
  const options = {
     uri: "https://www.parsehub.com/api/v2/projects/{PROJECT_TOKEN}/last_ready_run/data?api_key={API_KEY}&format=json",
     gzip: true,
     method: "GET",
     headers: {},
     body: ""
   };

   response = context.request(options);

   if (response.statusCode == 200) {
      json = JSON.parse(response.body);

    return { strains: json.strain };
  }
  // if you want to return http error code and message as property values. For this you need to define this properties in return values section
  return {
    http_status_code: response.statusCode,
    http_error_message: response.body
  };
}

Big thanks,
Jaime

17

Hi Jaime,

  1. I see that you donā€™t defined return values. The return value name will be strains
  2. you can remove the body: ā€œā€ line because it is not needed for GET request and can be the cause of the error. Donā€™t forget to remove the comma after headers line if you will return the body

If these steps will not solve your problem, just debug the logic with putting console.log(ā€™ā€¦ā€™)-s to see what and where things fail. You will see the log messages in the logs of the application which is using the plugin. Make sure that will use the latest version of the plugin with the application.

Important! Please remove the sceenshot from your post because it leaks some info that people should not see

18

Hey Serg,

I think I am throwing a return value (see line 15). I wrote ā€œreturn { strains: json.strain };ā€

Is that not correct? Let me know if I need to maybe move the return statement somewhere else.

Thatā€™s odd, looks like the forum didnā€™t update my photo hiding my api key. Thanks for the catch there! (donā€™t want that info leaking out :grimacing:)

For sure, great idea with using console.log to debug the logic. Itā€™s been a while evidently since Iā€™ve coded in JavaScript. Your help is much appreciated :pray:

19

What I meant was to add strains in return values section as seen in the screenshot

14

Please add http_status_code, http_error_message as well. At the moment I cannot verify whether the absence will cause an error or not, but without them you wonā€™t be able to access the values after the plugin action is finished

20

Weā€™re going to work on this next week

2 Likes