Need help with async server side plugin

[kevin4]

I’m writing a server-side plugin using a node package named Axios to load a URL into an object.

I’m using the .then/.catch structure as I saw comments in this forum about implementation issues with the context.async structure.

However, it seems as if none of the code within the .then is being executed.

I base this on my sprinkling of console.log messages through the code.

The code works as a node JS file, but when I copy that code into the plugin action, it’s not working quite the same.

The plugin code is below, and further below that are the server log messages showing the console.log messages that ARE being executed.

If anyone can shed any light, I would be extremely grateful.

=== Plugin code===
function(properties, context) {

let webpage = properties.webpage

console.log(“CustomPlugin:getLinks:Execution starting”)
console.log("CustomPlugin:getLinks:webpage: ", webpage)

const cheerio = require(“cheerio”)
const axios = require(“axios”)

let objectToReturn = {
_p_links: [
{
_p_href: “empty”,
_p_rel: “empty”,
_p_text: “empty”
}
],
_p_returnStatus: 200,
_p_returnStatusText: “OK”
}

console.log("CustomPlugin:getLinks:declared objectToReturn1: ", objectToReturn)

/*
const outputfile = “outputfile.htm”
const fs = require(“fs”)
function write2outputfile (data2write) {
fs.writeFile(outputfile, data2write, function(err) {
if (err) {
console.log("fs err: ", err)
}
}
)
}
*/

axios.get(webpage)
.then(response => {
const $ = cheerio.load(response.data)

    objectToReturn._p_returnStatus = response.status
    objectToReturn._p_returnStatusText = response.statusText

    console.log("objectToReturn._p_returnStatus: ", objectToReturn._p_returnStatus)
    console.log("objectToReturn._p_returnStatusText: ", objectToReturn._p_returnStatusText)
    console.log("objectToReturn.then1: ", objectToReturn)

    /* write2outputfile(response.data) */

    const webpagelinks = $("a")
    console.log("webpagelinks: ", webpagelinks)

    webpagelinks.each(function(){
        var link = {
            "_p_href": $(this).attr('href'),
            "_p_rel":  $(this).attr('rel'),
            "_p_text": $(this).text()
        }
        objectToReturn._p_links.push(link)
    })
    console.log("objectToReturn.then2: ", objectToReturn)
	return { "returnedObject" : objectToReturn }
}).catch(function (error) {
    if (error.response) {
        /* https://www.npmjs.com/package/axios#handling-errors */
        objectToReturn._p_returnStatus = error.response.status
        objectToReturn._p_returnStatusText = error.response.statusText
        console.log("objectToReturn.catch: ", objectToReturn)
        return { "returnedObject" : objectToReturn }
    }
})

console.log(“CustomPlugin:GetLinks:Execution stopping”)
}

=== server log messages ===
START RequestId: dad3583a-56c5-438e-914d-434dfaba10ee Version: $LATEST 2023-01-23T00:52:48.178Z dad3583a-56c5-438e-914d-434dfaba10ee INFO CustomPlugin:getLinks:Execution starting 2023-01-23T00:52:48.178Z dad3583a-56c5-438e-914d-434dfaba10ee INFO CustomPlugin:getLinks:webpage: https://organicgrowth.biz/link-building/you-do-not-want-backlinks-to-your-landing-pages/ 2023-01-23T00:52:49.920Z dad3583a-56c5-438e-914d-434dfaba10ee INFO CustomPlugin:getLinks:declared objectToReturn1: { _p_links: [ { _p_href: ‘empty’, _p_rel: ‘empty’, _p_text: ‘empty’ } ], _p_returnStatus: 200, _p_returnStatusText: ‘OK’ } 2023-01-23T00:52:50.197Z dad3583a-56c5-438e-914d-434dfaba10ee INFO CustomPlugin:GetLinks:Execution stopping END RequestId: dad3583a-56c5-438e-914d-434dfaba10ee REPORT RequestId: dad3583a-56c5-438e-914d-434dfaba10ee Duration: 2430.35 ms Billed Duration: 2431 ms Memory Size: 128 MB Max Memory Used: 84 MB Init Duration: 376.50 ms

[dorilama]

Bubble executes your code synchronously. Anything that is asynchronous needs to be used inside a single call of context.async where you return the final value using the callback.
Async code outside context.async is just lost.
You also have the sync method context.request to make requests,.
Also, my understanding is that there is no filesystem where your action is executed.

I hope this helps

Mariano

[kevin4]

Thank you.

I’ll rewrite it.

An irony is the first version of the Node JS file I wrote used an async function, which I then learned the Bubble plugin can’t accommodate.

So I rewrote it to use .then/.catch.

FWIW the fs stuff is in the original Node JS file, but is commented out in the Bubble plugin.

[dorilama]

The point is that Bubble runs your code in a synchronous way.
Any code that is asynchronous must be inside context.async or it will not be handled by bubble’s logic.
It doesn’t matter if you use async/await or then/catch.

Example with async/await:

const result = context.async(async (callback) => {
  try {
    const result = await doStuff();
    callback(null, result);
  } catch (error) {
    callback(error);
  }
});

Example with then/catch:

const result = context.async((callback) => {
  doStuff()
    .then((result) => {
      callback(null, result);
    })
    .catch((error) => {
      callback(error);
    });
});

[kevin4]

Your last comment REALLY helps.

I was misunderstanding something important here.

Thank you.

[kevin4]

I could use some more help. It’s possible I’m not understanding callbacks the way I thought.

I’m working off the belief that in your example, you showed a callback function being called and in order to do so that callback function was previously defined.

Below is the code that runs as a Node JS file.
=== Start

let webpage = "https://organicgrowth.biz/link-building/you-do-not-want-backlinks-to-your-landing-pages/"

console.log("CustomPlugin:getLinks:Execution starting")
const cheerio = require("cheerio")
const axios = require("axios")

let objectToReturn = {
    _p_links: [
        {
        _p_href: "",
        _p_rel: "",
        _p_text: ""
        }
    ],
    _p_returnStatus: null,
    _p_returnStatusText: ""
}

const processResponse = function processTheResponse (error, response) {
    if (response) {
        const $ = cheerio.load(response.data)

        objectToReturn._p_returnStatus = response.status
        objectToReturn._p_returnStatusText = response.statusText

        console.log("CustomPlugin:getLinks:objectToReturn._p_returnStatus: ", objectToReturn._p_returnStatus)
        console.log("CustomPlugin:getLinks:objectToReturn._p_returnStatusText: ", objectToReturn._p_returnStatusText)
        console.log("CustomPlugin:getLinks:objectToReturn.then: ", objectToReturn)
    }
    else if (error) {
        /* HERE>>> https://www.npmjs.com/package/axios#handling-errors */
        objectToReturn._p_returnStatus = error.response.status
        objectToReturn._p_returnStatusText = error.response.statusText
        console.log("CustomPlugin:getLinks:objectToReturn.catch: ", objectToReturn)
    }
}

const getTheLinks = async (url, callback) => {
    try {
        const response = await axios.get(url)
        callback(null, response)
    }
    catch (error) {
        if (error.response) {
            callback(error)
        }
    }
}

getTheLinks(webpage, processResponse)

console.log("CustomPlugin:getLinks:Execution stopping")

=== End

It runs in Node and produces expected results.

When I run it as is in the bubble plugin, I expect to, and see, the same synchronous timing issue I saw before.

Then… I modify the “core” function as follows:
=== Start

const getTheLinks = context.async(async(url, callback) => {
    try {
        const response = await axios.get(url)
        callback(null, response)
    }
    catch (error) {
        if (error.response){
            callback(error)
        }
    }
})

getTheLinks(webpage, response)

=== End

And after I do so, the promise times out.

The specific error messages I see are:
Plugin action testing error:
Plugin server action timed out

“testing” is the name of the plugin action.

I feel I’m close, but the precise syntax eludes me.

[dorilama]

Unformatted code is a pain to read. Have a look here to see how to properly write code in a post.
If you don’t understand how to do do it you should try first with a simpler async function. When it’s working with a simpler function then you can add your real logic.

Literally the example I posted is what you need to use, as is. Just replace const result = await doStuff(); with the code you need to run and be sure to call the calback.

@kevin4

catch (error) {
if (error.response)
callback(error)
}
})

this is not going to work, you always need to call the callback. This is probably the reason why you get that error.

[kevin4]

I too have been asked to read unformatted code before and I too have responded in similar fashion.

However, I had never before heard of the three backtick trick, so I thank you for that.

I edited my above post using the three backtick trick so the code is now better formatted.

I also did follow your advise about starting simple and getting gradually more complex.

And I THOUGHT I had it, up until what I expected would be the very last step.

It’s entirely possible where I’m falling down is in my understanding of how to implement the callbacks.

Would you mind taking one last look at it? With the better formatting of the code examples above?

[dorilama]

The misunderstanding is that context.async either returns a value when you call the callback callback(null, value) or breaks the execution of your code throwing an error when you call the callbavkcallback(error).
Also the function that you pass to context.async has only one parameter: the callback that you need to call to tell bubble that context.async has finished and can return the value.

// load here any data from lists or things

const otherResult = context.async(async (callback) => {
  try {
    const result = await axios.get('your url');
    // add here any operation, async or not
    const other = doSomething(result);
    callback(null, other);
  } catch (error) {
    // you must handle every error with a final call of the callback
    callback(error);
  }
});

const res = doYourstuff(otherResult);
// rerurns final values if you need
return {res}

I hope this helps.
Send me a message and let me know how it goes.

Cheers
Mariano

[kevin4]

I got it working, by once again stripping it down to it’s essence.

This time I’m literally going to add my logic back one little bit at a time, and I’ll test every tiny step right away.

Anyway,

Thank you for you help.

[robert16]

@kevin4 and @dorilama - thanks so much for this contribution! I have written up a server-side plugin and have been trying to figure out why I get errors. Am I understanding correctly that I cannot use async function(properties, context) and other async functions to follow, but instead should use context.async? Just making sure this applies to server-side functions as well… the error Bubble throws specifically instructs to use ‘async function’, so I feel like I’ve missed something…

Thanks again, cheers and happy Bubbling!

[dorilama]

you are missing the new v4 api announced 8 months ago. everything in this topic is deprecated.
If you see that error either there is a syntax error in your code or your first line is not like what buble wants (which is not valid syntax )

Updating to Plugin API v4

[kevin4]

Go with what dorilama says. He understands this much better than I do and was enormously helpful to me when I didn’t know how to use this stuff properly either.

[robert16]

Ahh! Ok, I’ve been through a maze of docs and missed this. Thank you!!