Forum Academy Marketplace Showcase Pricing Features

[New plugin] Metascraper - Extract metadata from any link

Plugin link : metascraper - links metadata Plugin | Bubble
Demo link : https://metascraper-demo.bubbleapps.io/version-test

This plugin allows you to extract metadata from any site on demand.

The metadata of a link is for example the title, description and image that are displayed when posting its URL on social networks.

Usage

This plugin adds an API Call (Metascraper) type used to represent the extracted metadata.

Metadata extraction is done through the action Plugins > Metascraper - Extract a list of metadata, which takes as parameter :

  • URLs to extract: the list of URLs to be analyzed.
  • Metadata Type: the type API Call (Metascraper) provided by the plugin

Types of metadata

In this first version, the extracted metadata are the following parameters:

  • Author
  • Date
  • Description
  • Image
  • Logo (clearbit)
  • Publisher
  • Title
  • URL

This plugin uses the https://metascraper.js.org library and tries to extract these parameters by taking a maximum of standards into account (Open Graph, Twitter, …) for greater reliability.

5 Likes

Ugh, could I have some help getting this to work? I have been wasting hours trying. I even emailed Bubble support - who said I needed to provide the plugin with a LIST of urls (evidently just a single line of text won’t do). So I did… and now it gives me a pop-up error that says invalid url. However I checked… and the url is fine. I checked multiple ways - checked the database to be sure… and even had it display the very field after creating it JUST before I have metascraper go. I am seeing a list of text, with one url in it, and the url looks fine.

I even tried the same url on your demo page and it worked fine. So I don’t know if maybe the most recent version of the Bubble editor isn’t compatible with the plugin now, or what. It’s driving me crazy. I just need some scraping

Thank you for whatever help you can give

Hey @CramerTech ! Sorry to hear you have problem with the plugin !

Could you provide me a project reproducing the error (with, at least, an “everyone can view” setting) ? We can also plan a call if it is simpler for you :slight_smile:

For others who experience issues scraping a webpage, be sure to include https:// before the rest of the URL. This missing part was causing errors to be thrown for me today.

1 Like

Thanks for the bug report !
I’ve pushed a new version, adding automatically the https:// if missing :slight_smile:

1 Like

Having an issue with the plugin.
Not sure why it’s not letting me select the domain domain from the parent group.

How do I also present the scraped data?
Like how do I show the scraped description?

@florian.briand

Hi @kristallis !

The plugin is expecting a “list of texts” as URLs to extract input parameter.
In your screenshot, Domain seems to be a single text or a table’s row, and not a “list of texts”.

You probably have access to the :converted to list operator on ShopsBeta or Domain object, which allows you to “cast” a single item as a “list”.
You can also go through a custom state of type “list of texts”.

I could add an option in the plugin to allow the use of a single URL input, but I’m overbooked right now, so not before 1 or 2 weeks.

The results are, then, available in the following workflow steps, as result of step N.

2 Likes

Hi @florian.briand , thanks a lot for sharing!

Do you have any screenshots where you could help us on how to figure this out?

best

lobo

Wow Florian! This is a great tool you’ve made. But,

the loading time for the scraper is way too high. It would be very awkward for the users.

Can you please optimise it and speed it up if that’s possible? Any tips?

I’m building an app, and this would be really helpful. I’m even ready to pay $10/mo if you can make it fast.

Let me know Florian!