Forum Academy Marketplace Showcase Pricing Features

[New plugin] Metascraper - Extract metadata from any link

Showcase
#1

Plugin link : metascraper - links metadata Plugin | Bubble
Demo link : https://metascraper-demo.bubbleapps.io/version-test

This plugin allows you to extract metadata from any site on demand.

The metadata of a link is for example the title, description and image that are displayed when posting its URL on social networks.

Usage

This plugin adds an API Call (Metascraper) type used to represent the extracted metadata.

Metadata extraction is done through the action Plugins > Metascraper - Extract a list of metadata, which takes as parameter :

  • URLs to extract: the list of URLs to be analyzed.
  • Metadata Type: the type API Call (Metascraper) provided by the plugin

Types of metadata

In this first version, the extracted metadata are the following parameters:

  • Author
  • Date
  • Description
  • Image
  • Logo (clearbit)
  • Publisher
  • Title
  • URL

This plugin uses the https://metascraper.js.org library and tries to extract these parameters by taking a maximum of standards into account (Open Graph, Twitter, …) for greater reliability.

6 Likes
#2

Ugh, could I have some help getting this to work? I have been wasting hours trying. I even emailed Bubble support - who said I needed to provide the plugin with a LIST of urls (evidently just a single line of text won’t do). So I did… and now it gives me a pop-up error that says invalid url. However I checked… and the url is fine. I checked multiple ways - checked the database to be sure… and even had it display the very field after creating it JUST before I have metascraper go. I am seeing a list of text, with one url in it, and the url looks fine.

I even tried the same url on your demo page and it worked fine. So I don’t know if maybe the most recent version of the Bubble editor isn’t compatible with the plugin now, or what. It’s driving me crazy. I just need some scraping

Thank you for whatever help you can give

#3

Hey @CramerTech ! Sorry to hear you have problem with the plugin !

Could you provide me a project reproducing the error (with, at least, an “everyone can view” setting) ? We can also plan a call if it is simpler for you :slight_smile:

#5

For others who experience issues scraping a webpage, be sure to include https:// before the rest of the URL. This missing part was causing errors to be thrown for me today.

1 Like
#6

Thanks for the bug report !
I’ve pushed a new version, adding automatically the https:// if missing :slight_smile:

2 Likes
#7

Having an issue with the plugin.
Not sure why it’s not letting me select the domain domain from the parent group.

image
image1919Ă—1035 48.4 KB

How do I also present the scraped data?
Like how do I show the scraped description?

@florian.briand

#8

Hi @kristallis !

The plugin is expecting a “list of texts” as URLs to extract input parameter.
In your screenshot, Domain seems to be a single text or a table’s row, and not a “list of texts”.

You probably have access to the :converted to list operator on ShopsBeta or Domain object, which allows you to “cast” a single item as a “list”.
You can also go through a custom state of type “list of texts”.

I could add an option in the plugin to allow the use of a single URL input, but I’m overbooked right now, so not before 1 or 2 weeks.

The results are, then, available in the following workflow steps, as result of step N.

3 Likes
#9

Hi @florian.briand , thanks a lot for sharing!

Do you have any screenshots where you could help us on how to figure this out?

best

lobo

#10

Wow Florian! This is a great tool you’ve made. But,

the loading time for the scraper is way too high. It would be very awkward for the users.

Can you please optimise it and speed it up if that’s possible? Any tips?

I’m building an app, and this would be really helpful. I’m even ready to pay $10/mo if you can make it fast.

Let me know Florian!

#11

Is there any change this can be used to scrape YouTube’s metadata? I’m looking to scrape

  • Video Title
  • Video Description
  • Video Thumbnail
  • Video Duration
1 Like
#12

Any reason why it stopped working today?

#13

I’ve been having the sam issue the past 2 days @florian.briand

#14

Shame this is broken now…

#15

Hi @florian.briand we’ve been using the plugin for a while now but we are experiencing an issue with IG posts.
When we try to scrape content from an IG post using its link, for example Coca-Cola on Instagram: "Music has the power to move us, but @rosalia.vt’s new hit is on a whole new level. Come celebrate transformation with the new Coca-Cola Move."
I get this

Screenshot 2023-05-21 at 20.54.33
Screenshot 2023-05-21 at 20.54.33460Ă—855 65.6 KB

but if this link can be added as
https://www.instagram.com/p/CpxuUe1h86g/embed?igshid=NTc4MTIwNjQ2YQ==

so I could get the Ig post shown correctly (as in the screenshot below)

Screenshot 2023-05-22 at 11.59.06
Screenshot 2023-05-22 at 11.59.06385Ă—669 63.6 KB

This is the link to my site
if you want to have a look. I’d appreciate if you could suggest how we can fix it.

Many thanks,
Laura