Integrate ChatGpt API with a Web Scrapper

I am building an app that utilizes Open AI API and Web Scrapper. A user can input a URL after the website is scrapped, the results should be used as input to create a prompt for ChatGpt. I would like to use the Mad Lip method for the ChatGpt prompt. The keys are the business name, product/service, address, core values, industry, and mission of the business. I am using a certain scenario for the outcome.
For example: At BUSINESS NAME, we take pride in being the industry leaders, serving the wonderful people of LOCATIONS. With years of experience and expertise under our belts, we are here to cater to all your PRODUCT/ SERVICE needs and exceed your expectations. From the moment you choose BUSINESS NAME, you’re choosing excellence. Our commitment to exceeding customer satisfaction sets us apart. At BUSINESS NAME, we believe in our CORE VALUE/ MOTTO: providing nothing but the best
What would be the best approach to build this app? Any tips or advice?

Hi there,

You can definitely do that yep. Pretty open ended question though, do you have any specific parts of it you’re looking for help with?

  1. Scrape URL using external API
  2. Insert scraped text in prompt
  3. Do what you want with response

If you only want to return relevant info from the URL, you need to vectorise it → upsert it to vector database → query it → insert only relevant sections in prompt.

Hey Oliver! Thank you for you reply
What I have accomplished so far is optimizing the web scrapper to get the information back from a URL and I need to inject it into the prompt of ChatGPT API. Instead of having all the key values, I tweeted the prompt to only have key and I created a backend workflow that turns the text into JSON to be used in the prompt so I am passing the raw text into the backend plug-in and I want to create a new thing in the database but the field data type is not allowing me to choose the plug in data type I created, I actually don’t see it as an option.

Thanks for the reply!
That’s the plan but I am stuck in step 2 since I am receiving a raw data from the web scraper and I am getting a 400 HTTP error when hitting the ChatGPT api. My plan is to transfer the response to JSON by hitting a back-end workflow I created.

What datatype is your chatgpt output field set to?

That means your ChatGPT API call isn’t set up right, so send the API connector screenshot and the action screenshot if you’d like help.

Its weird because its set to chatpt_output which is the plugin type but I can’t find it as a plugin type, I bet its a Bubble glitch because If I deleted it and re set, it won’t take the chatpt_output datatype

Im confused as to the way you’re approaching this, I would just be passing the JSON through a text field and then using that for your api connector, what is this other field type?

[UPDATE] I was able to set the filed to the correct type, I just had to reinitializ the plugin call.
What would be the best approach if Chptgpt API is expecting a JSON type of content?

JSON is just structured text, so just use a text field to store your JSON :slight_smile:

Well, I am injecting the response from the web scrapper to be analyzed by ChatGPT, since the response comes back as a text field, ChatGPT is getting back with an error 400 bad response.

then its a formatting error on your request, copy and paste the api call you’re making so we can see that element

I can’t share the scenario due to NDA; Sorry, but I figured out why its a bad request, the key that is being used in the ChatGPt prompt is wrapped in quotation marks. I just have to remove the quotation marks before injecting it into the prompt but don’t know how

Hahahha sure thing. Just use find and replace to format any specific characters you’d like

I appreciate it Oliver! I actually was able to get the prompt working, I realized that content key was set to “” so removing the quotation marks resolved the issue
Thank you for your replies and help!!