Hi there What would be the easiest way (at the moment) to have a support chatbot (typical chat widget at the bottom-right corner of the page) using ChatGPT?
The thing is, most of the questions I get via a real-time chatbot (e.g. Tawk) are very similar and are answered on my documentation pages. I want ChatGPT to learn what those docs say and give answers to questions based on such learnings.
So what Iâd need is:
Is there a Bubble plugin to do this or should I build something from scratch?
How does the âlearning processâ work, so that the chatbot focus on giving answers related to my business only?
Thank you so much, George! So it seems Chatbase and Botsonic could work, Iâll give them I try! Also looking forward to seeing FlexGPT in action!
Follow-up questions:
Which of the 2 options do you prefer, and why?
What is FlexGPT added value with respect to the previous 2 options?
I can also see that Botsonic (well, I think now itâs called Writesonic) has a Bubble plugin, but I think itâs more for content generation, rather than for the chatbot integration. Having a Bubble plugin that takes care of everything inside Bubble (e.g. uploading the docs) would be great.
Re. Chatbase and Botsonic, both are basically the same. You upload data and can talk to it. Botsonic (a product of Writesonic) has more customisation in terms of the embed.
Pros of Chatbase and Botsonic (compared to template):
Simple embed and pricing structure
Fully managed (donât have to set up vector database or upload APIs for example)
Cons of Chatbase and Botsonic:
Huge markup. 12.67/month for 100k words. 100k words in GPT-3.5 costs about $0.20 ( and in GPT-4 costs about $5.00 depending on how long your prompt and completion are.
Not as customiseable - canât add additional file upload types if you need them
No direct Bubble integration to link to database
Vendor lock in (well you get this with Bubble to) but if they increase chatbot pricing a lot you can get pretty f*ked over.
Iâll give an example. Suppose you want a chatbot on your site where users can ask questions about your documentation. With Chatbase/Botsonic you upload the documentation manually and can embed the chat widget (which is pretty much isolated from the rest of your site). Thatâs about it. If you go down the template route, you can build auto-syncing with your siteâs documentation so you donât have to upload manually, you can save conversations to the database, you can change the memory categories that are used to generate an answer based on the page or a dynamic field, just as a few examples. Of course, it requires a bit more effort to set up but for users who need this or just want to keep everything on Bubble, itâs great. Setting it up (creating the API accounts you need, adding them to the template, customising text and colours) takes about an hour, which really isnât that long.
Sorry for the essay
EDIT: @zhihong0321 is correct that each message costs an extra amount based on how much memory you load into it. I incorrectly assumed that Botsonicâs word quota includes the prompt. See post below for correction. It still works out cheaper to use an API compared to Botsonic/Chatbase (not including Bubble hosting costs).
Youâre partly right. It is of course the case that token usage billed by OpenAI includes the prompt. Chatbase factors this into their billing (message credits).
But, youâre falsely claiming that 4.5k token is a normal prompt length GPT-3.5 has a limit of 4096 tokens! If youâre using 4.5k tokens to generate a prompt for GPT-4 which does have a larger context limit, you can seriously optimise that.
FlexGPT.ioâs prompt is about 100 words + any memory thatâs returned (300-500 words normally). 4.5K tokens in just a prompt is wild and Iâve never seen anything like it.
Suppose an initial message with 750 word / 1K token prompt and 75 word / 100 token answer. For GPT-3.5:
OpenAI API = $0.002 (prompt) + $0.0002 (completion) = $0.0022 for that message. For $12.67, you get 5,759 initial messages.
Botsonic = 100,000/75 = 1,334 initial messages for the $12.67 price.
In this (simplified) example case, Botsonic has a 430% markup over the direct API pricing. If you have a HUGE prompt, Botsonic could become cheaper compared to API except that you canât give it a huge prompt because you donât have that level of customisation. You canât control what memory is returned, how much of it, the target chunk size of each memory.
Thatâs just not true. Perhaps you are summarising previous messages⌠thatâs been done before, many times. LangChain does it. Thatâs not bypassing the limit.
If you really do have a way to bypass it, you can let me know and Iâll pay you a lot of money for it
Okay, just to clarify for anyone that finds this thread and is trying to work out the revolutionary method the responder thinks heâs created:
GPT-3.5 will âforgetâ anything in the prompt that is more than 4096 tokens ago. It wonât use that part of prompt in its response.
You can summarise each message / completion so that each uses less tokens. This reduces the token usage so you can fit in more high level content but youâll obviously lose specific details. E.g take âi want to buy a red car near philadelphiaâ - that could only be condensed to âbuy red car near philadelphiaâ or âbuy red carâ or âbuy car near philadelphiaâ. In any case, youâre losing information so by no means bypassing the token limit.
You can use GPT-3.5 to generate an 8000 token text (higher than its token limit). Just ask it to generate a plan for the output, then run separate requests that generate each section of the text and later combine them when displaying them/saving them. This will generate 8000 tokens of total output (well, assuming you instruct it to output roughly that length). However, GPT doesnât know what has been said in the other sections - only as much information as was provided in the plan/prompt.
tldr; you cannot make GPT-3.5 responses that reference more than 4000 or so tokens. Itâs a limitation of the model. There are workarounds, but they donât do what this person claims. GPT-4-8k has about an 8000 token limit, and GPT-4-32k has about 32,000 token limit but the same principle applies.
OK guys, Iâm starting to use Botsonic, as itâs really straight forward (the only thing I donât like is its support team response time). Iâm going to pay for a higher layer just to make sure I can cover all the required words (they charge you by word) and to be able to train the chatbot with more than 1 source of info. But so far, so good! Thanks a lot for the suggestion, @georgecollier!
What would be the easiest way (at the moment) to have a support chatbot (typical chat widget at the bottom-right corner of the page) using ChatGPT?
