[UPDATED PLUGIN] - ChatGPT/LLM Toolkit - Assistant Streaming, 100+ models, custom Endpoints (eg Azure), and more!

Showcase Plugins
1

Hi all,

If you’re looking for an all-in-one package to build LLM apps on Bubble, check out ChatGPT/LLM Toolkit. We’ve recently added tons of new features and improvements, and have just added support for 100+ models, so you can use models from OpenAI, Anthropic, Mistral, Meta, Perplexity, etc.

The plugin also includes streaming, features for uploading/parsing files, searching the web, embedding text, vector search, and more!

Here’s the plugin page: ChatGPT/LLM Toolkit - w/ streaming Plugin | Bubble

Let me know if you have any questions, or come join us on discord: Low-Code AI

Note - this plugin was formerly listed under another name, and discussed in another thread that’s been closed - [Updated Plugin] ChatGPT with Real-Time Streaming (No more timeouts!)

1 Like
2

Just added streaming Speech-To-Text / Transcription via Whisper.

Experimental release, to help build voice-driven apps.

3

Hey all,

Just added support for Custom Endpoints, so you can bring your own Azure deployments and similar.

Setup instructions in Discord!

4

Hey all,

Just added support for JSON mode. If you need data back as structured JSON, just set this field to “yes”, and make sure to mention it in either your system message or user message.

Cheers

Screenshot from 2024-03-22 15-42-14

5

As a Bubble beginner, I’m kind of struggling to successfully get my Assistant API with the attached file working on Bubble. I’ve watched a couple of your YouTube videos and it might be better to buy this so I save up space etc.

For context, I want to build a Calories In Calories Out calculator (as a starting point) so for example…

I say I’ve had 500g of beef today and the AI will refer to the file (google sheet with the beef nutritional info) and then use that file to say how many calories are in 500g of beef.

6

Hey @luke.koletsios03 ,

This should definitely be doable with the Assistants API. Are you able to tell whether the file is being read at all? What specifically is giving you issues?

For more detailed help, feel free to send me a DM, or come over to our Discord server; we do lots of discussion over there!

Cheers,
Korey

1 Like
7

Hey all,

Just added support for a Custom Body. This gives you full control over the body of the Chat Completion request, just using standard JSON. Allows for more flexibility for anything missing from the plugin, and works in conjunction with Custom Endpoints, so that you can call arbitrary services (will continue to improve this as well).

Screenshot from 2024-03-26 11-56-43

8

Hey all,

Just released an update to support Assistant Streaming :robot: :partying_face:

v6.20.0 - Assistant Streaming

Support for streaming from the Assistants API. Added a new element, an “Assistants Container”, that will handle Assistant activity.

Two actions support streaming currently -

  • Create Thread and Run
  • Create Run

(there is one other endpoint that OpenAI supports streaming for - “Submit Tool Outputs”. This will be added in a coming update)

The Assistants Container also includes several states and event you can read from and hook into:

  • Current Thread Message
  • List of Thread Messages
  • Token Usage (input, output, total)
  • Run State (pending, completed, etc.)
  • Run Completed (event) - Fires when a run finishes

Lots more improvements to come, esp. actions to make saving/loading threads easier

Demo page is here if you want to take a look: Assistant Streaming

And here is a brief video demo:

If you try it out, let me know how it goes!

1 Like
9

This is great, thank-you. I love this plugin so far.

I feel like this is a stupid question but how do I send a second message on the same thread using the above functions?

Thanks

1 Like
10

Hey @ben57 ,

Glad you’re liking it!

Not a stupid question; Assistants are complicated, and things are still changing with the plugin.

You’ll want to make sure you’re on the newest version of the plugin, and currently it’s a 2-step process:

  • Run "Add Message to Container (Assistant’s Container)
  • Run “Create Message”, passing your thread ID.

The first action adds it on the front-end (your display)
The second action adds it to the API

After those 2 actions, you can then create another run.

Tutorials are currently being developed. Latest one covers file uploads with Assistants:

1 Like
11

Hey all,
Part 2 of this tutorial is now up:

  • Threads, Messages, Runs
  • Streaming UI setup
1 Like
12

Looks like it’s set to private…

1 Like
13

@localprosnet - yeah good catch lol. Flipped to public.

1 Like
14

:muscle: Looking forward to your server mini course video(s) as well. Found (coming soon) on github. Keep rolling! Your in depth tutorials have my full attention.

1 Like
16

Oh thanks for letting me know! Use this one instead:

17

Hey all,

If you’ve used LLM Toolkit in the past and have moved on, I’d love to hear from you. We’re trying to figure out what still sucks about the plugin lol.

If you have any feedback, as spicy as you like, please send me a DM!

Thanks!

18

Hey again all,

Another quick note:

Just published a super fast walkthrough of spinning up your own ChatGPT alternative in 2 minutes.

ChatGPT and the OpenAI API have been down today, which can be a massive pain if you’re using them for work. Thought it might be helpful to see how quickly you can create your own fallback system.

19

What’s the best way to use this for back end applications?

I want to use backend workflows (or even front end workflows) To send things to ChatGPT and then write the response directly to my database.

The problem I am having is that I think that the workflow is happening so quickly that it doesn’t have time to wait for the “streaming”

I tried using the non-streaming option but the JSON field collapses which makes it extremely difficult to edit it and add dynamic text.

20

Hey @ian11,

If you use the “Chat Completion (non-streaming) v2” action in a backend workflow (like the screenshot below), you can write the response to the database.

The Bubble editor is admittedly not great for long JSON strings. What I typically do is copy/paste the skeleton into an external text editor, then go back in the Bubbled editor and add the dynamic fields. It’s not ideal, but can work.

Another thing that I find really helpful is using placeholder strings in your prompt, and then running “:find and replace” on them. So for example:

{   
"model": "gpt-4o",
"messages": 
    [ 
        { "role": "user", 
          "content":  [
              { "type": "text",
                "text": "Answer the following question: --question-- "
              }
          ]
        }
    ]
}

So you could use a JSON structure like this, and run :find and replce on the --question-- string.

It makes it easier to edit it externally.

Just make sure if you’re doing this, and using line-breaks for readability in your editor, that you run “:find and replace” on newline characters (i.e., just click into the find box, press enter, and there will be an empty line at the top). I don’t think “:format as JSON-safe” will catch/fix this.

If you want to save to the database from a streaming action, I’d recommend this:

  1. Create a Conversation datatype in your database. (or call it Chat, Convo, whatever)
  2. Have a field on the datatype that is “list of texts”; call it Messages or something.
  3. Set up an event to trigger on “Message Generation Complete”, and save the “Data Container’s Message History” to the Conversation’s Messages field.

This is covered in more detail in the “Conversations” tutorial here:

Let me know if this helps!

Screenshot from 2024-08-19 08-45-56
Screenshot from 2024-08-19 08-45-561058×468 44.9 KB

21

Great! What wasn’t clear was that “choices” was a list that contained the message content.

I appreciate the response!!

1 Like