[UPDATED PLUGIN] - ChatGPT/LLM Toolkit - Assistant Streaming, 100+ models, custom Endpoints (eg Azure), and more!

Hi all,

If you’re looking for an all-in-one package to build LLM apps on Bubble, check out ChatGPT/LLM Toolkit. We’ve recently added tons of new features and improvements, and have just added support for 100+ models, so you can use models from OpenAI, Anthropic, Mistral, Meta, Perplexity, etc.

The plugin also includes streaming, features for uploading/parsing files, searching the web, embedding text, vector search, and more!

Here’s the plugin page: ChatGPT/LLM Toolkit - w/ streaming Plugin | Bubble

Let me know if you have any questions, or come join us on discord: Low-Code AI

Note - this plugin was formerly listed under another name, and discussed in another thread that’s been closed - [Updated Plugin] ChatGPT with Real-Time Streaming (No more timeouts!)

Just added streaming Speech-To-Text / Transcription via Whisper.

Experimental release, to help build voice-driven apps.

Hey all,

Just added support for Custom Endpoints, so you can bring your own Azure deployments and similar.

Setup instructions in Discord!

Hey all,

Just added support for JSON mode. If you need data back as structured JSON, just set this field to “yes”, and make sure to mention it in either your system message or user message.

Cheers

Screenshot from 2024-03-22 15-42-14

As a Bubble beginner, I’m kind of struggling to successfully get my Assistant API with the attached file working on Bubble. I’ve watched a couple of your YouTube videos and it might be better to buy this so I save up space etc.

For context, I want to build a Calories In Calories Out calculator (as a starting point) so for example…

I say I’ve had 500g of beef today and the AI will refer to the file (google sheet with the beef nutritional info) and then use that file to say how many calories are in 500g of beef.

Hey @luke.koletsios03 ,

This should definitely be doable with the Assistants API. Are you able to tell whether the file is being read at all? What specifically is giving you issues?

For more detailed help, feel free to send me a DM, or come over to our Discord server; we do lots of discussion over there!

Cheers,
Korey

1 Like

Hey all,

Just added support for a Custom Body. This gives you full control over the body of the Chat Completion request, just using standard JSON. Allows for more flexibility for anything missing from the plugin, and works in conjunction with Custom Endpoints, so that you can call arbitrary services (will continue to improve this as well).

Screenshot from 2024-03-26 11-56-43

Hey all,

Just released an update to support Assistant Streaming :robot: :partying_face:

v6.20.0 - Assistant Streaming

Support for streaming from the Assistants API. Added a new element, an “Assistants Container”, that will handle Assistant activity.

Two actions support streaming currently -

  • Create Thread and Run
  • Create Run

(there is one other endpoint that OpenAI supports streaming for - “Submit Tool Outputs”. This will be added in a coming update)

The Assistants Container also includes several states and event you can read from and hook into:

  • Current Thread Message
  • List of Thread Messages
  • Token Usage (input, output, total)
  • Run State (pending, completed, etc.)
  • Run Completed (event) - Fires when a run finishes

Lots more improvements to come, esp. actions to make saving/loading threads easier

Demo page is here if you want to take a look: Assistant Streaming

And here is a brief video demo:

If you try it out, let me know how it goes!