[Updated Plugin] ChatGPT with Real-Time Streaming (No more timeouts!)

Showcase
49

A Short-Term Roadmap

Here are the improvements coming over the next few days:

  1. :female_detective: Run without 3rd-party server - connect directly to OpenAI, so you don’t have to worry about data privacy issues. This has been built already, and I will release it tomorrow (May 8) after more thorough testing.
  2. :bar_chart: Health Check - easily check if service is connected, so that you can use a fallback option, like …
  3. :hammer_and_wrench: Non-streaming response Fallback - In case streaming is unavailable, fallback to a standard wait-for-response API call. The goal for the plugin is to never have this event trigger, but also to never have a request not complete because of some connectivity issue.
  4. :desktop_computer: Custom OpenAI endpoints - Support for OpenAI on Azure and other APIs exposing the ChatGPT models. This will be experimental.

Other features in the works:

  1. :eye: Code Syntax Highlighting - Auto-detect and format code snippets, as in the official interface
  2. :robot: System Message Improvements - Checkbox to always send most recent system message, so you don’t always need to set it.
  3. :bookmark: Better Message History Handling - Send more complete message history context, with fewer tokens, using conversation summarization from LangChain.

Let me know what else you’d like to see in this list!

4 Likes