The plugin now supports an alternative to Groq, Google Gemini plus a few additional options which are outlined below.
The tokenAuth page includes the Gemini API key as shown below.
You can now choose to have just the Groq key, or just the Gemini key or have both which enables you to switch dynamically between the two inference providers.
There will be at some point, a Hugging Face plugin released with all available tasks, hence the additional tab shown in the screenshot, but that’s not yet available.
Function calls will also be routed through Gemini if you have any setup, and so will any audio/image files if they’re uploaded using the ‘Send AI BOT message’ action.
Just documenting another update. Audio recognition has been included when using Gemini. The HTML is auto generated and the file exposed within the relevant conversation item, so you can display within a chat interface if required.
Hi @pork1977gm So many great updates here lately. I had a quick question: Since Groq has added support for Deepseek R1, do you have any plans of updating the plugin to allow for the content within the “think” tags to be removed from the message list? For a lot of use cases, it would be ideal to hide the thought process from the end user.
Thanks for the quick reply here, Paul. No, the Deepseek model functions a bit differently. It’s a reasoning model, and so it places its chain of thought in “think” tags before providing the actual response. You should be able to see what I mean by selecting that model and testing with the plugin.
You’d be better of using Gemini Flash 1. Its far better plus your context is huge. 1m tokens and it’s far cheaper than Groq.
Groq RL at 200 on dev level PAYG. Plus the reasoning does effect speed.
I tried DS and hid the -think- -/think- using a little JS and filter.
There’s an RG scrolling option also included and a couple of additional API calls that deal with TTS using FishAudio. I’ve made sure the doc page is kept up to date.
Remove think tags If you’re using a reasoning model, this option will remove any <think> tags from the response.
Scroll RG (attribute ID) If you have a repeating group configured to show the conversation history, providing an attribute ID of the repeating group here, will allow it to scroll to the bottom upon each reply from the AI BOT.
Thanks for the quick work on this, Paul! In testing the update, though, I find that the bot still includes the “think” tags and content with some frequency. Likewise, I’ve noticed that the bot replies often include several lines worth of empty space at the beginning. Have you noticed this behavior? I don’t know whether it is an issue with the model or the bot, but trimming extra whitespace from the replies would improve the UX.
I don’t suppose you have a URL you could send over to me do you? I can try it then and see what’s going on in a bit more detail.
I’ve just been running some tests on the demo page using the same model, and so far it seems to be behaving itself, but if you do have a public URL I can access, would you mind switching on the debug mode option too.