Hey fam, this is my first published plugin, super simple but useful for counting the token count of a inputted text.
Link: Tokenizer Plugin | Bubble
What does the Tokenizer do?
It accurately calculates the token count of a given text input using GPT-3 and GPT-4’s byte pair encoding (BPE) method. With seamless integration into your Bubble.io app’s workflows, this plugin helps you better understand text complexity and manage API usage when working with GPT-3 and GPT-4 powered applications.
Built using this package suggested by OpenAI - gpt-3-encoder - npm
Example of use:
- Inform users/you workflows when the text input exceeds a certain token count, prompting them/our workflows to shorten the text or break it into smaller chunks.
- Estimate the size of an API call before sending it, helping you manage API usage and avoid unexpected costs, and avoiding API errors.
How to use:
- Install plugin
- In workflows find ‘Get token count’.
- The input is for your text
- The output comes out in following workflow step using the expression “Results of step ‘#’ (Get token count)'s token count”
Notes:
Putting more than 10k word into it will potentially timeout the plugin, so test really large docs before putting in production and have measures in place if it fails. But for most use cases 10,000 words is enough!
The plugin page hasn’t provided a link yet, so just search for it.
Hope someone find its useful, but I built it for me so I’m happy regardless