OpenAI Vision API: can't pass image from Bubble

[jeff.warren.orr]

Hey all, been stuck on this for a few hours so hoping for another set of eyes.

Goal: user uploads an image, I send that image to OpenAI Vision API, it analyzes it and sends back a response that I display to the user.

Where I’m at:

1. I successfully “initialized” the OpenAI API.
2. I can send a hard-coded image URL (i.e. I copy/paste the image URL and send it via the API) and it works fine. I get a response back from OpenAI and see it populate in the database.
3. Where something is wrong is when I pass a user uploaded image into the backend workflow connected to OpenAI API. The image is being added to my Bubble database, but I am not received anything back from OpenAI. (Attached screenshots of my workflows)

Any tips are greatly appreciated!!

[raph1]

Hi Jeff, how are you doing ?

I was struggling with the same problem yesterday and stumbled on your post.
I think I found a way to overcome it.

1. You should be using this format in the API call :

[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What is in this image?",
                },
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"},
                },
            ]

Bubble doesnt do the base64 encoding by defaut, so it has to be specified somewhere

2. Backend workflow : I recommend you do this action in the frontend to test what’s wrong in the process

Hope it helps : )

Raphael