New Pricing Disaster

ryan8 · 2024-08-26T16:15:54.967Z

yes i was aware of the change BUT not aware of how it actually would affect my cost

ryan8 · 2024-08-26T16:43:10.354Z

does anyone know if bubble dings your for publishing the APP?

ryan8 · 2024-08-26T17:18:36.148Z

thank you yes i worked on it for a long time lol PLUS the industry changes faster then i can keep up

zelus_pudding · 2024-08-27T06:24:45.452Z

From the looks of it you should rebuild a number of backend workflows as Python APIs you selfhost (on something like Render or Heroku). I do this all the time and it’s almost always far faster and cheaper to run (and develop) than trying to bake the same logic into Bubble… especially if you’re regularly updating many items in your DB. For example, I once hired a (good) Bubble dev who spent 20 hours trying to build an airtable integration that could perform special groupbys/rankings all via backend workflows… It was a mess of like 16 workflows all with 2 to 8 steps in them… and it was sloooow. I had him stop developing and coded the same thing up in Python in roughly 2 hours. It was maybe 20x faster than what he’d made and cost basically nothing (like $10 a month but more like $0 since I already had the server in place for running other background tasks/only had to add an endpoint). Lot’s of AI stuff plays nicely with Python… hopefully you can sort this all out

ed19 · 2024-08-27T14:58:01.015Z

App looks great! So there’s that haha.

We run a platform that houses dashboards and other administerial functions for ~ 120 Shopify stores, which I too built back in 2022 on the legacy model. Once bubble transitioned over to displaying (but importantly not charging for) wf units, it highlighted just how inefficient my operations were, when previously I’d had no idea.

Our wf unit usage sat at upwards of 4m/month, which eclipses any new plan available. I worked it out as costing us a couple thousand pounds each month with overages (up from legacy production £600), which was obviously a huge problem.

For months I was dreading the change, unable to see how I could possibly make it that much more efficient, but funny story… for a humungous application, with only me working on it and pretty high stakes financially if I mess up, it took me one solid week (admittedly working until 2-3am every day) to reduce our monthly wf unit usage from 4 million plus, down to about 300k.

I guess what I’m trying to say is all hope is not lost! You’ve delivered a great client-facing product from the looks of it, and that is by far the hardest part.

gnelson · 2024-08-28T05:58:57.840Z

What new pricing?

gnelson · 2024-08-28T06:16:14.270Z

I HATE AIRTABLE! Just saying.

I’d love to learn how to take advantage of your python suggestion, but how would I find out what scenarios it would benefit?

Thanks for the suggestion.

ihsanzainal84 · 2024-08-28T08:12:15.121Z

I just want to add that I hate Airtable too!

Though it’s because of that annoyance with Airtable that led me to find Bubble.

zelus_pudding · 2024-08-28T16:08:43.440Z

Knowing when to write your own Python API to extend a Bubble app, for me that usually comes down to when doing compute heavy actions… Bubble’s greatest value proposition is “display stuff in a browser”… it’s not very good at “compute many things” to display in browser. For example, the cost calculator for one of my plugins is powered by a Python endpoint… could I have done that in Bubble? Probably. Is it way easier in Python - yes! Expand the hidden code below and think about how to do it in native Bubble… I probably could have baked this into an Expression element using javascript but… because I already have a Python API setup, just adding this endpoint was easy enough. Beyond that, if trying to fetch data from other APIs/was a longer running process, like I was here, then that’s even more reason to try to do it in Python.

Cost Calculator Endpoint Code

@app.get("/api/v1/get_search_provider_pricing")
def compute_search_provider_costs(
    search_provider: str = "algolia",
    average_record_kb: float = 0.6,
    number_of_records: int = 50000,
    monthly_searches: int = 50000,
    regions: int = 1,
    is_new_algolia_plan: bool = True,
):
    def select_typesense_cluster_base_cost(
        number_of_records=50000, average_record_kb=0.6, regions=1
    ):
        region_cluster_pricing = {
            1: {0.5: 14.40, 1: 21.60, 2: 36, 4: 64.80, 8: 122.40, 16: 180},
            3: {0.5: 64.80, 1: 86.40, 2: 129.60, 4: 216.00, 8: 432.00, 16: 669.60},
            5: {0.5: 108, 1: 144, 2: 252, 4: 432.00, 8: 828.00, 16: 1260},
        }
        cluster_pricing = region_cluster_pricing[regions]
        cluster_gb = average_record_kb * number_of_records / 1000000
        minimum_cluster_gb = cluster_gb * 2
        recommended_minimum_cluster_gb = [
            key for key in cluster_pricing.keys() if key > minimum_cluster_gb
        ][0]
        base_monthly_price = cluster_pricing[recommended_minimum_cluster_gb]
        return recommended_minimum_cluster_gb, base_monthly_price

    if search_provider == "algolia":
        if is_new_algolia_plan:
            if number_of_records <= 100000:
                cost_number_of_records = 0
            else:
                cost_number_of_records = (number_of_records - 100000) / 1000 * 0.4
            if monthly_searches <= 10000:
                cost_monthly_searches = 0
            else:
                cost_monthly_searches = (monthly_searches - 10000) / 1000 * 0.5
            cost = cost_monthly_searches + cost_number_of_records
        else:
            if number_of_records <= 10000:
                cost_number_of_records = 0
            else:
                cost_number_of_records = (number_of_records - 10000) / 1000 * 1.0
            if monthly_searches <= 10000:
                cost_monthly_searches = 0
            else:
                cost_monthly_searches = (monthly_searches - 10000) / 1000 * 1.0
            cost = cost_monthly_searches + cost_number_of_records
    elif search_provider == "typesense":
        recommended_minimum_cluster_gb, base_monthly_price = (
            select_typesense_cluster_base_cost(
                number_of_records=number_of_records,
                average_record_kb=average_record_kb,
                regions=regions,
            )
        )
        # works out to 720MB per day per node across a month multiplied by regions and 0.09 GB
        bandwidth_out_cost_base = 0.720 * 30.44 * regions * 0.09
        bandwidth_out_cost_searches = (
            monthly_searches * average_record_kb * 100 / 1000000 * 0.09
        )
        cost = (
            base_monthly_price + bandwidth_out_cost_base + bandwidth_out_cost_searches
        )
    return {"cost": cost}

georgecollier · 2024-08-28T16:37:26.109Z

+1

I now have a library of custom APIs that I use with various apps we work on. Some built with Python (using Flask), some with Node.js. I host them all on Google Cloud Platform for pennies.

Examples:

File conversion endpoint that converts any endpoint to text (saves using expensive external API)
Pinecone upsert endpoint that takes a text and custom metadata, chunks it, and upserts it all to Pinecone (saves loads of WU for large documents)
Progressive tax rate calculator that takes tax bands, deductions, and an amount, and returns the relevant tax (would be more efficient in a client side plugin but wasn’t a rabbit hole I was willing to go down)
Wasabi URL generator (the plugin I use has this integrated but Bubble SSAs take ages to spool up, so I just deployed it on Google Cloud).

The best thing? I coded all of them basically entirely with ChatGPT / Claude 3.5 Sonnet (Claude is better at this in my experience). I can’t write code - I can look at it and understand what’s going on, but I can’t be bothered with syntax (hey, that’s why I use Bubble!).

I dream that one day Bubble will add proper SSA building that’s reasonably priced and would actively encourage developers extending their app with custom code. However, it’s clear their product direction is ‘make bulk data better’ so that custom code becomes unnecessary.

I’m a Bubble-native all the way advocate wherever it’s reasonably possible, but Bubble isn’t perfect for every use case (and how could you expect it to be), which is why it’s okay to integrate it with other tools

senecadatabase · 2024-08-28T17:11:47.701Z

Nice idea you have.

I do think the code could be improved?

It makes the cost calculations easier.

Has error handling…

and rounds the final costs to 2 decimal places.

Just something you can consider.

gist.github.com

https://gist.github.com/senecadatabase/aad9734cadb57b142366891342afcadf

gistfile1.txt

from typing import Literal, Dict, Tuple
from fastapi import FastAPI

app = FastAPI()

# Constants
ALGOLIA_FREE_RECORDS = 100000
ALGOLIA_FREE_SEARCHES = 10000
ALGOLIA_NEW_RECORD_PRICE = 0.4
ALGOLIA_NEW_SEARCH_PRICE = 0.5

This file has been truncated. show original

ryan8 · 2024-08-28T18:30:21.269Z

bubble actually called me yesterday and a spoke with one of their engineers / product support guys, and we talked about adding the WU cost to the workflows themselves so that when you create a workflow you can run it to determine the total cost. He liked the idea and wanted to take it to the team… so maybe they will implement it. anything to cut down on log slogging

zelus_pudding · 2024-08-28T21:50:27.342Z

@ryan8 I kinda assumed this was something they’d thought of before. Jaw on floor if not true. Would have to be something normalized, though… so like reported as WUs per 100 records.

@senecadatabase Very cool, thanks for doing that

georgecollier · 2024-08-28T21:52:52.073Z

zelus_pudding:

Jaw on floor if true

From https://www.youtube.com/live/o-mrUwQa1aM?si=HGnU2TPrzGE_DnEO&t=986

Revamped workflow logs design

CleanShot 2024-08-28 at 22.51.56@2x1314×766 90 KB

ryan8 · 2024-08-28T23:17:23.709Z

well, I don’t wanna say bubble didn’t have the idea. They could’ve just been nice. Maybe they already had it on their Horizon. But I know they were gonna take it back to the team either way, even if it’s a maybe to help prioritizing that function. But it was a fun talk and nice to chat with them.

ryan8 · 2024-08-28T23:29:39.673Z

Maybe this is related to the big news at bubbleCon considering how important it is going forward

gnelson · 2024-09-04T14:23:38.235Z

Thank you!

gnelson · 2024-09-04T14:26:17.431Z

Great information. Thank you. I hadn’t heard of Claude. I’ve been using ChatGPT with somewhat satisfactory results. But since you say this is better I’ll give it a try.

gnelson · 2024-09-04T14:31:03.009Z

My experience was the other way. I was already familiar with bubble, then went to work for a company that is in love with airtable. Were way too big for AT. We’re a manufacturing/construction company. Way too many parts, pieces, tasks. We exceed the limits of AirTable ok just one project. So I work around it with bubble. LOL

matthewbuilds · 2024-09-04T15:01:31.567Z

I can personally vouch for Claude Sonnet. Been using it for about a month to build out some pretty complex stuff and it rarely misses. Even when it does, eventually it will figure out the solution, even if it means scrapping the original implementation. It’s much more flexible syntactically and often comes up with creative solutions due to that flexibility.

ChatGPT is too prone to making mistakes with code and often struggles with consistent layouts. Especially when you try to change something. It’s just too rigid and limited and has a bad habit of breaking code that works while attempting to correct code that doesn’t.

Claude Sonnet is superior in almost every respect. The only downside is I occasionally have to remind it about how I want something done after a long conversation…but that’s few and far between.