Bubble workflow inefficiency

[mitchbaylis]

I have a file storage structure
folder
file

I want a count of files per folder
So I add the aggregate count to do this - easy.

However, the aggregate count is cached and not refreshed when the data changes (files folder changes). So if I do something like - change files folder then the count doesn’t change on the page.

My setup was:
repeating group of folders with each row doing a “search for files:count”

The user usually only has 20 folders so this method seemed the simplest and cleanest way to handle getting the file count but it has issues due to caching. The caching seems to only be an issue within a repeating group since doing “search:count” outside a repeating group works fine.

I tried many methods to force the count to refresh - clear data, reload data etc - none work. Only a page reload resets it.

So now I have to store the count as a field on the data. This creates WU overhang just to update this simple number that is rarely used in the app (would be much more efficient to count it only when needed for this 1 repeating group).

search : count = 0.3 + 0.2 wu
= 0.5wu on each page load that has the count

instead I have to update the folder each time the underlying data changes on the file. This creates 3 trigger points:
1 - previous folder removed
2 - new folder added
3 - file deleted (only want to count active files)

Fine - I can add database triggers for those actions.
So now I have 2 database triggers
1 - folder changed on file (updates old and new folder counts)
2 - file deleted

Each time it then recalcs the aggregate on the folder so:
search:count = 0.3 + 0.2 wu

• server side workflow write action = 0.6wu
  total = 1.1 wu per update

Now if the user transfers 1000 files from folder A to folder B
The database trigger is triggered 1000 times
The folder aggregate search is redone 2000 times (1 for old and 1 for new folder)
The database trigger also has WU overhand of 0.05 wu when ANY FIELD is changed.

total = 2210 w (assuming 1000 files moved to a new folder)

It would be much more performant to just run the aggregate search once on page load when viewing the folder. Yet I have to build a multi layered system to keep the count updated just so I can show a list of folders with file counts to the user.

Is this really the only way to achieve this aggregate count in a way that isn’t cached for the user?

[sudsy]

@mitchbaylis

So if I do something like - change files folder then the count doesn’t change on the page.

I’ve always been able to achieve real-time count updates with just the regular count operator (no RG involved)…

Search for Things:count

…but maybe I’m misunderstanding what you’re trying to accomplish.

BTW, it’s not clear to me exactly which expression operator you’re using. I’m not aware of an operator called “aggregate count”. There’s only “count” and “cached aggregate” (which can be either “cached count” or “cached approximate count”).

[mitchbaylis]

I’ve retested this with a search:count that sits outside of a repeating group and the same issue occurs - maybe this is a new bubble bug as I’m pretty sure it used to dynamically update the count when data changed.

Simplified issue:
search:count

Change the underlying data in the search result and the count does not change.

ie search for files where album = album : count
then remove the album from a file - count should change but it doesn’t
refresh the page and the count changes to the correct count

[ihsanzainal84]

Search:count is supposed to be realtime versus :aggregate AFAIK. I rely on it for my booking app.

I did not see you explicitly saying trying a “reset group” action. Have you tried that just to test?

This does sound like a bug.

[sudsy]

Are you by chance setting the count in a “Page is loaded” event / workflow? In that case it, might not work as expected.

As a test, try putting the “search:count” directly in a text element on the page. That should leverage Bubble’s notifier mechanism and update in real time.

[mitchbaylis]

yes, I believe it is a bug and I have submitted a bug request

no, I’m not doing the the search:count on page load - it happens on the page in a text and in a repeating group

[boston85719]

Bug might not be search count updating. I noticed a bug crept back in yesterday where the list of things saved on another data type doesn’t automatically remove from the list the related thing that was deleted.

[mitchbaylis]

that’s an old bug - it’s never been resolved and it’s been that way for years in bubble

when you delete a thing it never removes from a list it was in because it is only ever “soft deleted”

the app editor shows it as deleted. the user can no longer see it. but the data is still linked to everything and still evaluates in expressions and still returns to the front end.

the same happens for option sets/data/styles/data tables/data fields etc - basically everything that you think actually gets deleted is never actually deleted.

there’s many forum posts about the “not deleted” issue and it’s never been resolved despite all the security concerns and wu overage charges (you get charged WU for the deleted data)

the search:count getting cached is a new bug from what I can tell.

** for additional clarity I am removing the item from a field/list not “deleting” its record and expecting it to be removed from the field/list

[boston85719]

I’m not talking about that issue…that is issue you mention is related to a data type or data field being deleted from the editor.

What I am referring to is a bug that comes and goes and in my most recent tests, was not present, which is if I have a data type with a field called “products” which is a list of “products” that is a data type and there are 5 products in the list of “products” field, and I delete from Product data type a Product that is in the list of 5, normally, that list in the other data type is updated and the Product is removed from the list…however, sometimes that bug creeps in and does not automatically delete the Product from the field list of Products on the other data type. This will cause the count to be wrong.

@mitchbaylis

for additional clarity I am removing the item from a field/list not “deleting” its record and expecting it to be removed from the field/list

That is the bug that I am talking about as having maybe crept back into the system, so what you are experiencing likely is a different bug.

[mitchbaylis]

to my knowledge bubble has always had the deleted thing still in a list bug

shopping cart has list of 5 cart items
user deletes a cart item (deletes the thing, but doesn’t remove it from the carts list of items)
shopping cart still has 5 cart items

I’ve always built the workflows as
remove from list
delete the thing

although I rarely use delete a thing and just soft delete it in the data instead

the bug I am reporting is search:count being cached when it should not be and didn’t use to be (from what I remember - but maybe I just didn’t notice it)