Possible to find the source of the elasticsearch queries?

[makingthingsapp]

Our app is getting quite complex now, and if I look at the network requests the app is making there are a bunch of requests to /elasticsearch/search and /elasticsearch/msearch. Some of the responses are quite big, and contain a lot of data that is not needed for the page.

I thought I could find the source of these by looking at the objects loaded in debug_mode, but it appears the searches are done before the debugger runs. There are thousands of workflow items and elements in the app so it’s hard to figure out what is causing the searches.

Is there any way to debug this or find out which elements are responsible for triggering which queries (or anything like that)?

Thanks!

[mishav]

@makingthingsapp

elasticsearch

I think it was @josh mentioned in one post that the backend is no longer using elastic search, and its named that way for backwards compatibility.

@makingthingsapp

Some of the responses are quite big, and contain a lot of data that is not needed for the page.

@josh mentioned in a another post that returning the results from a search not only returns all fields for the matching rows, but also all matching rows for any referenced thing.

For example, if User has field UserOrders (list of type Order), a search on Users will return the matching users, plus the full details of each of their Orders, plus any thing referenced by Order.

Performance Q&A guide

I’m going to be checking this thread periodically over the next week to answer questions on performance best practices, inspired by (among other people who have asked for this): This will probably get turned into a performance section in the manual, but I figure the forum is a good place to kick things off. I have to run for now but will chime in later today with some general tips + answers to blueback09’s questions above.

[makingthingsapp]

Thanks @mishav - yeah, I noticed it was returning all the referenced fields. The issue is there are thousands of workflow items, and thousands of elements in our app: and many have similar-but-slightly-different data clauses - so I’m having a really hard time figuring out the source of those referenced items.

There must be a stage where Bubble parses the app and bundles together all the required queries into the final big batches. I was hoping there was a way to debug and see where they came from.

For example, if you use debug_mode=true you can step through the workflows and inspect each item: but the data you see is already loaded by this time. At some point before then, Bubble went “this workflow will eventually run, so add this database query to the batch of queries”. Similarly they must look at all the things in the designer and say “Element X has a data source of table Y, which also needs to load data form table Z”

If we could see these mappings then it would make it much easier to analyze unnecessary or heavy database calls.

But if not, I’ll just start deleting big chunks of the app, monitor the change, and then undo it until I find the cause

[mishav]

@makingthingsapp

deleting big chunks of the app, monitor the change, and then undo

This sounds rather risky, relying on undo for large changes … maybe on a copy of the app? A good approach though : )

@makingthingsapp

figuring out the source of those referenced items

I suggest emailing Bubble support, this is something they help with on complex apps.

@makingthingsapp

thousands of workflow items, and thousands of elements in our app

How many on the bigger pages? Maybe the app is pushing the boundaries past a well-performing Bubble app?

Can some of the workflow logic be moved to server-only API workflows or to client-only javascript?

It’d be interesting to experiment with reuse of database search results.

One thing you didn’t mention is performance in terms of subjective response time, is that okay even with the large responses?

[makingthingsapp]

Thanks for the tips @mishav - I will try and figure out how I can break things up into smaller pieces and test them individually.

The response time is stable - in that it increases linearly with the size of the requests… but the issue is really the bandwidth: because the app will be used a lot on mobile devices, on mobile data plans, we want to limit any unnecessary calls.

I think we’ll get it with some good old trial-and-error and detective work, but I’ll log a feature request - because I think some kind of “Data mapping tool” could be useful to the bubble community.