Slow repeating group

[ryanck]

Hello there,

I was wondering if is there going to be an improvement of the repeating group showing data slow. Because we´ve to show more than 100 items and it´s really really slow that sometimes even breaks the app.

Can you us something about this @emmanuel

Thanks a lot.

[emmanuel]

Searching something in the database slow

We’re looking into this.

We’re working on this.

[ryanck]

Nice, I did not now it was related.

Thanks a lot.

[ryanck]

Any news about this please?

It´s our main complaining issue from users.

Thanks a lot.

[andrewgassen]

We’ve got a large repeating group with two medium sized repeating groups nested inside. We just pushed an update that changed how we queried the data, and our speed improved by at least 10x. How are you loading the data in your group? There might be some optimizations you can make, depending on your setup.

Only mentioning it because prior to our update, we lost a ton of users because of performance, and it turns out it was just cuz we weren’t making efficient data requests.

[ryanck]

Hi,

We´re loading the data with a “do a search for …”

Is there any other way? Since we´re loading invoices that could be more than 1000 for each user.

Thanks a lot.

[andrewgassen]

When I moved away from “do a search for,” that’s what caused my performance to jump up so significantly. I’m not sure what your comment about being more than 1000 for each user means? Last I heard, the limit to a list size was upped to 10,000, so if you’re concerned about that, maybe it’s not an issue anymore?

In my app, I have students and skills, and each student has a rating for each skill. Previously, I was doing this:

• do a search for ratings where student = parent group’s student and skill = current cell’s skill

Now, I’m doing this:

• Current cell’s student’s ratings:filtered (skill = parent group’s skill):first item

So, there is a field on the student called “Ratings,” and I’m specifically querying for that list of ratings. It’s the difference between saying, “Look through the entire universe and find me stuff that matches this constraint,” vs, “Look through this one student’s stuff and find me stuff that matches this constraint.” Much faster, for sure.

[ryanck]

I think I understand that but I´ve users that already has about 300 invoices and in the user I don´t have a field called “invoices” which is a list of invoices.

That´s what you mean? To have a field on the user of the “invoices” so I can retrieve a list instead of making a “do a search for …” ?

Thanks a lot for your help @potentialthings

Bests.

[andrewgassen]

I think that’s what I would do in your case, yes. Now, the challenge is going to be migrating the existing invoices to make it work with your existing data…interesting.

[ryanck]

Yep, that´s really interesting. Hehe

[Bradluffy]

I was doing filtered for a while, but @emmanuel had advised in other posts that do a search for would have the quickest results. Has something occurred that would change this guidance?

[andrewgassen]

I thought @emmanuel was saying that the “do a search for” would be slower? That’s what I’m basing my info off of, but in the real world, making the change away from that dramatically improved my performance. I wonder if maybe that’s just a happy coincidence? Hmm…

[Bradluffy]

Now I am curious. Although that would be a major shift in my own app.

[markf]

Also got me curious now as I’m currently building something that has a very similar feature. Definitely food for thought

[ryanck]

I also thought that the do a search for was the quickest one. Now I don´t even know what to do.

[mishav]

@potentialthings

In my app, I have students and skills, and each student has a rating for each skill. Previously, I was doing this:

do a search for ratings where student = parent group’s student and skill = current cell’s skill

Now, I’m doing this:

Current cell’s student’s ratings:filtered (skill = parent group’s skill):first item

Hi @josh, would you be able to comment on why the second scenario is quicker?

[andrewgassen]

I guess I’d say try things until you’re happy with the performance, which is a lame answer but also similar to regular software development. I guess for me, it feels like doing a search for would be intuitively slower because it’s going through the entire dataset available on your app, rather than starting at a smaller scope. The difference between looking for a fish in a kitchen sink vs looking for one in an ocean.

Then again, I’m not an engineer so I have no idea what I’m talking about 95% of the time

[ryanck]

I like that “Then again, I’m not an engineer so I have no idea what I’m talking about 95% of the time ”

Me too! Haha. That´s the fun part.

[Pat]

:filter downloads all datasets to the client and hence is definitely slower

Filtering versus Search for

It is always better to have as many constraints as possible in the search, the filter is more of a 2nd solution if there is a constraint you cannot express within our current search engine.

Repeating group loading - slow

Filter in general is client side, it’s better to have every constraint in the search if possible.

How to improve the speed to read from DB

Speed is something we work on constantly, but not a quick project. While things will get better over time, if you identify a pattern on a specific page / repeating group, email us and we can see if there is a quick win there. Saying ‘speed is generally quite slow’ isn’t specific enough for us to look into it. One thing to note is that you should avoid using ‘Filtered’ except if you absolutely have to, as it’s slower than using search constraints.

[andrewgassen]

I have no idea why my experience is the opposite of this. Reading this explanations makes sense, but my real-world experience says otherwise.

• Head to https://joyfulliteracy.com/version-test/sign-in
• Login as andrew@betterproduct.co
• Password = Iambubble2017
• It should navigate you directly to the https://joyfulliteracy.com/version-test/circle-charts
• Notice the time it takes to load all the circles.
• Now, manually navigate to https://joyfulliteracy.com/version-test/circle-charts-old
• Also play around with clicking the circles, and notice how the time to update the circle differs

Our teachers are reporting a difference of up to 3 minutes in loading time for the same data sets. The dataset in the example I shared is small, so the difference isn’t so drastic. Literally the only thing we changed was going from “do a search for” to a more focused search, shown in the images below.

Old Method

New Method

My Reaction