Search 2.0 - Summarizing what I've learned exploring Algolia

Idea
1

I’d like to contribute what I’ve learned about Algolia for anyone else exploring how to add a faster, elastic-style search to Bubble. Added, this now works in bubble (read on).

Algolia = paid tool for indexing and retrieving data through search bar like capabilities. It’s crazy fast. For us, we’re exploring as a way to improve the speed of repeating groups and create a site-wide search-bar.

Basic setup:

  1. Use Bubble connector to POST and PUT data into Algolia indexes.
  2. Use Algolia JS libraries, like instantsearch.js, to add search bar capabilities within Bubble.

Major Challenges:

  1. Keeping Bubble data in sync with Algolia index. This could be done either by real-time or batch CRUD.

  2. Formatting Algolia indexes to enable relationships between what are 2+ source tables in Bubble. For example, if you have a Users and Orders tables in Bubble, can either 1) add a lookup column in the user’s table or 2) flatten the tables into one to maintain the relationship.

Method#1 Post creates two indexes in Algolia
User [ { first_name: Joe, last_name: Bob, OrderIDs: {ID1, ID2, ID3} ]
Orders [ { OrderID: 1, item: Apples, Price: $5 } ]

Method#2 POST is a flatten table to create single index in Algolia.
User-Orders: [ { first_name: Joe, last_name: Bob, OrderID: 1, item: Apples, Price: $5 }]

There are trade offs to each method.

  1. Setting up a Algolia search-bar and getting the result set back into Bubble… Algolia has a set of JS libraries with widgets to create on page search. It’s custom code, but using @mishav’s tool kit, it’s possible to render the widgets within bubble. This works. Ideally, instead of custom coding how the widget appears on the page, there’d be a way to grab the result-set and make the result-set available to bubble elements, such as a repeat groups. This would greatly reduce the custom code.

For example, I’d was looking for a way to bring these results "Tom Smith, … " into a Bubble so that I can format this data within a bubble repeating group. This would avoid the need to custom code the way the widget looks because we could simply use bubble elements to display the results.

(Image of instantsearch.js widget within our Bubble app.)

image
image784×436 22.7 KB

Overall impression: Algolia is super fast and affordable. There is definitely overhead to keep Indexes in sync with Bubble DB. The main challenge, and where I got stuck, is how to bring the result-set back and make it available to bubble’s native elements (repeating groups in particular).

10 Likes
2

@sridharan.s

3

@Kfawcett, thanks for thinking of me. @kramwe is my business partner.

1 Like
4

ha! :grinning:

5

This was exactly my challenge too. I started working on creating an Algolia plugin for bubble some months back and it was for this reason i abandoned the plugin development.
Custom coding widgets makes it not ideal for everyone as a plugin.

I’m hoping maybe bubble themselves will create a plugin for Algolia for better integration.

1 Like
6

The search feature is so core to a product like Bubble that I wouldn’t invest time in Algolia or any other search engine. Amazon Cloud Search is another option that I once thought of.

It’s just a matter of time that @Bubble team comes up with a nice end-user implementation of Elastic Search which is what I believe they currently use in their stack.

Also, Elastic Search already has a premium out-of-the-box solution for site search similar to Algolia.

https://www.elastic.co/solutions/site-search

2 Likes
7

I think this method has its place particularly where you want to incorporate external datasets and not wholly store them within Bubble.

If you bring back a list of texts, each text item being a JSON of the search hit, you can feed the RG.
Extracting the individual values is a little clumsy. Here’s an example using an Expression for each value.

demo editor - Edit - now set for viewing!
demo runtime passing a query

14 Likes
8

HOLY COW!!! @mishav, you did it! For production apps, this is a game changer. @seanhoots ^

Our goals have been 1) page performance and 2) add site-wide search. This will do both. And it’s relatively easy to set it up (1-2days).

  1. Repeating groups have been a bottleneck for page performance because of the amount of data that is natively loaded to the page. This should solve that because an Algolia index can populated them faster than a Bubble query.

  2. Site-wide search. Aloglia has a full suite of tools for controlling the search result set eg. how words rank, typo tolerance, synonyms etc…

So awesome!

2 Likes
9

Good that its useful @kramwe :_)

A small improvement inside the repeating group, would be to retrieve text values at the same time in a list of texts, number items at the same time in a list of numbers, etc.

10

Okay, great. Thanks for the demo.

11

Nice one @mishav.
You should consider turning this into a “semi-plugin” so that those who are up to the task will be able to start with it.
You could make different algolia widgets (e.g. search, filters, pagination, etc) into different bubble elements that users could add to a page - this was what i was aiming for with my plugin.
Good job. :clap:t3:

4 Likes
12

I’d chip in some cash. Otherwise, the demo code is definitely usable as is.

13

I’m definitely giving this a try!
Will share my results.

1 Like
14

FYI I’ve updated the demo app to be more reliable on a slow network connection.

15

Interesting ideas @seanhoots! How far along did you get with it?

It’d be good to offer default styling, and also allow custom styling, along with the widgets.

Were you looking at providing API calls too?

Would you guys be interested in sponsoring it to become a free plugin?

4 Likes
16

@mishav, how much are you thinking?

2 Likes
17

Hey all
Does this make searching the bubble database quicker?
In the demo I could see no data and seam not to be savvy enough to understand where the results are coming from

18

@chad, Algolia requires you to load your data to an index in Algolia, which essentially is their version of a table that is optimized for retrieval speed. So, you’d pass your data to them via an API.

The advantage of using Algolia is speed and search robustness. Algolia gives you advanced search options (way beyond filters, misspellings, and fuzzy logic) and they return search results usually in less than 20 milliseconds.

You can either add records to that index each time a new thing is created or changed (a bit complicated to set-up and maintain, but enables near real-time searching of the data in Algolia) or you can sync the data in a batch process every x minutes (much easier to set-up and maintain, but it means the search results may not show for items just added to your database).

All of this is a fair amount of work to set-up and maintain for smaller apps / prototypes, but is normal for a production-grade application. And, Algolia happens to be incredibly affordable when makes this work well most anyone on Bubbler who wants to take the time to implement and maintain it.

2 Likes
19

What I’ve discovered so far: you need to eliminate any dash signs from the data fields in order to populate them via the JSON parsing, underscores work though.

What I’m trying to solve now is this:

  1. Show Algolia results based on creation date whereby newest data pops to the top.

  2. Show data from Algolia based on “filters”.

Imagine the test app, but let’s say that each user has brands that they follow and you only want to retrieve products that is made by the brands that the current users follows.

Brands are it’s own data type and so when a user follows a brand the unique ID of that brand is added onto the ‘Current User’s followed_brands’ which is a list of Unique ID’s representing brands.

Now each product has one of these Unique ID based on which brand’s product it is.

Each user should see only the brands that they follow.

Any ideas to solve this would be appreciated. :slight_smile:

2 Likes
20

Edit: I just realised this may be a little off topic should I pm or switch to another thread?

Thanks for the reply and fantastic explanation.

So I’m my instance a user may have up to 3 large supplier price lists some 3000 items.

Of which the user would build up a product list using these items.

Would this be beneficial in this? Do I still have those items in bubble and just refer back to them?
If so what would be the best way for users to have different price files (same supplier different pricing)

In this case you could imagine a trade hardware shop of which they give customers different discount depending on volume, however this discount is not dictated by me or the app but by the hardware supplier who provides each customer with a csv with their pricing

1 Like