App Interaction Statistics

Hi guys,

I need some help with something , mainly an opinion.

I am wanting to set up a way to track app usage data.

I would like to log use off my app - user clicks and element interactions because I’d like to be able to show - for example; how many external users have clicked on a link on the app, and I would like to have the option to display this in time periods or ranges in the future.

I’m worried about building a datatype that grows to become huge at some point. I understand the importance of separating data, so I’d like some suggestion on what will be the best way to handle this.

One way that I thought of approaching this, is by creating one general data type and called ‘statistics ‘ and then create individual data types to associate to this table in a related fashion.

So for example, under the statistics data type, they’ll be a field called ‘related image clicks’ which would be related to a separate data type ‘statistics - image clicks’. My logic is that I don’t want all the data records to be loaded each time the data type - Statistics is loaded or called on, rather have them broken down into individual statistic data type components, if that makes sense.

  1. My first question is, should I create a general ‘statistics’ data type and then relate additional data types to Fields within that data type in order to keep it organised as above, or simply create each data type for the static’s I’m trying to track; example ‘statistics - image clicks’, ‘statistics - read more, etc? I guess creating the general ‘statistics’ data type would mean I best both.

Is it better that’s having one data type called statistics and then have various fields that trigger when it’s clicked for images clicks, links, scrolls, etc, etc? I could use an option set up identify that each click relates to. Or am I better creating individual data types? Hope this makes sense.

  1. In recording the image, click statistics, should I create a row in my ‘statistics - image clicks’ data type - one for each click? Or, am I better to simply add it to a numerical field in that data type, for example ‘Statistics’ and then have a field for ‘Image Clicks’ within - thats is a numerical value, that adds up to a value every time it’s clicked?

I think the limitation with this is that, although the number will continually increase every time, the clicks will not be date or time stamped, so I won’t be able to filter the stats based on whether it’s been done, monthly, weekly yearly and the like survey.

The other option is to create a new record under’Statistics - image clicks’ each time it’s clicked.

I’ve tested this and it works well and I’m able to show a repeating group in the app and demonstrate this data.

My only concern is that over time and possibly very quickly, there will be ‘millions’ of records created and could cause a problem.

Is this a concern or is this the best way to go about doing it? Does anyone have any recommendation on how such click interaction data can be tracked in bubble.

I want to do this as I think it’s critical to showing my app subscribers the value of the app by way of user interaction.

Any direction will be great.

Thanks

Hi Guys,
Just wondering if anyone has any insight on this? :slight_smile:
Any feedback would be much appreciated.
Thanks

Hi Costas,

Let me take a quick stab at this. If you want to demonstrate clicks over time for each item seperated by user, then you need to track each click each time each user clicks each element. You’re right about it being millions of records if you have high usage.

This assumes you want to build it all yourself in vanilla Bubble, no plugins no js.

The way big analytic platforms (eg.: Youtube Studio) seems to do it, is to break it down data a priori into two categories: aggregate data and real-time data. The real-time data is granular and updates instantly, while the aggregate data updates once and then that number is there forever.

You’re right that would be a lot of compute, especially going back through a long time like a year.

And then you’d lose some of the most important feature of data, it’s relevance over time. So you have to determine, within the constraints of your use case (this is important, do the minimum required to achieve your goal), what data you need to collect to be able to show it real-time. Collect it and make a new record every time with the detail required for the views you’ll show on your dashboard. When enough time has elapsed that you don’t need the data to be granular, you can create an aggregate record and show that.

For example, you could keep the last 23 hourly aggregate records, 7 daily records, the last 12 monthly records and the rest aggregrated by year. You can decide whether it’s relevant to aggregate by user, by user group, app-wide, etc… That way you’re only sending the minimum number of records required for your view, and not asking the server to do math every time. With this method, you can show those aggregate records, and the real-time data as the last bar on your chart (eg.: stock price charts). Though I’m not actually sure what the threshold in number of records is for performance gains, and millions sounds like there would be some, this might actually be less performant for small datasets.

Practially speaking, while you build this:

To keep things swift on your Bubble app, especially if you have a lot of clicks coming in fast, you can use the :approximate count and :cached data built-in functions when fetching and showing your granular/real-time data. This would reduce your WU usage if you have a high volume of granular records updating constantly, but of course, wouldn’t be “real-time”. What’s close enough for your users?

To reduce compute for collection of your granular data, you can also set up custom state lists which keeps collection client-side, and only store to db when navigating away from the page or taking relevant actions actions (eg.: changing product categories, changing views), or by polling on a timer. Again, this wouldn’t be “real-time”, but is it close enough? Extra speed if you’re sending those to a backend workflow. I’m not sure how much detail you were looking for here, but I hope this helped some.

I would also welcome some insight on methods people are using to store their hotjar-like user behavior data with vanilla Bubble. How it compares to any third-party services being used.

1 Like

Assuming no external integrations, you can:

  1. Have a counter on a data type called ‘clicks’ and increase it by one when clicked
  2. Create a new Thing (Click) when something is clicked. To display the number of clicks, Do a search for Things.

I’d probably go down the second route as you can store more information like who clicked it, when they clicked etc.

That said, Google Analytics / Hotjar for example are tools explicitly designed for this, so whilst it may take longer to set up, using them will likely be way more WU efficient.

1 Like

Hi Guys (George & Duke),
Thanks very much for this insight. I really appreciate it. I will give it a go.
I understand for the most part and also some points to consider.
I like the cache idea though I have never played with this. Curious to see how this may work on some other sections of my app.
I think I will go with the create a thing record for each time its clicked but will create a set of data-types for the various link groups I am tracking so as to break this up into possibly 5-10 data-types for now. So e.g. I can show clicks for image, click for enlarge. I am just trying to implement (dare I say, a some what rudimentary) method of showing app usage to my app subscribers.

So, rather than create one data type and call is ‘Statistics’ and have a record create each time its clicked with, lets say a Option Set selection - for the type of click interaction it was, instead I am going to break it up and create a record for each type of click with a new data type for that click.

Last question, do you think I should still connect it back to the parent/a parent ‘Statistics’ data type?

Here is an example of my data base. The numbering is for ordering purposes and identification so please excuse that.

Hi! I think you had the right idea at first, I don’t want to have mislead you with my answer. Thanks for providing screens it will help me be specific. Given this, I suggest making max 2 Thing types, with the following fields:

  1. Thing called “Data”
    type field (text)
    uniqueID field (text)
    company field (text)
    User field (automatically added by bubble)
    Date field (auto added too)
  2. DataAgg
    timeAgg field (text)
    date field (date)
    view field (text)
    value field (number)

Data setup

Let’s say someone enlarges an image. Then you create a “Data” record with the type “enlarged” or whatever you want to name it. Text is fine and faster than option sets. If you don’t need to use compute/memory, don’t use it. Invent another word for written review, review item image, etc. and just change that field, not the “thing”.

Store the Image’s uniqueID, because you can sort/filter by text when you’re pulling or when you’re aggregating your data. Again, less compute/memory than storing the “image” bubble object. Text will also allow you to store different object types, for instance the uniqueID of a “review” and the uniqueID of an “image”, without having a bunch of empty fields for each record. When you’re loading a view like “image clicks”, you’ll instruct your program to find the records with the uniqueID related to the image anyway, and only from the records named “enlarged” anyway, so one field for all objects is fine. This is unless you’re composing views - for example all enlarged images by company A. In that case you’ll need to store company AND images. Again, it’s important to plan your views before your arch.

Caveat is if you’re using a field for Privacy Rules, then store the bubble object, because it’s easier to compose expressions in the privacy tab that way. But I’m quite certain storing an object is bigger in memory and requires more compute than storing a string.

Anyway, now you have all of your clicks on stuff stored in a lean, masterful table. You have the uniqueID of the image, or the review, and/or the company, ready to sort/filter/search. For a small dataset, let’s say less a few hundred relevant records after you’ve sorted by date/company/uniqueID, you need go no further, just put that in your chart. “do a search for” is really performant if you add good constraints. If/when you have tens of thousands of relevant records in a search, you can do the next step and create “DataAgg”.

Aggregating

You’ve already planned your data views, so schedule backend workflows to create new aggregate records and update the old ones for each view. Store the view as Type, date as the start date and time for the period. For example, if you’re pulling data by year, 2023’s date will be Jan 1, this month’s date would be Nov 1st, etc. Let’s say it makes sense for your use case to update your data every hour. Update the “value” field for all the "timeAgg"s, so Year, Month and Day and create a new Hour record that stores the :count value for the last hour’s clicks. Now when you show data, you can pull those aggregate values, which should be far fewer and be more performant. You can then “do a search for” only the new records that have been created since the last aggregation, which should also be far fewer, so faster.

But now you’re both running the searches AND using compute to recalculate all the values every hour. That’s why I said this method is less performant on small datasets. Finding the sweet spot between just running the searches as needed and doing aggregation is beyond my skill - trial and error is how I’d decide. Why not build both and see.

Let me know if that makes more sense, and maybe reconsider using many “things” for this task, just one or two max should be fine and easier to maintain.

Please use option sets for this! There’s much less room for error when you have a predefined set of event types compared to typing texts for each one…

2 Likes

Hi Duke,

Thanks again for detailed explanation. I need to digest this and think most of it makes sense. I will have a 3rd or 4th read then have an implementation play.

In short, your saying; to use one data type - statistics for example (leaving the aggregate out for the moment) and storing all data there?

So one data type that looks records that I can sort from based on what display I want to use?

I guess this makes sense as I can also use that data take on easier way to present data tracked in graphs?

Example or what you said;
each ‘image’ click or ‘link click’ will create a new record, but will be identified by another field? perhaps from a option set drop down option set (Thanks also George. Yes I like to use these as it helps me track things better and stay organised) image/link click , etc so that I can sort. Off course ‘company’ will be one field as it’s a SaaS and has multiple companies using.

Then when I search I can add filters based on drop-down value - type of click.

So in a nutshell, it’s ok to have one default data type ‘statistics’ rather than multiple datasets in the example of the screen shot I sent?

Offourse the aggregation is something that will come into play. I have never used this it sounds exciting.

Thanks Duke

Well a data type is a thing. What is a ‘Statistics’? Nothing very useful. What you probably want to track are Events. What’s an Event? Something interesting that happens in your app.

Create an Event whenever that Event happens. That’s all there is to it. If tracking clicks for example, create an Event and set the type field to Click (or another relevant option from your created option set), track the User who did it, the Thing they clicked on, and the date they did it, as well as any other info you might want to store.

I would have:

Event

  • User
  • Type (Click, View, Submission etc)
  • Group (Profile, Project etc - where the event happened)
  • date (date the event happened)

Hi George,

Thanks mate.

Yes makes sense. ‘Statistics’ is the name of my datatype for tracking such interactions - what I would log each event under.

I measure things in different other ways in the app, but this is for measuring how outside people seeing the content interact with it. Example; how many people clicked on review image, link, etc.

So from the example you gave offcourse reference to ‘Events’ logged is a datatype?

My largest concern was causing the datatype were these are being stored to grow to large that’s why I thought it may be an idea to break these up into separate data types as shown originally in my example.

Thanks again for help! :slight_smile:

Yes, in a nutshell that’s what it took me paragraphs to say.

I think you were on the right track when you guessed millions was too large. I think you can have a very preformant data table with tens or even hundreds of thousands of rows, as long as your search conditions are good.

Hi Duke,

Thanks. Sounds good. I will proceed to make the changes.
Really appreciate your help.