Forum Academy Marketplace Showcase Pricing Features

Automation Testing Recommendation

Hello Bubble Team!

Is there a recommendation on automating some testing for our Bubble Application? We do not have the resources to assign someone to manually test the bubble app before and after every change!

Thank you,
Nicholas Choi

Have you set-up automated tests before? In my experience, they always sound more helpful than they are – sometimes they still make sense though.

Automated tests are non-trivial to change. And, when they’re not fully updated they often don’t work very well. Plus, when there is a bug, it’s hard to know whether the bug is in your app or in your automated tests. All of this compounds when the person coding your Bubble app is different from the person writing to your tests because now the two of them have to coordinate on everything.

So, sometimes it’s easier to do manual testing even though it’s not the most fun thing to do. And, you only really need to test the pieces that might have changed with the most recent code release.

Of course, if you’re building a larger, production-grade application then automated tests may make sense. They can work, for sure, but they come at a pretty large cost in my experience.

3 Likes

Thanks for the response! I have set up automated tests before and have run into those issues exactly. However, it feels like manual testing is not a great option either: we are trying to create a production-grade application as a very small team. We need to make sure that our changes are not breaking things for the customer, but we are also need to iterate very quickly.

In short, it seems that we do not have the human resources necessary to click through the application every hour, but we absolutely need the application to work as intended. Does this make sense as the way forward, or should we really be looking at manual testing?

Makes sense. I think both approaches can work, but they both cost time/effort. It’s easy to think of automated tests as being “automated” but the problem is that the automation needs to continually change as your product changes, so it has a large cost to it. And, it’s usually hard to keep it perfectly in sync with your product changes so what usually happens is the piece of the product you just changed doesn’t have automated tests yet, so you have to test it manually to ensure that the updates work.

With custom code, I think it’s really useful to add unit tests that validate all of the small pieces. With Bubble, there’s no corollary. We had built functional tests for our app to fill this gap and found it more costly to explain/fix with the outsourced team than to simply manually test it ourselves.

I don’t think the “answer” is simply using manual tests or using functional tests. I think the right solution largely depends on how you implement manual testing and/or functional testing. If I were coding functional tests, then it’d be easy to keep them in sync with my Bubble code (same for my business partner). But, using an outsourced dev team had a lag. And, we were thinking of automated tests as code them once and they’re done, but that assumption turned out to be way wrong (we’re highly iterative with our product and the tests would have needed to iterate as well). These two problems led to functional tests not working well for us.

Our current approach: we don’t manually test everything all of the time. We thoroughly test pieces anytime we make major changes to it. And, we test more superficially whenever there are minor changes. Bugs still get through. Some bugs are even Bubble bugs, and those change over time as Bubble updates their code (some get fixed, but new ones arise).

So, I feel your pain. We’re looking for a great solution here too, but don’t have the answer. We’ve dropped our functional tests and just test manually. We also use our product with a lot of users and work to build relationships so that they’ll tell us whenever they see something that’s broken. It works, but sure would be nice to have a better solution.

Bubble also makes debugging really difficult - the logs are not readily searchable; they’re super slow, etc. We’re spending enormous amounts of dev time walking logs! We’re on a dedicated server so that Bubble can’t keep introducing new bugs (which is required for us), but we still have to update Bubble fairly often to get the new bug fixes (which then often breaks other things).

Perhaps there are other people who have smarter approaches than we do. I’d love to hear how people are managing these types of challenges with production-grade products.

4 Likes

Thank you sridharan! That’s an extremely detailed and well thought out response. We’re relatively new and will update here with whatever we figure out. I (and the team I work with) are real fans of a lot of aspects of Bubble, but this has been a real challenge.

1 Like

Hi! we are having the same problems! we have made at least two products that need to work perfectly because the transactions that the business make are above of $1000 USD, so what we made is to have the critical processes in a function, (lambda, google cloud, etc) and have it properly tested with unit testing and some mocks.

For the things that we made in bubble, we always, always, create a custom event, in the workflows, the input is a database element, and the output is the modification of that element, that custom event is isolated from the view components, (elements), we test that custom events in other pages, we simulate our unit tests in bubble, once those custom events work as expected, we copy those custom events to the pages we want to use them. Finally we connect the visual element to the custom events with the events triggered by buttons or other events.

Also we try to use states in visual elements to change the view of those elements, we create those states trying to have a local copy (stored in the visual element) of a database element, so once we finish to modify the states in the visual elements, finally we copy the values of those states into the database. (or we use binding in some cases)

Having the visual part and the “workflow” part isolated from each other and mixing together in an organized way, is one of our solution that we believe that is suitable to have some quality in our code.

Also using api workflows for some complex operations, and calling them from multiple pages, assure us to have only one failure point in case of failure.

And finally for our end to end test we use this tool
https://preflight.com/

The bubble team should have more initiative into writing some best practices to work with bubble or have a video conference or something.
I know that they have a lot of work to do, and bubble is an excellent tool, it was just a suggestion

Cheers

3 Likes

Thank you for sharing your method and this recommendation, this tool is very useful.

I second preflight too. We make sure we don’t mess up our workflows while updating our web app. Since preflight is no-code as well, it is actually faster to create the tests than the web app pages. It takes 20x less than per page.