Thank you Joeyg. I was able to figure out the connection strings. Thanks for your input.
If anyone else needs it, this is what I used.
mysql://:@<ec2’s Public IPv4 DNS>:3306/
Notes from bubble docs:
If there are special characters in the password, they must be URL-encoded, for instance replacing # with %23 . The rest of the encodings are listed here
On top of this, I had to open an inbound rule in my security group, to allow all connections through. Much more has been written down here.