Geocoding WARNING...lol.... Scary lesson to learn

Im building an app to pull up homeowner information within my city, I loaded up over 250,000 addresses into my bubble app last night under a data type set to “city”… and then… used a repeating group to list all of the addresses i had loaded, and then… set a geo code to view thousands of locations lol.!

I pulled the info from the local city website to test out an idea lol… Little did i know that the google geocoding api should be used with caution and not toyed with because it can cost you a lot of $$$ lol. Within 1 hour i had over 97,000 geocoding requests that i only caught because of an email i received from google lol.

So lesson of the day, Don’t try to load thousands of geographic information and then make repeating groups display them on your app without research lol. If i had let it run all day and not caught it, luckily it only cost $365, screen shot from 7:55 am of google cloud account

30 mins later…

Lol. Lesson learned.

Screen Shot 2021-03-26 at 9.12.26 PM|690x225

4 Likes

Ouch…Yes, I’ve discussed the cost of Google API on the forum in various places before

Here is a post from @shpak.serhiy that helped me understand another more cost effective way to do distance based search as well.

My opinion is Leafy Maps by @vini_brito is one of the top plugins on the marketplace because of the value it provides in terms of savings from Google, and using that in conjunction with the method @shpak.serhiy put me onto using the Ultimate Toolkit Plugin is the most cost effective way to have a ‘map heavy’ application not let you throw all your profits into the pockets of Google.

Thank you for the info^ Yeah the ideal situation would be to make a map that isnt connected to any apis and plot all the locations to use in a kind of “offline” mode. The trick i think is figuring out how to get the map to sync with the current users location in real time maybe as a layer or so. There is def a better way to do it i’m 100% sure of that. Ill look into those articles

If you have 100 users and each “user” needs to be able to access 500,000 different data sets at any given time its got to be static right?

How do you create 50 million static maps? I think it is that the user needs to access the data dynamically.

Did you ever figure anything out? I have an app for finding skateparks. My maps don’t do anything dynamic on the app, and updates are handled internally. The map is really just displaying a list of markers, and I don’t even need to have it recognize the user’s location. User defines a region before the map populates, and then the map calls all the locations within that region.

@boston85719 I’ll check Leafy maps. Thanks for the tip.

Reducing the max number of results displayed in a geo search box would reduce the amount of calls to google’s api (since less entries are being displayed)?

Well i was originally playing around with a lot of different things to try and achieve what you see in this image. I can zoom in on a map anywhere and pull up all the local data for every home / sale etc in the area. The experimenting to do it is how i ended up getting slammed with geocoding charges. I tried loading in hundreds of thousands of public records on a google sheet and i didnt know that google has a reverse geocode feature that will load addresses from a sheet but it costs a lot if you’re loading them like i am in this photo. I tried using the mapbox plugin however the plugin could not load this many points of data without crashing,

1 Like

You can see in the right side of the picture i am showing the performance waterfall of downloading records, i had to use firebase as a backend to pump data quickly into the map (theres over 2 million houses i can load based on zoom level in my state. Looking back though, If youre trying to accomplish something like this theres ways to do is by using real estate api’s however you don’t own the data.

To answer your question also, Yes. After spending much time i found that using the actual Mapbox service allows to accomplish this with a few tweaks. Go to mapbox.com and make an account, they have a no code map builder that you can actually customize pretty nicely. All of the dots you see from my image are a single svg image that i uploaded into mapbox’s builder. It is best to get the latitude and longitude coordinates of the places you want to pin also, you can use addresses, but keep in mind that most mapping systems prefer coordinates if you want to expand on the data you have and avoid geocoding charges. Lastly you can make the icons interactive such as i did, mapbox has what they call “formulas” that you can pre-set on each location to show an image or change color based on the value of a state you create similar to bubble but for maps. In order to get this map to show in bubble, You use an HTML element, drop it on the page, and copy and paste the Mapbox JS code from their site and place it into your element & it will show. You’ll need to download a code editor if you really want to tweak it though, something like atom is what i used (i had 0 coding experience) i only did it to tweak the map which was a fun experiment as mapbox provides lots of documentation and demos showing the full code to do whatever you like.

In order to get bubbles page to talk to the map, You can use the “Toolbox Plugin”, And use a method called window.parent (you add a small line of code inside the mapbox code placed into the html element on your page that allows the map to talk to bubble elements) if you google the term “Javascript - window.parent method” theres tons of documentation on it, You just have to have a little patience because it takes some toying around haha.

1 Like


Heres a photo of a popup when clicking a building from mapbox js put into bubble as an example of what you can do. Clicking the buttons in the popup changes the states of the markers in mapbox.

You can use the Mapbox Plugin to do quite a bit depending on your use case (if your loading like 50-100 points on a map it works like a charm.

Another side note if your looking to give directions to people using buttons, I found that using links to open native apps is way smoother than trying to do it within the same map. Not to cut corners by any means, but for practicality, and there are some pretty powerful tools with deeplinks that i was totally unaware were possible, such as making a button that links the directions from the users current location to the destination can be done with a simple line of code.

Heres an example (And you can get pretty creative with this on how you can customize buttons with the link etc) check below.
You can switch out Maps.google with Maps.Apple and open this natively in an iphone too…

This is a link to get directions to an address: saddr= is the “start” “address” daddr= is the “destination” “address”

https://maps.google.com/?saddr=535+Legget+Drive+Kanata+Ontario&daddr=Parliament+Hill+Ottawa

You can make a form a user fills out to get an address, and use regex to create ways that save the addresses in these formats which you can attach to buttons on your page.

This is a standard location search link:
http://maps.google.com/?q=100+Main+Street+Buffalo+NY

Just a cool tip.

1 Like

Yes but id depends on how you are pulling the data. I didn’t realize when i started working with maps that people actually go to school full time for cartography lol i had no clue it was a thing.
There are various ways you can pull data to display with a map to consider.

  • One way is by using the BBOX or Bounding Box that grabs all of the data within the viewport basically of a screen.

  • Another way is by using your geo location and calculating it against the bounding box by an area. Theres a formula called the Haversine theory that is used in calculating the radius of the earth to use with pulling data. You can put it into a Javascript formula like this:

function haversine_distance(pos1, pos2) {
var R = 3958.8; // Radius of the Earth in miles
var rlat1 = pos1.lat * (Math.PI / 180); // Convert degrees to radians
var rlat2 = pos2.lat * (Math.PI / 180); // Convert degrees to radians
var difflat = rlat2 - rlat1; // Radian difference (latitudes)
var difflon = (pos2.lng - pos1.lng) * (Math.PI / 180); // Radian difference (longitudes)

var d = 2 * R * Math.asin(Math.sqrt(Math.sin(difflat / 2) * Math.sin(difflat / 2) + Math.cos(rlat1) * Math.cos(rlat2) * Math.sin(difflon / 2) * Math.sin(difflon / 2)));
return d;

}

Then you want to load a list of places which you would use something like this
const loadList = async (flag) => {
if (document.getElementById(‘myDisplayList’) etc…

Think of Latitude and Longitude like this for calculations…If you take the first (3) digits of a latitude and first (3) digits of a longitude, you will get an area the size of a small city on a map. The first 4 digits is a big neighborhood, 5 digits is a house, 6 digits is a car, and 7+ digits is what the military uses to do air strikes lol.

In short, You can do searches based on the first “x” digits with lat and lng to pull area data. Thats what i did, and then start tweaking it from there. You can use bubble to do this fairly easy if your not doing too big of lists, If your doing really big lists, youll need a database, im using firebase (you’ll need to use Geohashing which is a specific way google makes coordinates into a hash for firebase for that) Or you can use Mongo DB which has built in geohashing out of the box and is very fast also.

Pulling data like this, You do not want to use addresses, you need coordinates for the reasons mentioned above ^ With only having addresses, Google is going to turn that address into coordinates for you to map it either way which takes a lot longer and costs more to process.

Hope that gives you some things to ponder ^

1 Like

Thanks, this is great info to have!