Tile server status

Lynn Deffenbaugh

Wow, what a difference a software upgrade makes!  My tile server (http://ldeffenb.dnsalias.net:14171/hot/%z/%x/%y.png) has been struggling recently to keep up with the OSM planet-wide updates and getting them applied to my local DB.  At times, osm2pgsql would spike to 40+GB of RAM consumption while applying an update which dropped the throughput dramatically as Ubuntu couldn't use the RAM as disk cache any more.

Well, I knew there was recent development going on with osm2pgsql adding a new "flex" output to it, but I've only been casually monitoring it.  So I took a closer look the other day and found out that there was a FIX (November 29, 2019 https://github.com/openstreetmap/osm2pgsql/releases/tag/1.2.1) for the RAM consumption issue!  I applied that fix and now instead of taking 46 to 54+ hours to apply 48 hours worth of updates, it's now only taking 24-25 hours for 48 hours of updates!  SWEET!  And I forced the first update run with the new version to pick up on an update set that I knew triggered the bug and it is definitely gone.

Here's the "lag" graph over the past week.  It's pretty obvious when I applied the updated version, wouldn't you say?

Data import lag

Bottom line is that my tile server won't really get a lot faster (except for not being busy just trying to keep current), but at least the tiles it generates and serves will be more current.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

PS.  I have been further behind, and was really catching up until someone started tweaking the relations that were triggering the bug.  Here's the year graph of my lag.

yearly graph

And here's the Memory usage showing the issue.  As you can imagine, when all cache becomes application RAM and the swapper gets into the action, things get REAL SLOW on the tile server!
monthly graphmonthly graph

Maybe now that the update issue is cured, I can fire up the blockchain and see about using IPFS or swarm (likely the former) for meta-tile distribution, access, and archive.  See https://github.com/ethersphere/swarm/wiki/IPFS-&-SWARM or have some fun with google...