MBTiles TileSet storage


Lynn Deffenbaugh
 

Here's an old, but nearly accurate description of firing up an MBTiles file for database storage.  It does not describe how to configure an MBTiles file for https:// access but relies on the server in the legacy tile set configuration block.

And the URL that it mentions is http://sqlitebrowser.org but I didn't notice it in that writeup.

More details to follow, but this is the posting that I thought was around around somewhere!

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

-------- Forwarded Message --------
Subject: [aprsisce] MBTiles TileSet storage
Date: Thu, 1 Feb 2018 17:37:08 -0500
From: 'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce] <aprsisce@...>
Reply-To: aprsisce@...
To: APRSISCE Group <aprsisce@...>


So, you're wondering about the <MBTiles> and <OSM.MBTiles> elements are doing in the APRSIS32.XML file? They are the beginnings of support for using MBTiles (SQLite) databases to store the map tile images instead of directory trees of individual files.  You can set this up yourself if you'd like to test it with the following procedure.

0) Make a copy of your working APRSIS32 instance directory so that if you bork it up, you still have a good working copy!

1) Download an empty MBTiles database from http://ldeffenb.dnsalias.net/MBTiles/* - It's called Empty.mbtiles and is only 4,096 bytes.

2) Copy that file to a new name appropriate to the tile set that you'll be placing into it.  It is easiest to put it in the default execution directory alongside your APRSIS32.XML file.

3) Edit your APRSIS32.XML file and locate the actual <TileServer> that you want to test with.  If you want, you can copy an entire <TileServer>...</TileServer> block to a new entry and change the Name value within the <TileServer Name="xxxx"> element.

4) In that block, add the name that you called the copy of Empty.mbtiles, including the .mbtiles extension and the full path unless it is in your default execution directory.  Yes, with full paths multiple instances of APRSIS32 can share a single .mbtiles file.  It is important that each instance is drawing from the same tile server or thing can get really confusing really quickly.

5) If you are testing with the currently selected TileSet, then you also need to put the same value in the <OSM.MBTiles> element.  Or you can just fire up APRSIS32, switch to a different tile set, and then switch back to the one that is using the <MBTiles> entry.

6) Done.

But what does it actually DO and how can I tell if it is working?  Well, that's a bit trickier.

The easiest way I now of is to make a new copy of a <TileServer> entry and point its <Path> to a newly-created empty directory and specify the <MBTiles> entry as well pointing to a suitably-named copy of the Empty.mbtiles file.  Then when you select that new TileServer in a running APRSIS32, you should see the map tiles get downloaded, but nothing should appear in the directory tree.  And the .mbtiles file should be growing in size as new tiles are downloaded. Zooming and panning around to fetch lots of tiles should show the .mbtiles file continuing to grow.

Then, when you want to see what you've got, open the new .mbtiles file up in the SQLite database browser mentioned earlier and execute the following SQL command:

SELECT zoom_level, count(*), min(tile_column), max(tile_column), min(tile_row), max(tile_row) from tiles group by zoom_level order by zoom_level;

Or you can just restart APRSIS32 and look in the sqlite trace log to see if it has any zoom level ranges in the .mbtiles file.  That output will look something like:

WinMain:2018-02-01T21:44:21.183 Test.mbtiles z[1] 2 Tiles x:1-1 y:0-1
WinMain:2018-02-01T21:44:21.183 Test.mbtiles z[2] 2 Tiles x:3-3 y:1-2
WinMain:2018-02-01T21:44:21.183 Test.mbtiles z[3] 3 Tiles x:7-7 y:2-4
WinMain:2018-02-01T21:44:21.183 Test.mbtiles z[4] 20 Tiles x:0-15 y:1-8
WinMain:2018-02-01T21:44:21.184 Test.mbtiles z[5] 60 Tiles x:0-13 y:6-15
WinMain:2018-02-01T21:44:21.184 Test.mbtiles z[6] 206 Tiles x:0-26 y:14-32
WinMain:2018-02-01T21:44:21.185 Test.mbtiles z[7] 137 Tiles x:16-37 y:41-54
WinMain:2018-02-01T21:44:21.185 Test.mbtiles z[8] 360 Tiles x:36-78 y:86-108
WinMain:2018-02-01T21:44:21.187 Test.mbtiles z[9] 535 Tiles x:77-153 y:176-217
WinMain:2018-02-01T21:44:21.188 Test.mbtiles z[10] 591 Tiles x:158-305 y:355-431
WinMain:2018-02-01T21:44:21.190 Test.mbtiles z[11] 498 Tiles x:323-608 y:750-860
WinMain:2018-02-01T21:44:21.191 Test.mbtiles z[12] 494 Tiles x:652-1213 y:1502-1718
WinMain:2018-02-01T21:44:21.192 Test.mbtiles z[13] 426 Tiles x:1564-2423 y:3008-3434
WinMain:2018-02-01T21:44:21.193 Test.mbtiles z[14] 138 Tiles x:4197-4844 y:6018-6866
WinMain:2018-02-01T21:44:21.193 Test.mbtiles z[15] 123 Tiles x:8333-9684 y:11516-12816
WinMain:2018-02-01T21:44:21.200 Test.mbtiles z[16] 73 Tiles x:16771-16781 y:25623-25629

This says that my Test.mbtiles file has tiles from zooms 1 through 16 with a concentration of them (300-500/level) in zooms 8 through 13.  The x and y ranges show the actual OSM tile numbers that are cached in that level.  Note that this is only the min and max, it does not mean that every tile in the entire x/y rectangle is actually present in the file.

If any of you are brave enough to give this a shot, let me know how it goes or especially if it goes bust!

And as with the overlays, I'll be incorporating this into the Configure / Map / TileSets / New... user interface.  If you use the SQLite database browser and look at the contents of the "metadata" table, you'll notice that it includes TileServer definition information.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

PS.  Note that not all .mbtiles files are compatible with APRSIS32 tile storage.  Some .mbtiles files have been optimized to remove redundant tile images.  These .mbtiles files can be used as overlay tilesets, but not as main tile storage as their internal table format is different.  As long as the SQLite database browser shows only 2 tables called "metadata" and "tiles", it should be good.

For instance, MBTiles files from mapbox have the "tiles" as a view rather than an explicit table.  These files can be used as overlay MBTiles, but not as TileSet .mbtiles storage.

https://www.mapbox.com/help/define-mbtiles/

and of course, vector tiles are something completely different!

https://openmaptiles.com/downloads/planet/

Notice that the satellite views .mbtiles file (available for pay) is 196GB in size.  Yes, 196 GIGAbytes!




------------------------------------
Posted by: "Lynn W Deffenbaugh (Mr)" <KJ4ERJ@...>
------------------------------------


------------------------------------

Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/aprsisce/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/aprsisce/join
(Yahoo! ID required)

<*> To change settings via email:
aprsisce-digest@... aprsisce-fullfeatured@...

<*> To unsubscribe from this group, send an email to:
aprsisce-unsubscribe@...

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/


Join APRSISCE@groups.io to automatically receive all group messages.