Re: MBTiles?

Greg D
 

Since I run APRSIS32 under Wine on my Linux desktop, I wonder what is the efficiency of the Tile storage with the EXT4 file system?

If EXT4 is efficient, and if Android can use EXT4 file systems on uSD cards, perhaps the easy answer is just to re-format the uSD card to EXT4 before using it? 

Greg  KO6TH


Rob Giuliano kb8rco@... [aprsisce] wrote:

 
The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.

Therefore, I like the idea of a database style, single file map interface.
 
Robert Giuliano
KB8RCO





From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]"
To: APRSISCE Group ; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?

 
Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32




Join APRSISCE@groups.io to automatically receive all group messages.