Date   
Re: Box w/Cross is What?

Fred Hillhouse
 

For ambiguity is one reason.

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Monday, April 03, 2017 4:23 AM
To: aprsisce@...
Subject: [aprsisce] Re: Box w/Cross is What?

 

 

Unfortunately not...  I take it the "house" is not actually where depicted but somewhere within the grid square? I know some devices, like a WSPRlite can only transmit a grid square by digital format, but why might an APRS user do so?



Re: Box w/Cross is What?

bjessee@...
 

Unfortunately not...  I take it the "house" is not actually where depicted but somewhere within the grid square? I know some devices, like a WSPRlite can only transmit a grid square by digital format, but why might an APRS user do so?

Re: Box w/Cross is What?

Lynn Deffenbaugh
 

Fred is correct about the grid square.  Did you happen to get the callsign-SSID of the house?  We could look at the raw packet history at aprs.fi to confirm that the station was transmitting both grid squares and possibly posits which would subsequently trump the approximate location provided by the grid square.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 4/2/2017 4:32 PM, fmhillhouse fmhillhouse@... [aprsisce] wrote:
That is the center of a grid square I believe. 



Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone


-------- Original message --------
From: "bjessee@... [aprsisce]"
Date: 4/2/17 14:12 (GMT-05:00)
To: aprsisce@...
Subject: [aprsisce] Box w/Cross is What?

 

Today I saw a House symbol in the center of a rectangle with a cross, the house symbol in the center inside a circle as shown here - what is it? I've not seen a house symbol at that location near me before and it was gone after a while.
[URL=http://s29.photobucket.com/user/cbjesseeNH/media/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg.html][IMG]http://i29.photobucket.com/albums/c290/cbjesseeNH/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg[/IMG][/URL]


Re: Box w/Cross is What?

Fred Hillhouse
 

That is the center of a grid square I believe. 



Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone


-------- Original message --------
From: "bjessee@... [aprsisce]" <aprsisce@...>
Date: 4/2/17 14:12 (GMT-05:00)
To: aprsisce@...
Subject: [aprsisce] Box w/Cross is What?

 

Today I saw a House symbol in the center of a rectangle with a cross, the house symbol in the center inside a circle as shown here - what is it? I've not seen a house symbol at that location near me before and it was gone after a while.
[URL=http://s29.photobucket.com/user/cbjesseeNH/media/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg.html][IMG]http://i29.photobucket.com/albums/c290/cbjesseeNH/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg[/IMG][/URL]

Box w/Cross is What?

bjessee@...
 

Today I saw a House symbol in the center of a rectangle with a cross, the house symbol in the center inside a circle as shown here - what is it? I've not seen a house symbol at that location near me before and it was gone after a while.
[URL=http://s29.photobucket.com/user/cbjesseeNH/media/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg.html][IMG]http://i29.photobucket.com/albums/c290/cbjesseeNH/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg[/IMG][/URL]

Re: MBTiles?

Greg D
 

Since I run APRSIS32 under Wine on my Linux desktop, I wonder what is the efficiency of the Tile storage with the EXT4 file system?

If EXT4 is efficient, and if Android can use EXT4 file systems on uSD cards, perhaps the easy answer is just to re-format the uSD card to EXT4 before using it? 

Greg  KO6TH


Rob Giuliano kb8rco@... [aprsisce] wrote:

 
The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.

Therefore, I like the idea of a database style, single file map interface.
 
Robert Giuliano
KB8RCO





From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]"
To: APRSISCE Group ; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?

 
Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32




Re: MBTiles?

Greg D
 

There are techniques to extend the lifetime of SSDs under Linux.  I don't know what of these techniques are used by or applicable to Android (which is Linux at the core).  For your laptops, I recommend this write-up:

https://sites.google.com/site/easylinuxtipsproject/ssd

The keys are noatime and discard / TRIM.  Lacking a spinning place for Swap, add more RAM to remove or lessen the need for it.

Greg.

Rob Giuliano @KB8RCO [aprsisce] wrote:



Solid State drives (including SDs and thumb drives) have always had a
read/write limitation concern.  Most list something in the range of 10,000
to 100,000 read/write cycles (or at least that was an old range).


When we got new laptops (Lenovos) at work, they came with solid state
drives (great for battery life!).  We were concerned about the life
expectancy.  We compile software which creates lots of temporary files. 
We also do a lot of testing which generates a lot of data files.


Taking 50,000 as an average and 200 work days a year, that is only 250
read/writes per year.  Of course that is for a given "area" of the device.
BUT, since the drives are only 250 GB (vs. the old spinning drives which
were 512 - 1000 MB), our laptop drives are typically very full.  That
means the small space that is available is seeing all the "read/writes
cycles".


These newer drives are obviously going beyond that.  I have many coworkers
with their laptops at > 95% capacity for at least 3 years now, and haven't
heard of any crashing yet.


I suspect newer USB drives and uSD cards are in that same category (or at
least hope so!)

Robert Giuliano
KB8RCO



*From:* "'Fred Hillhouse' fmhillhouse@... [aprsisce]" (
aprsisce@... )
*To:* aprsisce@...
*Sent:* Friday, March 31, 2017 12:44 PM
*Subject:* RE: [aprsisce] MBTiles?


I used a 120GB USB hard drive for 5 years or so in the heat and cold. So
far it has been the best storage medium. It has been the most reliable
drive and it is still working. It does stay at home now after paying its
dues.

My 128GB uSD lasted three months which came to a $1/day usage fee before
failure. My 64 GB uSD has last 2 years now.

Most of the hard drive failures still allowed access to the majority of
files while all of the uSD cards failed completely. My USB flash drives
fail the same as the uSD.

I guess the real point of all this is back up your data. Losing a card on
the phone usually means you lost all media (pictures, video) as well. I
have as many apps as possible save the data to my uSD card.

Best regards,
Fred N7FMH


*From:* aprsisce@... [ mailto:aprsisce@... (
aprsisce@... ) ]
*Sent:* Friday, March 31, 2017 9:34 AM
*To:* aprsisce@...
*Subject:* Re: [aprsisce] MBTiles?


Having just replaced a failing uSD card in my phone (3 years-ish old), the
failure mode was that the card become non-writable.   When Android tried
to write to it, the card spontaneously dismounted causing ALL of my map
tiles to disappear (I was making changes to APRSISMO at the time and
really thought I had borked everything until I found the dismounted SD
card).

I'm suspecting that the single file really isn't any more prone to failure
modes than the file system on the card containing lots of tiles.  In
either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the
active one if it is enabled for writing.  (Yes, I'm thinking to provide
for Read-Only MBTiles tile sets that never fetch tiles, but only display
what you already have in the file.)  (And yes, I'm also considering using
an MBTiles file for what's available within it and using other TileSets
for displaying other areas.)  (And yes, I just put the code in APRSIS32
that stores freshly downloaded tiles inside an MBTile database.)  (And
yes, this is the last parenthetical!).

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32


On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... [aprsisce]
wrote:

I am thinking more along the mobile/portable application where vibration
and temperature variation might cause a failure and there is no access to
the back up since it is home. This is not unsurmountable since uSD cards
are small.

Best regards,
Fred N7FMH




*From:* aprsisce@... [ mailto:aprsisce@... (
aprsisce@... ) ]
*Sent:* Thursday, March 30, 2017 7:12 PM
*To:* aprsisce@...
*Subject:* RE: [aprsisce] MBTiles?


Yes, but even if you keep a bankup (like the -safe xml) the space could be
smaller than the separate tile files.  At least on FAT32 disks.
Robert Giuliano
KB8RCO



My only real concern is if the single file becomes corrupt then the whole
thing is lost. Whereas in the other case, only a single tile may be gone.

It could be argued that there are fewer disk accesses so there would be
less chance of failure.

Best regards,
Fred N7FMH



*From:* aprsisce@... [ mailto:aprsisce@... (
aprsisce@... ) ]
*Sent:* Wednesday, March 29, 2017 11:26 PM
*To:* aprsisce@...
*Subject:* Re: [aprsisce] MBTiles?


The real issue here is that we still format uSD (thumb drives, etc) in a
vey inefficient FAT32 format.  Place the same files on NTFS and the two
numbers (actual vs. on disk) match much closer.  Unfortunately it is the
most "cross platform" file system out there.

Therefore, I like the idea of a database style, single file map interface.


Robert Giuliano
KB8RCO



*From:* "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" < aprsisce@...
*To:* APRSISCE Group < aprsisce@... >; APRSISMO@...

*Sent:* Wednesday, March 29, 2017 3:51 PM
*Subject:* [aprsisce] MBTiles?


Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate

Re: MBTiles?

Rob Giuliano
 

Interestingly, I grabbed a completely empty 64GB thumb drive and only placed the OSMTiles on it.
I experimented with formatting, dragging and dropping the same directory as the only thing on the drive after each format:

Formatted as Fat32:    
   95.4 MB, size on disk 581 MB, Disk Properties disk space used: 316 MB
Format exFAT  (4k Allocations)
   95.4 MB, size on disk 581 MB, Disk Properties disk space used:
Format exFAT  (16k Allocations)
   95.4 MB, size on disk 581 MB, Disk Properties disk space used:

Format NTFS  (4k Allocations)
   95.4 MB, size on disk 581 MB, Disk Properties disk space used: 245 MB
Format NTFS  (256b Allocations)
   95.4 MB, size on disk 581 MB, Disk Properties disk space used: 229 MB

It appears the Windows directory properties reporting system is reporting differently than the actual used disk space.

On my Windows 7 laptop's hard drive (SSD) it reports
   95.4 MB, size on disk 136 MB, Disk Properties disk space used:

Robert Giuliano
KB8RCO



From: "'W. Lim' wellmanl@... [aprsisce]"
To: "aprsisce@..."
Sent: Friday, March 31, 2017 1:20 PM
Subject: [aprsisce] Re: MBTiles?

 
Note that if the SD card is 8GB or less the cluster size is 4kb .
If over 8, then the cluster size is 32 kb. So if you moved your
tiles to an 8GB card, it might just fit.


Re: MBTiles?

Rob Giuliano
 

Solid State drives (including SDs and thumb drives) have always had a read/write limitation concern.  Most list something in the range of 10,000 to 100,000 read/write cycles (or at least that was an old range).

When we got new laptops (Lenovos) at work, they came with solid state drives (great for battery life!).  We were concerned about the life expectancy.  We compile software which creates lots of temporary files.  We also do a lot of testing which generates a lot of data files.

Taking 50,000 as an average and 200 work days a year, that is only 250 read/writes per year.  Of course that is for a given "area" of the device.  BUT, since the drives are only 250 GB (vs. the old spinning drives which were 512 - 1000 MB), our laptop drives are typically very full.  That means the small space that is available is seeing all the "read/writes cycles".

These newer drives are obviously going beyond that.  I have many coworkers with their laptops at > 95% capacity for at least 3 years now, and haven't heard of any crashing yet. 

I suspect newer USB drives and uSD cards are in that same category (or at least hope so!)
 
Robert Giuliano
KB8RCO



From: "'Fred Hillhouse' fmhillhouse@... [aprsisce]"
To: aprsisce@...
Sent: Friday, March 31, 2017 12:44 PM
Subject: RE: [aprsisce] MBTiles?

 
I used a 120GB USB hard drive for 5 years or so in the heat and cold. So far it has been the best storage medium. It has been the most reliable drive and it is still working. It does stay at home now after paying its dues.
 
My 128GB uSD lasted three months which came to a $1/day usage fee before failure. My 64 GB uSD has last 2 years now.
 
Most of the hard drive failures still allowed access to the majority of files while all of the uSD cards failed completely. My USB flash drives fail the same as the uSD.
 
I guess the real point of all this is back up your data. Losing a card on the phone usually means you lost all media (pictures, video) as well. I have as many apps as possible save the data to my uSD card.
 
Best regards,
Fred N7FMH
 
 
From: aprsisce@... [mailto:aprsisce@...]
Sent: Friday, March 31, 2017 9:34 AM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?
 
 
Having just replaced a failing uSD card in my phone (3 years-ish old), the failure mode was that the card become non-writable.   When Android tried to write to it, the card spontaneously dismounted causing ALL of my map tiles to disappear (I was making changes to APRSISMO at the time and really thought I had borked everything until I found the dismounted SD card).

I'm suspecting that the single file really isn't any more prone to failure modes than the file system on the card containing lots of tiles.  In either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the active one if it is enabled for writing.  (Yes, I'm thinking to provide for Read-Only MBTiles tile sets that never fetch tiles, but only display what you already have in the file.)  (And yes, I'm also considering using an MBTiles file for what's available within it and using other TileSets for displaying other areas.)  (And yes, I just put the code in APRSIS32 that stores freshly downloaded tiles inside an MBTile database.)  (And yes, this is the last parenthetical!).

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32


On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... [aprsisce] wrote:
I am thinking more along the mobile/portable application where vibration and temperature variation might cause a failure and there is no access to the back up since it is home. This is not unsurmountable since uSD cards are small.
 
Best regards,
Fred N7FMH
 
 
 
 
From: aprsisce@... [mailto:aprsisce@...]
Sent: Thursday, March 30, 2017 7:12 PM
To: aprsisce@...
Subject: RE: [aprsisce] MBTiles?
 
 
Yes, but even if you keep a bankup (like the -safe xml) the space could be smaller than the separate tile files.  At least on FAT32 disks.
Robert Giuliano
KB8RCO
 
 
My only real concern is if the single file becomes corrupt then the whole thing is lost. Whereas in the other case, only a single tile may be gone.
 
It could be argued that there are fewer disk accesses so there would be less chance of failure.
 
Best regards,
Fred N7FMH
 
 
 
From: aprsisce@... [mailto:aprsisce@...]
Sent: Wednesday, March 29, 2017 11:26 PM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?
 
 
The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.
 
Therefore, I like the idea of a database style, single file map interface.
 
Robert Giuliano
KB8RCO
 
 

From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: APRSISCE Group <aprsisce@...>; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?
 
 
Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32
 
 

Avast logo
This email has been checked for viruses by Avast antivirus software.
www.avast.com
 
 

Avast logo
This email has been checked for viruses by Avast antivirus software.
www.avast.com
 
 



Avast logo
This email has been checked for viruses by Avast antivirus software.
www.avast.com



Re: MBTiles?

w.lim
 

Note that if the SD card is 8GB or less the cluster size is 4kb .
If over 8, then the cluster size is 32 kb. So if you moved your
tiles to an 8GB card, it might just fit.

Re: MBTiles?

Fred Hillhouse
 

I used a 120GB USB hard drive for 5 years or so in the heat and cold. So far it has been the best storage medium. It has been the most reliable drive and it is still working. It does stay at home now after paying its dues.

 

My 128GB uSD lasted three months which came to a $1/day usage fee before failure. My 64 GB uSD has last 2 years now.

 

Most of the hard drive failures still allowed access to the majority of files while all of the uSD cards failed completely. My USB flash drives fail the same as the uSD.

 

I guess the real point of all this is back up your data. Losing a card on the phone usually means you lost all media (pictures, video) as well. I have as many apps as possible save the data to my uSD card.

 

Best regards,

Fred N7FMH

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Friday, March 31, 2017 9:34 AM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?

 

 

Having just replaced a failing uSD card in my phone (3 years-ish old), the failure mode was that the card become non-writable.   When Android tried to write to it, the card spontaneously dismounted causing ALL of my map tiles to disappear (I was making changes to APRSISMO at the time and really thought I had borked everything until I found the dismounted SD card).

I'm suspecting that the single file really isn't any more prone to failure modes than the file system on the card containing lots of tiles.  In either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the active one if it is enabled for writing.  (Yes, I'm thinking to provide for Read-Only MBTiles tile sets that never fetch tiles, but only display what you already have in the file.)  (And yes, I'm also considering using an MBTiles file for what's available within it and using other TileSets for displaying other areas.)  (And yes, I just put the code in APRSIS32 that stores freshly downloaded tiles inside an MBTile database.)  (And yes, this is the last parenthetical!).


Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... [aprsisce] wrote:

I am thinking more along the mobile/portable application where vibration and temperature variation might cause a failure and there is no access to the back up since it is home. This is not unsurmountable since uSD cards are small.

 

Best regards,

Fred N7FMH

 

 

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Thursday, March 30, 2017 7:12 PM
To: aprsisce@...
Subject: RE: [aprsisce] MBTiles?

 

 

Yes, but even if you keep a bankup (like the -safe xml) the space could be smaller than the separate tile files.  At least on FAT32 disks.

Robert Giuliano

KB8RCO

 

 

My only real concern is if the single file becomes corrupt then the whole thing is lost. Whereas in the other case, only a single tile may be gone.

 

It could be argued that there are fewer disk accesses so there would be less chance of failure.

 

Best regards,

Fred N7FMH

 

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Wednesday, March 29, 2017 11:26 PM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?

 

 

The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.

 

Therefore, I like the idea of a database style, single file map interface.

 

Robert Giuliano

KB8RCO

 

 


From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: APRSISCE Group <aprsisce@...>; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?

 

 

Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

 

 


Avast logo

This email has been checked for viruses by Avast antivirus software.
www.avast.com

 

 


Avast logo

This email has been checked for viruses by Avast antivirus software.
www.avast.com

 

 



Re: Tiles

Fred Hillhouse
 

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Friday, March 31, 2017 10:20 AM
To: aprsisce@...
Subject: [aprsisce] Tiles

 

 

I know that if you turn on the nopurge, that it will not purge.  If it is off,
what is the time that it will automatically purge. Also, how does it decide to
get a new tile. 

If the PURGE is OFF, it will not purge or update.

I guess I am asking if I "touch" my map files, can I get a
condition where it will update the tiles I am currently looking at. At present, I have
two directories with map tiles and manually move between to update them.

That sounds about right.

For my desktop, I turn on PURGE and set the time to a few weeks for tiles (e.g, OSM) that update.

For the case of the none updating tiles (e.g, US Topo) I don’t purge. I just collect them as they appear.

Best regards,

Fred N7FMH

 



Re: MBTiles?

Rob Giuliano
 

My phone doesn't have any external storage (uSD).
User backup is really the only option.
I have a SanDisk dual port SUB drive (uB and A).  I have my phone backups (including map tiles) on that.  I bought it and started carrying it on my vacation.  Too many pictures and such to store on a phone with no external storage.  With the extra room, I now have old family pictures to show as well (on family vacations).
Robert Giuliano KB8RCO

From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: aprsisce@...
Sent: Friday, March 31, 2017 10:20 AM
Subject: Re: [aprsisce] MBTiles?

  I tried compression on an MBTiles file and it's not really helpful.  Remember that most of the space inside the SQLite database is going to be the PNG Blobs which are not very compressible.

I like the idea of keeping copies in two different spaces, but on too many phones it's not obvious to the program which path goes where.  I'm leaning toward just having APRSISCE/32/MO maintain MBTiles files and leave the copying/redundancy protection up to the user and his/her favorite file manager application.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 3/31/2017 10:12 AM, Rob Giuliano @KB8RCO [aprsisce] wrote:

(And yes . . . )
The longevity of such devices varies. I have a uSD from a very old (Samsung Galaxy Player 4.0).  I used that fro several years and moved files around (music, pictures, and app data).  After at least 5 years in that device, I now use it in a Raspberry Pi Zero.  Since it is an old uSD (and I expect it to fail soon), I use it to experiment with and it gets "wiped and reloaded" with different Pi distros very often.  Still going strong!
For a phone setup (although some may have limited space), it might be worth investigating Phone space vs. uSD.  Backup on one, active on other.  Any options to easily compress the backup?   Robert Giuliano KB8RCO

From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: aprsisce@...
Sent: Friday, March 31, 2017 9:38 AM
Subject: Re: [aprsisce] MBTiles?

  Having just replaced a failing uSD card in my phone (3 years-ish old), the failure mode was that the card become non-writable.   When Android tried to write to it, the card spontaneously dismounted causing ALL of my map tiles to disappear (I was making changes to APRSISMO at the time and really thought I had borked everything until I found the dismounted SD card).

I'm suspecting that the single file really isn't any more prone to failure modes than the file system on the card containing lots of tiles.  In either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the active one if it is enabled for writing.  (Yes, I'm thinking to provide for Read-Only MBTiles tile sets that never fetch tiles, but only display what you already have in the file.)  (And yes, I'm also considering using an MBTiles file for what's available within it and using other TileSets for displaying other areas.)  (And yes, I just put the code in APRSIS32 that stores freshly downloaded tiles inside an MBTile database.)  (And yes, this is the last parenthetical!).

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... [aprsisce] wrote:

I am thinking more along the mobile/portable application where vibration and temperature variation might cause a failure and there is no access to the back up since it is home. This is not unsurmountable since uSD cards are small.   Best regards, Fred N7FMH         From: aprsisce@... [mailto:aprsisce@...]
Sent: Thursday, March 30, 2017 7:12 PM
To: aprsisce@...
Subject: RE: [aprsisce] MBTiles?     Yes, but even if you keep a bankup (like the -safe xml) the space could be smaller than the separate tile files.  At least on FAT32 disks. Robert Giuliano KB8RCO
    My only real concern is if the single file becomes corrupt then the whole thing is lost. Whereas in the other case, only a single tile may be gone.   It could be argued that there are fewer disk accesses so there would be less chance of failure.   Best regards, Fred N7FMH       From: aprsisce@... [mailto:aprsisce@...]
Sent: Wednesday, March 29, 2017 11:26 PM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?     The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.   Therefore, I like the idea of a database style, single file map interface.   Robert Giuliano KB8RCO     From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: APRSISCE Group <aprsisce@...>; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?     Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32    
| | This email has been checked for viruses by Avast antivirus software.
www.avast.com |

 



| | This email has been checked for viruses by Avast antivirus software.
www.avast.com |








#yiv4051740290 #yiv4051740290 -- #yiv4051740290ygrp-mkp {border:1px solid #d8d8d8;font-family:Arial;margin:10px 0;padding:0 10px;}#yiv4051740290 #yiv4051740290ygrp-mkp hr {border:1px solid #d8d8d8;}#yiv4051740290 #yiv4051740290ygrp-mkp #yiv4051740290hd {color:#628c2a;font-size:85%;font-weight:700;line-height:122%;margin:10px 0;}#yiv4051740290 #yiv4051740290ygrp-mkp #yiv4051740290ads {margin-bottom:10px;}#yiv4051740290 #yiv4051740290ygrp-mkp .yiv4051740290ad {padding:0 0;}#yiv4051740290 #yiv4051740290ygrp-mkp .yiv4051740290ad p {margin:0;}#yiv4051740290 #yiv4051740290ygrp-mkp .yiv4051740290ad a {color:#0000ff;text-decoration:none;}#yiv4051740290 #yiv4051740290ygrp-sponsor #yiv4051740290ygrp-lc {font-family:Arial;}#yiv4051740290 #yiv4051740290ygrp-sponsor #yiv4051740290ygrp-lc #yiv4051740290hd {margin:10px 0px;font-weight:700;font-size:78%;line-height:122%;}#yiv4051740290 #yiv4051740290ygrp-sponsor #yiv4051740290ygrp-lc .yiv4051740290ad {margin-bottom:10px;padding:0 0;}#yiv4051740290 #yiv4051740290actions {font-family:Verdana;font-size:11px;padding:10px 0;}#yiv4051740290 #yiv4051740290activity {background-color:#e0ecee;float:left;font-family:Verdana;font-size:10px;padding:10px;}#yiv4051740290 #yiv4051740290activity span {font-weight:700;}#yiv4051740290 #yiv4051740290activity span:first-child {text-transform:uppercase;}#yiv4051740290 #yiv4051740290activity span a {color:#5085b6;text-decoration:none;}#yiv4051740290 #yiv4051740290activity span span {color:#ff7900;}#yiv4051740290 #yiv4051740290activity span .yiv4051740290underline {text-decoration:underline;}#yiv4051740290 .yiv4051740290attach {clear:both;display:table;font-family:Arial;font-size:12px;padding:10px 0;width:400px;}#yiv4051740290 .yiv4051740290attach div a {text-decoration:none;}#yiv4051740290 .yiv4051740290attach img {border:none;padding-right:5px;}#yiv4051740290 .yiv4051740290attach label {display:block;margin-bottom:5px;}#yiv4051740290 .yiv4051740290attach label a {text-decoration:none;}#yiv4051740290 blockquote {margin:0 0 0 4px;}#yiv4051740290 .yiv4051740290bold {font-family:Arial;font-size:13px;font-weight:700;}#yiv4051740290 .yiv4051740290bold a {text-decoration:none;}#yiv4051740290 dd.yiv4051740290last p a {font-family:Verdana;font-weight:700;}#yiv4051740290 dd.yiv4051740290last p span {margin-right:10px;font-family:Verdana;font-weight:700;}#yiv4051740290 dd.yiv4051740290last p span.yiv4051740290yshortcuts {margin-right:0;}#yiv4051740290 div.yiv4051740290attach-table div div a {text-decoration:none;}#yiv4051740290 div.yiv4051740290attach-table {width:400px;}#yiv4051740290 div.yiv4051740290file-title a, #yiv4051740290 div.yiv4051740290file-title a:active, #yiv4051740290 div.yiv4051740290file-title a:hover, #yiv4051740290 div.yiv4051740290file-title a:visited {text-decoration:none;}#yiv4051740290 div.yiv4051740290photo-title a, #yiv4051740290 div.yiv4051740290photo-title a:active, #yiv4051740290 div.yiv4051740290photo-title a:hover, #yiv4051740290 div.yiv4051740290photo-title a:visited {text-decoration:none;}#yiv4051740290 div#yiv4051740290ygrp-mlmsg #yiv4051740290ygrp-msg p a span.yiv4051740290yshortcuts {font-family:Verdana;font-size:10px;font-weight:normal;}#yiv4051740290 .yiv4051740290green {color:#628c2a;}#yiv4051740290 .yiv4051740290MsoNormal {margin:0 0 0 0;}#yiv4051740290 o {font-size:0;}#yiv4051740290 #yiv4051740290photos div {float:left;width:72px;}#yiv4051740290 #yiv4051740290photos div div {border:1px solid #666666;height:62px;overflow:hidden;width:62px;}#yiv4051740290 #yiv4051740290photos div label {color:#666666;font-size:10px;overflow:hidden;text-align:center;white-space:nowrap;width:64px;}#yiv4051740290 #yiv4051740290reco-category {font-size:77%;}#yiv4051740290 #yiv4051740290reco-desc {font-size:77%;}#yiv4051740290 .yiv4051740290replbq {margin:4px;}#yiv4051740290 #yiv4051740290ygrp-actbar div a:first-child {margin-right:2px;padding-right:5px;}#yiv4051740290 #yiv4051740290ygrp-mlmsg {font-size:13px;font-family:Arial, helvetica, clean, sans-serif;}#yiv4051740290 #yiv4051740290ygrp-mlmsg table {font-size:inherit;font:100%;}#yiv4051740290 #yiv4051740290ygrp-mlmsg select, #yiv4051740290 input, #yiv4051740290 textarea {font:99% Arial, Helvetica, clean, sans-serif;}#yiv4051740290 #yiv4051740290ygrp-mlmsg pre, #yiv4051740290 code {font:115% monospace;}#yiv4051740290 #yiv4051740290ygrp-mlmsg * {line-height:1.22em;}#yiv4051740290 #yiv4051740290ygrp-mlmsg #yiv4051740290logo {padding-bottom:10px;}#yiv4051740290 #yiv4051740290ygrp-msg p a {font-family:Verdana;}#yiv4051740290 #yiv4051740290ygrp-msg p#yiv4051740290attach-count span {color:#1E66AE;font-weight:700;}#yiv4051740290 #yiv4051740290ygrp-reco #yiv4051740290reco-head {color:#ff7900;font-weight:700;}#yiv4051740290 #yiv4051740290ygrp-reco {margin-bottom:20px;padding:0px;}#yiv4051740290 #yiv4051740290ygrp-sponsor #yiv4051740290ov li a {font-size:130%;text-decoration:none;}#yiv4051740290 #yiv4051740290ygrp-sponsor #yiv4051740290ov li {font-size:77%;list-style-type:square;padding:6px 0;}#yiv4051740290 #yiv4051740290ygrp-sponsor #yiv4051740290ov ul {margin:0;padding:0 0 0 8px;}#yiv4051740290 #yiv4051740290ygrp-text {font-family:Georgia;}#yiv4051740290 #yiv4051740290ygrp-text p {margin:0 0 1em 0;}#yiv4051740290 #yiv4051740290ygrp-text tt {font-size:120%;}#yiv4051740290 #yiv4051740290ygrp-vital ul li:last-child {border-right:none !important;}#yiv4051740290

Tiles

w.lim
 

I know that if you turn on the nopurge, that it will not purge.  If it is off,
what is the time that it will automatically purge. Also, how does it decide to
get a new tile.  I guess I am asking if I "touch" my map files, can I get a
condition where it will update the tiles I am currently looking at. At present, I have
two directories with map tiles and manually move between to update them.



Re: MBTiles?

Lynn Deffenbaugh
 

I tried compression on an MBTiles file and it's not really helpful. Remember that most of the space inside the SQLite database is going to be the PNG Blobs which are not very compressible.

I like the idea of keeping copies in two different spaces, but on too many phones it's not obvious to the program which path goes where. I'm leaning toward just having APRSISCE/32/MO maintain MBTiles files and leave the copying/redundancy protection up to the user and his/her favorite file manager application.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

On 3/31/2017 10:12 AM, Rob Giuliano @KB8RCO [aprsisce] wrote:


(And yes . . . )

The longevity of such devices varies. I have a uSD from a very old (Samsung Galaxy Player 4.0). I used that fro several years and moved files around (music, pictures, and app data). After at least 5 years in that device, I now use it in a Raspberry Pi Zero. Since it is an old uSD (and I expect it to fail soon), I use it to experiment with and it gets "wiped and reloaded" with different Pi distros very often. Still going strong!

For a phone setup (although some may have limited space), it might be worth investigating Phone space vs. uSD. Backup on one, active on other. Any options to easily compress the backup?
Robert Giuliano
KB8RCO


------------------------------------------------------------------------
*From:* "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
*To:* aprsisce@...
*Sent:* Friday, March 31, 2017 9:38 AM
*Subject:* Re: [aprsisce] MBTiles?

Having just replaced a failing uSD card in my phone (3 years-ish old), the failure mode was that the card become non-writable. When Android tried to write to it, the card spontaneously dismounted causing ALL of my map tiles to disappear (I was making changes to APRSISMO at the time and really thought I had borked everything until I found the dismounted SD card).

I'm suspecting that the single file really isn't any more prone to failure modes than the file system on the card containing lots of tiles. In either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the active one if it is enabled for writing. (Yes, I'm thinking to provide for Read-Only MBTiles tile sets that never fetch tiles, but only display what you already have in the file.) (And yes, I'm also considering using an MBTiles file for what's available within it and using other TileSets for displaying other areas.) (And yes, I just put the code in APRSIS32 that stores freshly downloaded tiles inside an MBTile database.) (And yes, this is the last parenthetical!).

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... <mailto:fmhillhouse@...> [aprsisce] wrote:
I am thinking more along the mobile/portable application where vibration and temperature variation might cause a failure and there is no access to the back up since it is home. This is not unsurmountable since uSD cards are small.
Best regards,
Fred N7FMH
*From:*aprsisce@... <mailto:aprsisce@...> [mailto:aprsisce@...]
*Sent:* Thursday, March 30, 2017 7:12 PM
*To:* aprsisce@... <mailto:aprsisce@...>
*Subject:* RE: [aprsisce] MBTiles?
Yes, but even if you keep a bankup (like the -safe xml) the space could be smaller than the separate tile files. At least on FAT32 disks.
Robert Giuliano
KB8RCO

My only real concern is if the single file becomes corrupt then
the whole thing is lost. Whereas in the other case, only a single
tile may be gone.
It could be argued that there are fewer disk accesses so there
would be less chance of failure.
Best regards,
Fred N7FMH
*From:*aprsisce@... <mailto:aprsisce@...>
[mailto:aprsisce@...]
*Sent:* Wednesday, March 29, 2017 11:26 PM
*To:* aprsisce@... <mailto:aprsisce@...>
*Subject:* Re: [aprsisce] MBTiles?
The real issue here is that we still format uSD (thumb drives,
etc) in a vey inefficient FAT32 format. Place the same files on
NTFS and the two numbers (actual vs. on disk) match much closer.
Unfortunately it is the most "cross platform" file system out there.
Therefore, I like the idea of a database style, single file map
interface.
Robert Giuliano
KB8RCO
------------------------------------------------------------------------
*From:*"'Lynn W Deffenbaugh (Mr)' kj4erj@...
<mailto:kj4erj@...> [aprsisce]" <aprsisce@...
<mailto:aprsisce@...>>
*To:* APRSISCE Group <aprsisce@...
<mailto:aprsisce@...>>; APRSISMO@...
<mailto:APRSISMO@...>
*Sent:* Wednesday, March 29, 2017 3:51 PM
*Subject:* [aprsisce] MBTiles?
Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off
of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB,
not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on
disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an
uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called
"meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a
single
meta-tile file with a hashed name that serves to distribute the
files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct
image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely
non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and
transport.
It amounts to an SQLite database storing blobs of image data
inside a
single file rather than the current directory tree of individual
files.
There are tools that can generate MBTiles-formatted files from
various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of
Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's
internal
renderer from an Overpass downloaded data file with the web map
tiles
disabled. Zoom 18 is taking FOREVER and will end up generating
41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any
experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile
store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles
files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I
would
also hope to provide tools to migrate a current TileSet into an
MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more):
https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an
(u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a
whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32
------------------------------------------------------------------------
Avast logo <https://www.avast.com/antivirus>

This email has been checked for viruses by Avast antivirus software.
www.avast.com <https://www.avast.com/antivirus>



------------------------------------------------------------------------
Avast logo <https://www.avast.com/antivirus>
This email has been checked for viruses by Avast antivirus software.
www.avast.com <https://www.avast.com/antivirus>





Re: MBTiles?

Rob Giuliano
 

(And yes . . . )
The longevity of such devices varies. I have a uSD from a very old (Samsung Galaxy Player 4.0).  I used that fro several years and moved files around (music, pictures, and app data).  After at least 5 years in that device, I now use it in a Raspberry Pi Zero.  Since it is an old uSD (and I expect it to fail soon), I use it to experiment with and it gets "wiped and reloaded" with different Pi distros very often.  Still going strong!
For a phone setup (although some may have limited space), it might be worth investigating Phone space vs. uSD.  Backup on one, active on other.  Any options to easily compress the backup? Robert Giuliano KB8RCO

From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: aprsisce@...
Sent: Friday, March 31, 2017 9:38 AM
Subject: Re: [aprsisce] MBTiles?

  Having just replaced a failing uSD card in my phone (3 years-ish old), the failure mode was that the card become non-writable.   When Android tried to write to it, the card spontaneously dismounted causing ALL of my map tiles to disappear (I was making changes to APRSISMO at the time and really thought I had borked everything until I found the dismounted SD card).

I'm suspecting that the single file really isn't any more prone to failure modes than the file system on the card containing lots of tiles.  In either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the active one if it is enabled for writing.  (Yes, I'm thinking to provide for Read-Only MBTiles tile sets that never fetch tiles, but only display what you already have in the file.)  (And yes, I'm also considering using an MBTiles file for what's available within it and using other TileSets for displaying other areas.)  (And yes, I just put the code in APRSIS32 that stores freshly downloaded tiles inside an MBTile database.)  (And yes, this is the last parenthetical!).

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... [aprsisce] wrote:

I am thinking more along the mobile/portable application where vibration and temperature variation might cause a failure and there is no access to the back up since it is home. This is not unsurmountable since uSD cards are small.   Best regards, Fred N7FMH         From: aprsisce@... [mailto:aprsisce@...]
Sent: Thursday, March 30, 2017 7:12 PM
To: aprsisce@...
Subject: RE: [aprsisce] MBTiles?     Yes, but even if you keep a bankup (like the -safe xml) the space could be smaller than the separate tile files.  At least on FAT32 disks. Robert Giuliano KB8RCO
    My only real concern is if the single file becomes corrupt then the whole thing is lost. Whereas in the other case, only a single tile may be gone.   It could be argued that there are fewer disk accesses so there would be less chance of failure.   Best regards, Fred N7FMH       From: aprsisce@... [mailto:aprsisce@...]
Sent: Wednesday, March 29, 2017 11:26 PM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?     The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.   Therefore, I like the idea of a database style, single file map interface.   Robert Giuliano KB8RCO     From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: APRSISCE Group <aprsisce@...>; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?     Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32    
| | This email has been checked for viruses by Avast antivirus software.
www.avast.com |

 



| | This email has been checked for viruses by Avast antivirus software.
www.avast.com |




#yiv5311559716 #yiv5311559716 -- #yiv5311559716ygrp-mkp {border:1px solid #d8d8d8;font-family:Arial;margin:10px 0;padding:0 10px;}#yiv5311559716 #yiv5311559716ygrp-mkp hr {border:1px solid #d8d8d8;}#yiv5311559716 #yiv5311559716ygrp-mkp #yiv5311559716hd {color:#628c2a;font-size:85%;font-weight:700;line-height:122%;margin:10px 0;}#yiv5311559716 #yiv5311559716ygrp-mkp #yiv5311559716ads {margin-bottom:10px;}#yiv5311559716 #yiv5311559716ygrp-mkp .yiv5311559716ad {padding:0 0;}#yiv5311559716 #yiv5311559716ygrp-mkp .yiv5311559716ad p {margin:0;}#yiv5311559716 #yiv5311559716ygrp-mkp .yiv5311559716ad a {color:#0000ff;text-decoration:none;}#yiv5311559716 #yiv5311559716ygrp-sponsor #yiv5311559716ygrp-lc {font-family:Arial;}#yiv5311559716 #yiv5311559716ygrp-sponsor #yiv5311559716ygrp-lc #yiv5311559716hd {margin:10px 0px;font-weight:700;font-size:78%;line-height:122%;}#yiv5311559716 #yiv5311559716ygrp-sponsor #yiv5311559716ygrp-lc .yiv5311559716ad {margin-bottom:10px;padding:0 0;}#yiv5311559716 #yiv5311559716actions {font-family:Verdana;font-size:11px;padding:10px 0;}#yiv5311559716 #yiv5311559716activity {background-color:#e0ecee;float:left;font-family:Verdana;font-size:10px;padding:10px;}#yiv5311559716 #yiv5311559716activity span {font-weight:700;}#yiv5311559716 #yiv5311559716activity span:first-child {text-transform:uppercase;}#yiv5311559716 #yiv5311559716activity span a {color:#5085b6;text-decoration:none;}#yiv5311559716 #yiv5311559716activity span span {color:#ff7900;}#yiv5311559716 #yiv5311559716activity span .yiv5311559716underline {text-decoration:underline;}#yiv5311559716 .yiv5311559716attach {clear:both;display:table;font-family:Arial;font-size:12px;padding:10px 0;width:400px;}#yiv5311559716 .yiv5311559716attach div a {text-decoration:none;}#yiv5311559716 .yiv5311559716attach img {border:none;padding-right:5px;}#yiv5311559716 .yiv5311559716attach label {display:block;margin-bottom:5px;}#yiv5311559716 .yiv5311559716attach label a {text-decoration:none;}#yiv5311559716 blockquote {margin:0 0 0 4px;}#yiv5311559716 .yiv5311559716bold {font-family:Arial;font-size:13px;font-weight:700;}#yiv5311559716 .yiv5311559716bold a {text-decoration:none;}#yiv5311559716 dd.yiv5311559716last p a {font-family:Verdana;font-weight:700;}#yiv5311559716 dd.yiv5311559716last p span {margin-right:10px;font-family:Verdana;font-weight:700;}#yiv5311559716 dd.yiv5311559716last p span.yiv5311559716yshortcuts {margin-right:0;}#yiv5311559716 div.yiv5311559716attach-table div div a {text-decoration:none;}#yiv5311559716 div.yiv5311559716attach-table {width:400px;}#yiv5311559716 div.yiv5311559716file-title a, #yiv5311559716 div.yiv5311559716file-title a:active, #yiv5311559716 div.yiv5311559716file-title a:hover, #yiv5311559716 div.yiv5311559716file-title a:visited {text-decoration:none;}#yiv5311559716 div.yiv5311559716photo-title a, #yiv5311559716 div.yiv5311559716photo-title a:active, #yiv5311559716 div.yiv5311559716photo-title a:hover, #yiv5311559716 div.yiv5311559716photo-title a:visited {text-decoration:none;}#yiv5311559716 div#yiv5311559716ygrp-mlmsg #yiv5311559716ygrp-msg p a span.yiv5311559716yshortcuts {font-family:Verdana;font-size:10px;font-weight:normal;}#yiv5311559716 .yiv5311559716green {color:#628c2a;}#yiv5311559716 .yiv5311559716MsoNormal {margin:0 0 0 0;}#yiv5311559716 o {font-size:0;}#yiv5311559716 #yiv5311559716photos div {float:left;width:72px;}#yiv5311559716 #yiv5311559716photos div div {border:1px solid #666666;height:62px;overflow:hidden;width:62px;}#yiv5311559716 #yiv5311559716photos div label {color:#666666;font-size:10px;overflow:hidden;text-align:center;white-space:nowrap;width:64px;}#yiv5311559716 #yiv5311559716reco-category {font-size:77%;}#yiv5311559716 #yiv5311559716reco-desc {font-size:77%;}#yiv5311559716 .yiv5311559716replbq {margin:4px;}#yiv5311559716 #yiv5311559716ygrp-actbar div a:first-child {margin-right:2px;padding-right:5px;}#yiv5311559716 #yiv5311559716ygrp-mlmsg {font-size:13px;font-family:Arial, helvetica, clean, sans-serif;}#yiv5311559716 #yiv5311559716ygrp-mlmsg table {font-size:inherit;font:100%;}#yiv5311559716 #yiv5311559716ygrp-mlmsg select, #yiv5311559716 input, #yiv5311559716 textarea {font:99% Arial, Helvetica, clean, sans-serif;}#yiv5311559716 #yiv5311559716ygrp-mlmsg pre, #yiv5311559716 code {font:115% monospace;}#yiv5311559716 #yiv5311559716ygrp-mlmsg * {line-height:1.22em;}#yiv5311559716 #yiv5311559716ygrp-mlmsg #yiv5311559716logo {padding-bottom:10px;}#yiv5311559716 #yiv5311559716ygrp-msg p a {font-family:Verdana;}#yiv5311559716 #yiv5311559716ygrp-msg p#yiv5311559716attach-count span {color:#1E66AE;font-weight:700;}#yiv5311559716 #yiv5311559716ygrp-reco #yiv5311559716reco-head {color:#ff7900;font-weight:700;}#yiv5311559716 #yiv5311559716ygrp-reco {margin-bottom:20px;padding:0px;}#yiv5311559716 #yiv5311559716ygrp-sponsor #yiv5311559716ov li a {font-size:130%;text-decoration:none;}#yiv5311559716 #yiv5311559716ygrp-sponsor #yiv5311559716ov li {font-size:77%;list-style-type:square;padding:6px 0;}#yiv5311559716 #yiv5311559716ygrp-sponsor #yiv5311559716ov ul {margin:0;padding:0 0 0 8px;}#yiv5311559716 #yiv5311559716ygrp-text {font-family:Georgia;}#yiv5311559716 #yiv5311559716ygrp-text p {margin:0 0 1em 0;}#yiv5311559716 #yiv5311559716ygrp-text tt {font-size:120%;}#yiv5311559716 #yiv5311559716ygrp-vital ul li:last-child {border-right:none !important;}#yiv5311559716

Re: MBTiles?

Lynn Deffenbaugh
 

Having just replaced a failing uSD card in my phone (3 years-ish old), the failure mode was that the card become non-writable. When Android tried to write to it, the card spontaneously dismounted causing ALL of my map tiles to disappear (I was making changes to APRSISMO at the time and really thought I had borked everything until I found the dismounted SD card).

I'm suspecting that the single file really isn't any more prone to failure modes than the file system on the card containing lots of tiles. In either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the active one if it is enabled for writing. (Yes, I'm thinking to provide for Read-Only MBTiles tile sets that never fetch tiles, but only display what you already have in the file.) (And yes, I'm also considering using an MBTiles file for what's available within it and using other TileSets for displaying other areas.) (And yes, I just put the code in APRSIS32 that stores freshly downloaded tiles inside an MBTile database.) (And yes, this is the last parenthetical!).

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... [aprsisce] wrote:


I am thinking more along the mobile/portable application where vibration and temperature variation might cause a failure and there is no access to the back up since it is home. This is not unsurmountable since uSD cards are small.

Best regards,

Fred N7FMH

*From:*aprsisce@... [mailto:aprsisce@...]
*Sent:* Thursday, March 30, 2017 7:12 PM
*To:* aprsisce@...
*Subject:* RE: [aprsisce] MBTiles?

Yes, but even if you keep a bankup (like the -safe xml) the space could be smaller than the separate tile files. At least on FAT32 disks.

Robert Giuliano

KB8RCO

My only real concern is if the single file becomes corrupt then
the whole thing is lost. Whereas in the other case, only a single
tile may be gone.

It could be argued that there are fewer disk accesses so there
would be less chance of failure.

Best regards,

Fred N7FMH

*From:*aprsisce@... <mailto:aprsisce@...>
[mailto:aprsisce@...]
*Sent:* Wednesday, March 29, 2017 11:26 PM
*To:* aprsisce@... <mailto:aprsisce@...>
*Subject:* Re: [aprsisce] MBTiles?

The real issue here is that we still format uSD (thumb drives,
etc) in a vey inefficient FAT32 format. Place the same files on
NTFS and the two numbers (actual vs. on disk) match much closer.
Unfortunately it is the most "cross platform" file system out there.

Therefore, I like the idea of a database style, single file map
interface.

Robert Giuliano

KB8RCO

------------------------------------------------------------------------

*From:*"'Lynn W Deffenbaugh (Mr)' kj4erj@...
<mailto:kj4erj@...> [aprsisce]" <aprsisce@...
<mailto:aprsisce@...>>
*To:* APRSISCE Group <aprsisce@...
<mailto:aprsisce@...>>; APRSISMO@...
<mailto:APRSISMO@...>
*Sent:* Wednesday, March 29, 2017 3:51 PM
*Subject:* [aprsisce] MBTiles?

Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off
of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB,
not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on
disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles"
the
way an OSM tile server does. It stores 8x8 squares of tiles in a
single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely
non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and
transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual
files.
There are tools that can generate MBTiles-formatted files from
various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of
Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's
internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any
experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile
store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles
files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I
would
also hope to provide tools to migrate a current TileSet into an
MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an
(u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a
whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

------------------------------------------------------------------------

Avast logo <https://www.avast.com/antivirus>



This email has been checked for viruses by Avast antivirus software.
www.avast.com <https://www.avast.com/antivirus>



------------------------------------------------------------------------
Avast logo <https://www.avast.com/antivirus>

This email has been checked for viruses by Avast antivirus software.
www.avast.com <https://www.avast.com/antivirus>




Re: MBTiles?

Fred Hillhouse
 

I am thinking more along the mobile/portable application where vibration and temperature variation might cause a failure and there is no access to the back up since it is home. This is not unsurmountable since uSD cards are small.

 

Best regards,

Fred N7FMH

 

 

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Thursday, March 30, 2017 7:12 PM
To: aprsisce@...
Subject: RE: [aprsisce] MBTiles?

 

 

Yes, but even if you keep a bankup (like the -safe xml) the space could be smaller than the separate tile files.  At least on FAT32 disks.

Robert Giuliano

KB8RCO

 

 

My only real concern is if the single file becomes corrupt then the whole thing is lost. Whereas in the other case, only a single tile may be gone.

 

It could be argued that there are fewer disk accesses so there would be less chance of failure.

 

Best regards,

Fred N7FMH

 

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Wednesday, March 29, 2017 11:26 PM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?

 

 

The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.

 

Therefore, I like the idea of a database style, single file map interface.

 

Robert Giuliano

KB8RCO

 

 


From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: APRSISCE Group <aprsisce@...>; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?

 

 

Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

 

 


Avast logo

This email has been checked for viruses by Avast antivirus software.
www.avast.com

 



Re: MBTiles?

Rob Giuliano
 

Yes, but even if you keep a bankup (like the -safe xml) the space could be smaller than the separate tile files.  At least on FAT32 disks.

Robert Giuliano
KB8RCO

On Thu, Mar 30, 2017 at 2:00 PM, 'Fred Hillhouse' fmhillhouse@... [aprsisce]
wrote:
 

My only real concern is if the single file becomes corrupt then the whole thing is lost. Whereas in the other case, only a single tile may be gone.

 

It could be argued that there are fewer disk accesses so there would be less chance of failure.

 

Best regards,

Fred N7FMH

 

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Wednesday, March 29, 2017 11:26 PM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?

 

 

The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.

 

Therefore, I like the idea of a database style, single file map interface.

 

Robert Giuliano

KB8RCO

 

 


From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: APRSISCE Group <aprsisce@...>; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?

 

 

Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

 




Avast logo

This email has been checked for viruses by Avast antivirus software.
www.avast.com


Re: MBTiles?

Fred Hillhouse
 

My only real concern is if the single file becomes corrupt then the whole thing is lost. Whereas in the other case, only a single tile may be gone.

 

It could be argued that there are fewer disk accesses so there would be less chance of failure.

 

Best regards,

Fred N7FMH

 

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Wednesday, March 29, 2017 11:26 PM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?

 

 

The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.

 

Therefore, I like the idea of a database style, single file map interface.

 

Robert Giuliano

KB8RCO

 

 


From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: APRSISCE Group <aprsisce@...>; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?

 

 

Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32