Re: MBTiles?

Greg D
 

There are techniques to extend the lifetime of SSDs under Linux.  I don't know what of these techniques are used by or applicable to Android (which is Linux at the core).  For your laptops, I recommend this write-up:

https://sites.google.com/site/easylinuxtipsproject/ssd

The keys are noatime and discard / TRIM.  Lacking a spinning place for Swap, add more RAM to remove or lessen the need for it.

Greg.

Rob Giuliano @KB8RCO [aprsisce] wrote:



Solid State drives (including SDs and thumb drives) have always had a
read/write limitation concern.  Most list something in the range of 10,000
to 100,000 read/write cycles (or at least that was an old range).


When we got new laptops (Lenovos) at work, they came with solid state
drives (great for battery life!).  We were concerned about the life
expectancy.  We compile software which creates lots of temporary files. 
We also do a lot of testing which generates a lot of data files.


Taking 50,000 as an average and 200 work days a year, that is only 250
read/writes per year.  Of course that is for a given "area" of the device.
BUT, since the drives are only 250 GB (vs. the old spinning drives which
were 512 - 1000 MB), our laptop drives are typically very full.  That
means the small space that is available is seeing all the "read/writes
cycles".


These newer drives are obviously going beyond that.  I have many coworkers
with their laptops at > 95% capacity for at least 3 years now, and haven't
heard of any crashing yet.


I suspect newer USB drives and uSD cards are in that same category (or at
least hope so!)

Robert Giuliano
KB8RCO



*From:* "'Fred Hillhouse' fmhillhouse@... [aprsisce]" (
aprsisce@... )
*To:* aprsisce@...
*Sent:* Friday, March 31, 2017 12:44 PM
*Subject:* RE: [aprsisce] MBTiles?


I used a 120GB USB hard drive for 5 years or so in the heat and cold. So
far it has been the best storage medium. It has been the most reliable
drive and it is still working. It does stay at home now after paying its
dues.

My 128GB uSD lasted three months which came to a $1/day usage fee before
failure. My 64 GB uSD has last 2 years now.

Most of the hard drive failures still allowed access to the majority of
files while all of the uSD cards failed completely. My USB flash drives
fail the same as the uSD.

I guess the real point of all this is back up your data. Losing a card on
the phone usually means you lost all media (pictures, video) as well. I
have as many apps as possible save the data to my uSD card.

Best regards,
Fred N7FMH


*From:* aprsisce@... [ mailto:aprsisce@... (
aprsisce@... ) ]
*Sent:* Friday, March 31, 2017 9:34 AM
*To:* aprsisce@...
*Subject:* Re: [aprsisce] MBTiles?


Having just replaced a failing uSD card in my phone (3 years-ish old), the
failure mode was that the card become non-writable.   When Android tried
to write to it, the card spontaneously dismounted causing ALL of my map
tiles to disappear (I was making changes to APRSISMO at the time and
really thought I had borked everything until I found the dismounted SD
card).

I'm suspecting that the single file really isn't any more prone to failure
modes than the file system on the card containing lots of tiles.  In
either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the
active one if it is enabled for writing.  (Yes, I'm thinking to provide
for Read-Only MBTiles tile sets that never fetch tiles, but only display
what you already have in the file.)  (And yes, I'm also considering using
an MBTiles file for what's available within it and using other TileSets
for displaying other areas.)  (And yes, I just put the code in APRSIS32
that stores freshly downloaded tiles inside an MBTile database.)  (And
yes, this is the last parenthetical!).

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32


On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... [aprsisce]
wrote:

I am thinking more along the mobile/portable application where vibration
and temperature variation might cause a failure and there is no access to
the back up since it is home. This is not unsurmountable since uSD cards
are small.

Best regards,
Fred N7FMH




*From:* aprsisce@... [ mailto:aprsisce@... (
aprsisce@... ) ]
*Sent:* Thursday, March 30, 2017 7:12 PM
*To:* aprsisce@...
*Subject:* RE: [aprsisce] MBTiles?


Yes, but even if you keep a bankup (like the -safe xml) the space could be
smaller than the separate tile files.  At least on FAT32 disks.
Robert Giuliano
KB8RCO



My only real concern is if the single file becomes corrupt then the whole
thing is lost. Whereas in the other case, only a single tile may be gone.

It could be argued that there are fewer disk accesses so there would be
less chance of failure.

Best regards,
Fred N7FMH



*From:* aprsisce@... [ mailto:aprsisce@... (
aprsisce@... ) ]
*Sent:* Wednesday, March 29, 2017 11:26 PM
*To:* aprsisce@...
*Subject:* Re: [aprsisce] MBTiles?


The real issue here is that we still format uSD (thumb drives, etc) in a
vey inefficient FAT32 format.  Place the same files on NTFS and the two
numbers (actual vs. on disk) match much closer.  Unfortunately it is the
most "cross platform" file system out there.

Therefore, I like the idea of a database style, single file map interface.


Robert Giuliano
KB8RCO



*From:* "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" < aprsisce@...
*To:* APRSISCE Group < aprsisce@... >; APRSISMO@...

*Sent:* Wednesday, March 29, 2017 3:51 PM
*Subject:* [aprsisce] MBTiles?


Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate

Join APRSISCE@groups.io to automatically receive all group messages.