Date   

Re: Is my iGate working correctly?

Lynn Deffenbaugh
 

KISS (not Simply()) is only for Kenwood radios and their internal TNCs.  Using this mode on a KPC-3 will knock the TNC out of KISS mode the first time you close the program or disable the port.  Please do the following:

1) Delete your KISS port and disable any other RF-type ports.
2) Close APRSISCE/32
3) Bring up a terminal emulator to your TNC
4) Make sure it is in KISS mode
5) Cycle power on your TNC
6) Make sure it is still in KISS mode
7) Monitor the TNC for received packets.  You should not be able to read the headers, but the APRS payloads are humanly readable.
8) Close the terminal emulator
9) Open APRSIS32 and configure a new Simply(KISS) type port
10) Go to Enables / Logging / Port() and enable that port
11) Monitor the Trace log and the packet scroller on the main screen
12) The packet scroller will show RF-received packets with a * next to them.
13) Also, the port status display should show APRS-IS OK alternating with OK

What is the callsign-SSID of the IGate instance?  There are things that can be learned remotely as well.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 4/5/2017 12:50 AM, kb7qps@... [aprsisce] wrote:

I'm not sure APRSIS32 is connected properly to my TNC. When I view the log for packets, I only get APRS-IS packets. No APRS-RF packets show in my log. When I click transmit, my tnc/radio does not transmit.


I am running a KPC-3 (not plus). I first had it setup in SimplyKISS mode and now have switched over to regular KISS mode.


Any help would be appreciated.



---In aprsisce@..., wrote :

The other issue I found is some aprs servers are 'faster' than others.
In the aprsisce.xml file is the server selection information.  Don't edit this file unless you know what you are doing, so I won't provide specifics.  APRSISCE/32 defaults to pick any server in the world.  I change it to only select servers in North America.  If your server happens to be in Sydney Australia, just because of travel time, other IGates will appear to have done more than you expect.  There are more ways to tell what is happening by looking at server statistics, but that's beyond this.
A little more information is here:

Arnold
KQ6DI

From: "Rob Giuliano kb8rco@... [aprsisce]"
To: "aprsisce"
Sent: Tuesday, April 4, 2017 8:39:31 PM
Subject: Re: [aprsisce] Is my iGate working correctly?

 


The IS system is a first come first stored system.  There is no need to keep hundreds of the same piece of data, so the first packet that makes into the system is "the one". 

You can look ath the logs of APRSIS32 to see what "your IGATE" is doing.  From the "Enables" menu choose "Logging" then "IGate" and IGate(NOT).  This will show what you heard and passed along. 

If "IGate" is empty, then ask questions.  Otherwise, it is completely up to the IS system to determine what to do with the data.
 
Robert Giuliano
KB8RCO


From: "kb7qps@... [aprsisce]"
To: aprsisce@...
Sent: Tuesday, April 4, 2017 11:29 PM
Subject: [aprsisce] Is my iGate working correctly?

 

I recently setup an iGate using APRSIS32 software. I am running a Kantronics KPC-3 with an Icom radio.

I have set it up for a few days in a high traffic area but when I look at APRS.fi, I find what appears to be little usage of my iGate. When I hover over other tracks, lines appear to other iGates that are used to transfer the packets from RF to IS and I can find no lines that trace back to my iGate.

How can I tell if my iGate is setup correctly? And if it is, why are packets being captured by other iGates rather than the one I setup?








Re: WXNOW.TXT FILE

Lynn Deffenbaugh
 

APRSISCE/32 will check the wxnow.txt file for changes after the configured object Interval has elapsed since the last transmission.  If the contents of the wxnow.txt file have not changed, then the object will not be transmitted yet, but the file will be polled more frequently.  When it has finally changed, and at least the object Interval has elapsed, a new transmission will be made.

Also, if the wxnow.txt file is inaccessible for more than 3 times in a row, then the weather object will be disabled and an internal message generated.  An internal message is generated on each individual access failure as well.

Is there a specific weather object callsign-SSID that you are wondering about?  If it is in your own instance, you should have a trace log called the same as your weather object that you can enable for additional details on the internal workings.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 4/4/2017 7:36 PM, Greg D ko6th.greg@... [aprsisce] wrote:
Do you want more or less often? 

I believe APRSISCE/32 imports it whenever it changes, so that's dependent on your Wx software.  But there must be a maximum poll rate too, so it's imported whenever it changes, but no more often than "x".  I just don't know what "x" is.

Greg  KO6TH


tmtlu3@... [aprsisce] wrote:
 

Is there a way to change how often APRSISCE imports the data from the WXNOW .TXT file?




Re: Is my iGate working correctly?

Greg Magone
 

I'm not sure APRSIS32 is connected properly to my TNC. When I view the log for packets, I only get APRS-IS packets. No APRS-RF packets show in my log. When I click transmit, my tnc/radio does not transmit.


I am running a KPC-3 (not plus). I first had it setup in SimplyKISS mode and now have switched over to regular KISS mode.


Any help would be appreciated.



---In aprsisce@..., <kq6di@...> wrote :

The other issue I found is some aprs servers are 'faster' than others.
In the aprsisce.xml file is the server selection information.  Don't edit this file unless you know what you are doing, so I won't provide specifics.  APRSISCE/32 defaults to pick any server in the world.  I change it to only select servers in North America.  If your server happens to be in Sydney Australia, just because of travel time, other IGates will appear to have done more than you expect.  There are more ways to tell what is happening by looking at server statistics, but that's beyond this.
A little more information is here:

Arnold
KQ6DI

From: "Rob Giuliano kb8rco@... [aprsisce]" <aprsisce@...>
To: "aprsisce" <aprsisce@...>
Sent: Tuesday, April 4, 2017 8:39:31 PM
Subject: Re: [aprsisce] Is my iGate working correctly?

 


The IS system is a first come first stored system.  There is no need to keep hundreds of the same piece of data, so the first packet that makes into the system is "the one". 

You can look ath the logs of APRSIS32 to see what "your IGATE" is doing.  From the "Enables" menu choose "Logging" then "IGate" and IGate(NOT).  This will show what you heard and passed along. 

If "IGate" is empty, then ask questions.  Otherwise, it is completely up to the IS system to determine what to do with the data.
 
Robert Giuliano
KB8RCO


From: "kb7qps@... [aprsisce]" <aprsisce@...>
To: aprsisce@...
Sent: Tuesday, April 4, 2017 11:29 PM
Subject: [aprsisce] Is my iGate working correctly?

 

I recently setup an iGate using APRSIS32 software. I am running a Kantronics KPC-3 with an Icom radio.

I have set it up for a few days in a high traffic area but when I look at APRS.fi, I find what appears to be little usage of my iGate. When I hover over other tracks, lines appear to other iGates that are used to transfer the packets from RF to IS and I can find no lines that trace back to my iGate.

How can I tell if my iGate is setup correctly? And if it is, why are packets being captured by other iGates rather than the one I setup?







Re: Is my iGate working correctly?

Arnold Harding. - KQ6DI
 

The other issue I found is some aprs servers are 'faster' than others.
In the aprsisce.xml file is the server selection information.  Don't edit this file unless you know what you are doing, so I won't provide specifics.  APRSISCE/32 defaults to pick any server in the world.  I change it to only select servers in North America.  If your server happens to be in Sydney Australia, just because of travel time, other IGates will appear to have done more than you expect.  There are more ways to tell what is happening by looking at server statistics, but that's beyond this.
A little more information is here:

Arnold
KQ6DI


From: "Rob Giuliano kb8rco@... [aprsisce]"
To: "aprsisce"
Sent: Tuesday, April 4, 2017 8:39:31 PM
Subject: Re: [aprsisce] Is my iGate working correctly?

 


The IS system is a first come first stored system.  There is no need to keep hundreds of the same piece of data, so the first packet that makes into the system is "the one". 

You can look ath the logs of APRSIS32 to see what "your IGATE" is doing.  From the "Enables" menu choose "Logging" then "IGate" and IGate(NOT).  This will show what you heard and passed along. 

If "IGate" is empty, then ask questions.  Otherwise, it is completely up to the IS system to determine what to do with the data.
 
Robert Giuliano
KB8RCO



From: "kb7qps@... [aprsisce]"
To: aprsisce@...
Sent: Tuesday, April 4, 2017 11:29 PM
Subject: [aprsisce] Is my iGate working correctly?

 

I recently setup an iGate using APRSIS32 software. I am running a Kantronics KPC-3 with an Icom radio.

I have set it up for a few days in a high traffic area but when I look at APRS.fi, I find what appears to be little usage of my iGate. When I hover over other tracks, lines appear to other iGates that are used to transfer the packets from RF to IS and I can find no lines that trace back to my iGate.

How can I tell if my iGate is setup correctly? And if it is, why are packets being captured by other iGates rather than the one I setup?







Re: Is my iGate working correctly?

Rob Giuliano
 

The IS system is a first come first stored system.  There is no need to keep hundreds of the same piece of data, so the first packet that makes into the system is "the one". 

You can look ath the logs of APRSIS32 to see what "your IGATE" is doing.  From the "Enables" menu choose "Logging" then "IGate" and IGate(NOT).  This will show what you heard and passed along. 

If "IGate" is empty, then ask questions.  Otherwise, it is completely up to the IS system to determine what to do with the data.
 
Robert Giuliano
KB8RCO



From: "kb7qps@... [aprsisce]"
To: aprsisce@...
Sent: Tuesday, April 4, 2017 11:29 PM
Subject: [aprsisce] Is my iGate working correctly?

 
I recently setup an iGate using APRSIS32 software. I am running a Kantronics KPC-3 with an Icom radio.

I have set it up for a few days in a high traffic area but when I look at APRS.fi, I find what appears to be little usage of my iGate. When I hover over other tracks, lines appear to other iGates that are used to transfer the packets from RF to IS and I can find no lines that trace back to my iGate.

How can I tell if my iGate is setup correctly? And if it is, why are packets being captured by other iGates rather than the one I setup?




Is my iGate working correctly?

Greg Magone
 

I recently setup an iGate using APRSIS32 software. I am running a Kantronics KPC-3 with an Icom radio.

I have set it up for a few days in a high traffic area but when I look at APRS.fi, I find what appears to be little usage of my iGate. When I hover over other tracks, lines appear to other iGates that are used to transfer the packets from RF to IS and I can find no lines that trace back to my iGate.

How can I tell if my iGate is setup correctly? And if it is, why are packets being captured by other iGates rather than the one I setup?



Re: WXNOW.TXT FILE

Greg D
 

Do you want more or less often? 

I believe APRSISCE/32 imports it whenever it changes, so that's dependent on your Wx software.  But there must be a maximum poll rate too, so it's imported whenever it changes, but no more often than "x".  I just don't know what "x" is.

Greg  KO6TH


tmtlu3@... [aprsisce] wrote:

 

Is there a way to change how often APRSISCE imports the data from the WXNOW .TXT file?



WXNOW.TXT FILE

Thomas Testa
 

Is there a way to change how often APRSISCE imports the data from the WXNOW .TXT file?


Re: Box w/Cross is What?

Kurt Savegnago
 

Ohhhhhhhhhhhhhhhhhhhhhhhhh, ahhhhhhhhhhhhhhhhhh,  So that's what those things are.  I always wondered but never asked!   Kurt KC9LDH



From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: aprsisce@...
Sent: Sunday, April 2, 2017 4:33 PM
Subject: Re: [aprsisce] Box w/Cross is What?

 
Fred is correct about the grid square.  Did you happen to get the callsign-SSID of the house?  We could look at the raw packet history at aprs.fi to confirm that the station was transmitting both grid squares and possibly posits which would subsequently trump the approximate location provided by the grid square.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 4/2/2017 4:32 PM, fmhillhouse fmhillhouse@... [aprsisce] wrote:
That is the center of a grid square I believe. 



Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone


-------- Original message --------
From: "bjessee@... [aprsisce]"
Date: 4/2/17 14:12 (GMT-05:00)
To: aprsisce@...
Subject: [aprsisce] Box w/Cross is What?

 
Today I saw a House symbol in the center of a rectangle with a cross, the house symbol in the center inside a circle as shown here - what is it? I've not seen a house symbol at that location near me before and it was gone after a while.
[URL=http://s29.photobucket.com/user/cbjesseeNH/media/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg.html][IMG]http://i29.photobucket.com/albums/c290/cbjesseeNH/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg[/IMG][/URL]




Re: Box w/Cross is What?

Fred Hillhouse
 

For ambiguity is one reason.

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Monday, April 03, 2017 4:23 AM
To: aprsisce@...
Subject: [aprsisce] Re: Box w/Cross is What?

 

 

Unfortunately not...  I take it the "house" is not actually where depicted but somewhere within the grid square? I know some devices, like a WSPRlite can only transmit a grid square by digital format, but why might an APRS user do so?




Re: Box w/Cross is What?

bjessee@...
 

Unfortunately not...  I take it the "house" is not actually where depicted but somewhere within the grid square? I know some devices, like a WSPRlite can only transmit a grid square by digital format, but why might an APRS user do so?


Re: Box w/Cross is What?

Lynn Deffenbaugh
 

Fred is correct about the grid square.  Did you happen to get the callsign-SSID of the house?  We could look at the raw packet history at aprs.fi to confirm that the station was transmitting both grid squares and possibly posits which would subsequently trump the approximate location provided by the grid square.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 4/2/2017 4:32 PM, fmhillhouse fmhillhouse@... [aprsisce] wrote:
That is the center of a grid square I believe. 



Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone


-------- Original message --------
From: "bjessee@... [aprsisce]"
Date: 4/2/17 14:12 (GMT-05:00)
To: aprsisce@...
Subject: [aprsisce] Box w/Cross is What?

 

Today I saw a House symbol in the center of a rectangle with a cross, the house symbol in the center inside a circle as shown here - what is it? I've not seen a house symbol at that location near me before and it was gone after a while.
[URL=http://s29.photobucket.com/user/cbjesseeNH/media/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg.html][IMG]http://i29.photobucket.com/albums/c290/cbjesseeNH/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg[/IMG][/URL]



Re: Box w/Cross is What?

Fred Hillhouse
 

That is the center of a grid square I believe. 



Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone


-------- Original message --------
From: "bjessee@... [aprsisce]" <aprsisce@...>
Date: 4/2/17 14:12 (GMT-05:00)
To: aprsisce@...
Subject: [aprsisce] Box w/Cross is What?

 

Today I saw a House symbol in the center of a rectangle with a cross, the house symbol in the center inside a circle as shown here - what is it? I've not seen a house symbol at that location near me before and it was gone after a while.
[URL=http://s29.photobucket.com/user/cbjesseeNH/media/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg.html][IMG]http://i29.photobucket.com/albums/c290/cbjesseeNH/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg[/IMG][/URL]


Box w/Cross is What?

bjessee@...
 

Today I saw a House symbol in the center of a rectangle with a cross, the house symbol in the center inside a circle as shown here - what is it? I've not seen a house symbol at that location near me before and it was gone after a while.
[URL=http://s29.photobucket.com/user/cbjesseeNH/media/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg.html][IMG]http://i29.photobucket.com/albums/c290/cbjesseeNH/A7017074-1F35-4DF3-ADCE-09A4AA58452D_zpstkwnsrvd.jpg[/IMG][/URL]


Re: MBTiles?

Greg D
 

Since I run APRSIS32 under Wine on my Linux desktop, I wonder what is the efficiency of the Tile storage with the EXT4 file system?

If EXT4 is efficient, and if Android can use EXT4 file systems on uSD cards, perhaps the easy answer is just to re-format the uSD card to EXT4 before using it? 

Greg  KO6TH


Rob Giuliano kb8rco@... [aprsisce] wrote:

 
The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.

Therefore, I like the idea of a database style, single file map interface.
 
Robert Giuliano
KB8RCO





From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]"
To: APRSISCE Group ; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?

 
Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32





Re: MBTiles?

Greg D
 

There are techniques to extend the lifetime of SSDs under Linux.  I don't know what of these techniques are used by or applicable to Android (which is Linux at the core).  For your laptops, I recommend this write-up:

https://sites.google.com/site/easylinuxtipsproject/ssd

The keys are noatime and discard / TRIM.  Lacking a spinning place for Swap, add more RAM to remove or lessen the need for it.

Greg.

Rob Giuliano @KB8RCO [aprsisce] wrote:



Solid State drives (including SDs and thumb drives) have always had a
read/write limitation concern.  Most list something in the range of 10,000
to 100,000 read/write cycles (or at least that was an old range).


When we got new laptops (Lenovos) at work, they came with solid state
drives (great for battery life!).  We were concerned about the life
expectancy.  We compile software which creates lots of temporary files. 
We also do a lot of testing which generates a lot of data files.


Taking 50,000 as an average and 200 work days a year, that is only 250
read/writes per year.  Of course that is for a given "area" of the device.
BUT, since the drives are only 250 GB (vs. the old spinning drives which
were 512 - 1000 MB), our laptop drives are typically very full.  That
means the small space that is available is seeing all the "read/writes
cycles".


These newer drives are obviously going beyond that.  I have many coworkers
with their laptops at > 95% capacity for at least 3 years now, and haven't
heard of any crashing yet.


I suspect newer USB drives and uSD cards are in that same category (or at
least hope so!)

Robert Giuliano
KB8RCO



*From:* "'Fred Hillhouse' fmhillhouse@... [aprsisce]" (
aprsisce@... )
*To:* aprsisce@...
*Sent:* Friday, March 31, 2017 12:44 PM
*Subject:* RE: [aprsisce] MBTiles?


I used a 120GB USB hard drive for 5 years or so in the heat and cold. So
far it has been the best storage medium. It has been the most reliable
drive and it is still working. It does stay at home now after paying its
dues.

My 128GB uSD lasted three months which came to a $1/day usage fee before
failure. My 64 GB uSD has last 2 years now.

Most of the hard drive failures still allowed access to the majority of
files while all of the uSD cards failed completely. My USB flash drives
fail the same as the uSD.

I guess the real point of all this is back up your data. Losing a card on
the phone usually means you lost all media (pictures, video) as well. I
have as many apps as possible save the data to my uSD card.

Best regards,
Fred N7FMH


*From:* aprsisce@... [ mailto:aprsisce@... (
aprsisce@... ) ]
*Sent:* Friday, March 31, 2017 9:34 AM
*To:* aprsisce@...
*Subject:* Re: [aprsisce] MBTiles?


Having just replaced a failing uSD card in my phone (3 years-ish old), the
failure mode was that the card become non-writable.   When Android tried
to write to it, the card spontaneously dismounted causing ALL of my map
tiles to disappear (I was making changes to APRSISMO at the time and
really thought I had borked everything until I found the dismounted SD
card).

I'm suspecting that the single file really isn't any more prone to failure
modes than the file system on the card containing lots of tiles.  In
either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the
active one if it is enabled for writing.  (Yes, I'm thinking to provide
for Read-Only MBTiles tile sets that never fetch tiles, but only display
what you already have in the file.)  (And yes, I'm also considering using
an MBTiles file for what's available within it and using other TileSets
for displaying other areas.)  (And yes, I just put the code in APRSIS32
that stores freshly downloaded tiles inside an MBTile database.)  (And
yes, this is the last parenthetical!).

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32


On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... [aprsisce]
wrote:

I am thinking more along the mobile/portable application where vibration
and temperature variation might cause a failure and there is no access to
the back up since it is home. This is not unsurmountable since uSD cards
are small.

Best regards,
Fred N7FMH




*From:* aprsisce@... [ mailto:aprsisce@... (
aprsisce@... ) ]
*Sent:* Thursday, March 30, 2017 7:12 PM
*To:* aprsisce@...
*Subject:* RE: [aprsisce] MBTiles?


Yes, but even if you keep a bankup (like the -safe xml) the space could be
smaller than the separate tile files.  At least on FAT32 disks.
Robert Giuliano
KB8RCO



My only real concern is if the single file becomes corrupt then the whole
thing is lost. Whereas in the other case, only a single tile may be gone.

It could be argued that there are fewer disk accesses so there would be
less chance of failure.

Best regards,
Fred N7FMH



*From:* aprsisce@... [ mailto:aprsisce@... (
aprsisce@... ) ]
*Sent:* Wednesday, March 29, 2017 11:26 PM
*To:* aprsisce@...
*Subject:* Re: [aprsisce] MBTiles?


The real issue here is that we still format uSD (thumb drives, etc) in a
vey inefficient FAT32 format.  Place the same files on NTFS and the two
numbers (actual vs. on disk) match much closer.  Unfortunately it is the
most "cross platform" file system out there.

Therefore, I like the idea of a database style, single file map interface.


Robert Giuliano
KB8RCO



*From:* "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" < aprsisce@...
*To:* APRSISCE Group < aprsisce@... >; APRSISMO@...

*Sent:* Wednesday, March 29, 2017 3:51 PM
*Subject:* [aprsisce] MBTiles?


Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate


Re: MBTiles?

Rob Giuliano
 

Interestingly, I grabbed a completely empty 64GB thumb drive and only placed the OSMTiles on it.
I experimented with formatting, dragging and dropping the same directory as the only thing on the drive after each format:

Formatted as Fat32:    
   95.4 MB, size on disk 581 MB, Disk Properties disk space used: 316 MB
Format exFAT  (4k Allocations)
   95.4 MB, size on disk 581 MB, Disk Properties disk space used:
Format exFAT  (16k Allocations)
   95.4 MB, size on disk 581 MB, Disk Properties disk space used:

Format NTFS  (4k Allocations)
   95.4 MB, size on disk 581 MB, Disk Properties disk space used: 245 MB
Format NTFS  (256b Allocations)
   95.4 MB, size on disk 581 MB, Disk Properties disk space used: 229 MB

It appears the Windows directory properties reporting system is reporting differently than the actual used disk space.

On my Windows 7 laptop's hard drive (SSD) it reports
   95.4 MB, size on disk 136 MB, Disk Properties disk space used:

Robert Giuliano
KB8RCO



From: "'W. Lim' wellmanl@... [aprsisce]"
To: "aprsisce@..."
Sent: Friday, March 31, 2017 1:20 PM
Subject: [aprsisce] Re: MBTiles?

 
Note that if the SD card is 8GB or less the cluster size is 4kb .
If over 8, then the cluster size is 32 kb. So if you moved your
tiles to an 8GB card, it might just fit.



Re: MBTiles?

Rob Giuliano
 

Solid State drives (including SDs and thumb drives) have always had a read/write limitation concern.  Most list something in the range of 10,000 to 100,000 read/write cycles (or at least that was an old range).

When we got new laptops (Lenovos) at work, they came with solid state drives (great for battery life!).  We were concerned about the life expectancy.  We compile software which creates lots of temporary files.  We also do a lot of testing which generates a lot of data files.

Taking 50,000 as an average and 200 work days a year, that is only 250 read/writes per year.  Of course that is for a given "area" of the device.  BUT, since the drives are only 250 GB (vs. the old spinning drives which were 512 - 1000 MB), our laptop drives are typically very full.  That means the small space that is available is seeing all the "read/writes cycles".

These newer drives are obviously going beyond that.  I have many coworkers with their laptops at > 95% capacity for at least 3 years now, and haven't heard of any crashing yet. 

I suspect newer USB drives and uSD cards are in that same category (or at least hope so!)
 
Robert Giuliano
KB8RCO



From: "'Fred Hillhouse' fmhillhouse@... [aprsisce]"
To: aprsisce@...
Sent: Friday, March 31, 2017 12:44 PM
Subject: RE: [aprsisce] MBTiles?

 
I used a 120GB USB hard drive for 5 years or so in the heat and cold. So far it has been the best storage medium. It has been the most reliable drive and it is still working. It does stay at home now after paying its dues.
 
My 128GB uSD lasted three months which came to a $1/day usage fee before failure. My 64 GB uSD has last 2 years now.
 
Most of the hard drive failures still allowed access to the majority of files while all of the uSD cards failed completely. My USB flash drives fail the same as the uSD.
 
I guess the real point of all this is back up your data. Losing a card on the phone usually means you lost all media (pictures, video) as well. I have as many apps as possible save the data to my uSD card.
 
Best regards,
Fred N7FMH
 
 
From: aprsisce@... [mailto:aprsisce@...]
Sent: Friday, March 31, 2017 9:34 AM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?
 
 
Having just replaced a failing uSD card in my phone (3 years-ish old), the failure mode was that the card become non-writable.   When Android tried to write to it, the card spontaneously dismounted causing ALL of my map tiles to disappear (I was making changes to APRSISMO at the time and really thought I had borked everything until I found the dismounted SD card).

I'm suspecting that the single file really isn't any more prone to failure modes than the file system on the card containing lots of tiles.  In either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the active one if it is enabled for writing.  (Yes, I'm thinking to provide for Read-Only MBTiles tile sets that never fetch tiles, but only display what you already have in the file.)  (And yes, I'm also considering using an MBTiles file for what's available within it and using other TileSets for displaying other areas.)  (And yes, I just put the code in APRSIS32 that stores freshly downloaded tiles inside an MBTile database.)  (And yes, this is the last parenthetical!).

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32


On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... [aprsisce] wrote:
I am thinking more along the mobile/portable application where vibration and temperature variation might cause a failure and there is no access to the back up since it is home. This is not unsurmountable since uSD cards are small.
 
Best regards,
Fred N7FMH
 
 
 
 
From: aprsisce@... [mailto:aprsisce@...]
Sent: Thursday, March 30, 2017 7:12 PM
To: aprsisce@...
Subject: RE: [aprsisce] MBTiles?
 
 
Yes, but even if you keep a bankup (like the -safe xml) the space could be smaller than the separate tile files.  At least on FAT32 disks.
Robert Giuliano
KB8RCO
 
 
My only real concern is if the single file becomes corrupt then the whole thing is lost. Whereas in the other case, only a single tile may be gone.
 
It could be argued that there are fewer disk accesses so there would be less chance of failure.
 
Best regards,
Fred N7FMH
 
 
 
From: aprsisce@... [mailto:aprsisce@...]
Sent: Wednesday, March 29, 2017 11:26 PM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?
 
 
The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.
 
Therefore, I like the idea of a database style, single file map interface.
 
Robert Giuliano
KB8RCO
 
 

From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: APRSISCE Group <aprsisce@...>; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?
 
 
Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32
 
 

Avast logo
This email has been checked for viruses by Avast antivirus software.
www.avast.com
 
 

Avast logo
This email has been checked for viruses by Avast antivirus software.
www.avast.com
 
 



Avast logo
This email has been checked for viruses by Avast antivirus software.
www.avast.com




Re: MBTiles?

w.lim
 

Note that if the SD card is 8GB or less the cluster size is 4kb .
If over 8, then the cluster size is 32 kb. So if you moved your
tiles to an 8GB card, it might just fit.


Re: MBTiles?

Fred Hillhouse
 

I used a 120GB USB hard drive for 5 years or so in the heat and cold. So far it has been the best storage medium. It has been the most reliable drive and it is still working. It does stay at home now after paying its dues.

 

My 128GB uSD lasted three months which came to a $1/day usage fee before failure. My 64 GB uSD has last 2 years now.

 

Most of the hard drive failures still allowed access to the majority of files while all of the uSD cards failed completely. My USB flash drives fail the same as the uSD.

 

I guess the real point of all this is back up your data. Losing a card on the phone usually means you lost all media (pictures, video) as well. I have as many apps as possible save the data to my uSD card.

 

Best regards,

Fred N7FMH

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Friday, March 31, 2017 9:34 AM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?

 

 

Having just replaced a failing uSD card in my phone (3 years-ish old), the failure mode was that the card become non-writable.   When Android tried to write to it, the card spontaneously dismounted causing ALL of my map tiles to disappear (I was making changes to APRSISMO at the time and really thought I had borked everything until I found the dismounted SD card).

I'm suspecting that the single file really isn't any more prone to failure modes than the file system on the card containing lots of tiles.  In either case, you're hosed.

I will look into keeping a "safe" copy of the MBTiles file, especially the active one if it is enabled for writing.  (Yes, I'm thinking to provide for Read-Only MBTiles tile sets that never fetch tiles, but only display what you already have in the file.)  (And yes, I'm also considering using an MBTiles file for what's available within it and using other TileSets for displaying other areas.)  (And yes, I just put the code in APRSIS32 that stores freshly downloaded tiles inside an MBTile database.)  (And yes, this is the last parenthetical!).


Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32



On 3/31/2017 9:24 AM, 'Fred Hillhouse' fmhillhouse@... [aprsisce] wrote:

I am thinking more along the mobile/portable application where vibration and temperature variation might cause a failure and there is no access to the back up since it is home. This is not unsurmountable since uSD cards are small.

 

Best regards,

Fred N7FMH

 

 

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Thursday, March 30, 2017 7:12 PM
To: aprsisce@...
Subject: RE: [aprsisce] MBTiles?

 

 

Yes, but even if you keep a bankup (like the -safe xml) the space could be smaller than the separate tile files.  At least on FAT32 disks.

Robert Giuliano

KB8RCO

 

 

My only real concern is if the single file becomes corrupt then the whole thing is lost. Whereas in the other case, only a single tile may be gone.

 

It could be argued that there are fewer disk accesses so there would be less chance of failure.

 

Best regards,

Fred N7FMH

 

 

 

From: aprsisce@... [mailto:aprsisce@...]
Sent: Wednesday, March 29, 2017 11:26 PM
To: aprsisce@...
Subject: Re: [aprsisce] MBTiles?

 

 

The real issue here is that we still format uSD (thumb drives, etc) in a vey inefficient FAT32 format.  Place the same files on NTFS and the two numbers (actual vs. on disk) match much closer.  Unfortunately it is the most "cross platform" file system out there.

 

Therefore, I like the idea of a database style, single file map interface.

 

Robert Giuliano

KB8RCO

 

 


From: "'Lynn W Deffenbaugh (Mr)' kj4erj@... [aprsisce]" <aprsisce@...>
To: APRSISCE Group <aprsisce@...>; APRSISMO@...
Sent: Wednesday, March 29, 2017 3:51 PM
Subject: [aprsisce] MBTiles?

 

 

Greetings all of you mappers out there.

I recent had occasion to copy all of my accumulated map tiles off of one
(failing) uSD card onto another. And I was SHOCKED to see how
inefficiently these tiles are stored on a 64GB uSD card! I've got a
total of 118,405 tiles across 4,624 folders whose size is 667MB, not bad
until you notice that they're occupying 14.4GB (yes GIGAbytes) on disk!
My zoom 20 is 7,925 files of 8.58MB occupying 989MB on disk. And my
zoom 18 is 22,001 files of 50.9MB occupying 2.67GB on the card. I've
got to do something to make this better. (Note that this is on an uSD
card under Android. The waste on an NTFS-formatted Windows volume is
nowhere near that bad.)

I've been considering for some time storing so-called "meta-tiles" the
way an OSM tile server does. It stores 8x8 squares of tiles in a single
meta-tile file with a hashed name that serves to distribute the files
across a file system rather than the strict z/x/y structure of the
actual tile names. The tile server knows how to get the correct image
to serve a single tile from an OSM-formatted URL, but they're not
actually stored on the server in that structure. But I really didn't
like this because it would make my cached tiles completely non-portable.

Enter MBTiles. This is the MapBox spec for tile storage and transport.
It amounts to an SQLite database storing blobs of image data inside a
single file rather than the current directory tree of individual files.
There are tools that can generate MBTiles-formatted files from various
data sources, including from the OSM Overpass API. I'm experimenting
with Maperitive right now generating an MBTiles file for all of Palm Bay
(my hometown) from zooms 0 through 18. I'm using Maperitive's internal
renderer from an Overpass downloaded data file with the web map tiles
disabled. Zoom 18 is taking FOREVER and will end up generating 41,989
tiles, most of which I'll never need because they're out in the
never-never swamp that surrounds the city.

But, enough on that. The question is: do any of you have any experience
or knowledge of MBTiles and do you have any
suggestions/comments/criticism of incorporating that as a tile store in
my APRS clients (yes, both on APRSISCE/32 and APRSISMO).

My plan would be to support importing externally-created MBTiles files
as well as storing all web-downloaded tiles locally in an MBTiles
database rather than as individual files in a folder hierarchy. I would
also hope to provide tools to migrate a current TileSet into an MBTiles
database and vice-versa, exploding an MBTiles database into a folder
hierarchy.

You can read about this at the following URLs:

Specification: https://github.com/mapbox/mbtiles-spec

Maperitive Home: http://maperitive.net/

Maperitive documentation: http://maperitive.net/docs/default.html

Tutorial using Maperitive: http://mapit-gis.com/offline-maps/

MapTiler MBTiles viewer (and more): https://www.maptiler.com/download/

Oh, and even though the storage waste on NTFS isn't as bad as an (u)SD
card, I'm planning this for APRSISCE/32 in the name of map tile
portability. It's a whole lot easier to copy a single file than a whole
tree structure of individual tiles.

Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

 

 


Avast logo

This email has been checked for viruses by Avast antivirus software.
www.avast.com

 

 


Avast logo

This email has been checked for viruses by Avast antivirus software.
www.avast.com