Segment Loss


Ian S Deans <ian@...>
 

Over the last six days or so I have been checking and trying a number of
things to determine why after 3 months of trouble free receiving ( at this
end ) I am now suffering from segment loss which can vary from about 30 HRIT
in twelve hours to over 60.
During this I noticed on the Tellique readme that they recommended a change
to the "rec ini" details to " allow buffered writing of log data " something
I had never done since day one. This evening I made the alteration to the
"rec ini" and since then the segment loss has trebled.
Clearly I will have to go back to the old setting, but can anyone suggest a
reason why this recommended change has made the problem even worse.

Thanks.

Ian.


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

--- In MSG-1@yahoogroups.com, "Ian S Deans" <ian@i...> wrote:
Over the last six days or so I have been checking and trying a
number of things to determine why after 3 months of trouble free
receiving ( at this end ) I am now suffering from segment loss which
can vary from about 30 HRIT in twelve hours to over 60.
During this I noticed on the Tellique readme that they recommended
a change to the "rec ini" details to " allow buffered writing of log
data " something I had never done since day one. This evening I made
the alteration to the "rec ini" and since then the segment loss has
trebled.
Clearly I will have to go back to the old setting, but can anyone
suggest a reason why this recommended change has made the problem
even worse.

Thanks.
Ian,

First: are you sure it isn't signal level or QRM related.

Second: I would also recommend making the change. It should read:

[locations]
log_file=>>recv.log

I have made this change on both of my systems.

Third: In a Eumetsat install, they appear to reduce the size of the
FSY files, the database used by the program. I have been advised
that this should be 250MB. Check the total size of the files in:

C:&#92;Tellique&#92;receiving&#92;tmp

On one system I have 2 x 100MB files and 1 26MB file, on the other 2
x 100 MB and 1 x 18MB. Arne van Belle is a great believer in
deleteing these files when you restart the Tellique client. It is
possible that you can't defrag these files (as they may be held
open) so letting the program recreate them is a good idea! My own
files seem to have increased to this very high level at the start of
my recent complete outage due to snow.

I have been advised:

"If you see "File system full" messages in the log files, increase
the parameter: "file_database_size=" in the recv.ini file.

"If you can't find this parameter, the default value of 250 MB is
used. This value should be sufficient even if you receive all data
from Eumetcast."


Other suggestions: card cooling - is it the 2.3 or 2.6B version you
have? Tighten connections? Any better after restarting the PC from
a mains-off reboot? Search back over this forum?
Bit clutching at straws, I'm afraid.


Cheers,
David


Ian S Deans <ian@...>
 

David, many thanks for your detailed reply to my missing segment problem.
My FSY files were all removed when I reloaded Tellique and I am fairly
certain that signal level etc is not a problem. I am using the new version
DVB card and actually had additional fan fitted about 4 weeks ago when the
chip fan packed up and had to be replaced.
I note that you also recommend the change in "rec ini" to allow buffered
writing of log data, but that change has caused me even more problems (
trebled the segment loss ) so I have reverted back to the old setting ---
strange.
I have now resigned myself to accepting that there is little more I can do.
For looking at individual images it is not a problem with the vast amount of
data we get, but unfortunately animations which in my opinion are an
important part of MSG are now mostly spoiled.

Many thanks again for your help and as others have mentioned for all your
superb support throughout the year.

Merry Xmas to you and Cecilia.

Ian


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

--- In MSG-1@yahoogroups.com, "Ian S Deans" <ian@i...> wrote:
[]
I am using the new version DVB card and actually had additional
fan fitted about 4 weeks ago when the chip fan packed up and had to
be replaced.

Driver version? V4.2.2 and not V4.2.8?

[]
Many thanks again for your help and as others have mentioned for
all your superb support throughout the year.

Merry Xmas to you and Cecilia.
Many thanks, Ian. To you, and all on the list, Merry Christmas and
a very Happy New Year!

Cheers,
David


Guy Martin <agm@...>
 

Merry Xmas to you both and thanks for all the help

Cheers, Guy

----- Original Message -----
From: David Taylor
To: MSG-1@yahoogroups.com
Sent: Wednesday, December 24, 2003 2:23 PM
Subject: [MSG-1] Re: Segment Loss


--- In MSG-1@yahoogroups.com, "Ian S Deans" <ian@i...> wrote:
[]
> I am using the new version DVB card and actually had additional
fan fitted about 4 weeks ago when the chip fan packed up and had to
be replaced.

Driver version? V4.2.2 and not V4.2.8?

[]
> Many thanks again for your help and as others have mentioned for
all your superb support throughout the year.
>
> Merry Xmas to you and Cecilia.

Many thanks, Ian. To you, and all on the list, Merry Christmas and
a very Happy New Year!

Cheers,
David



To unsubscribe from this group, send an email to:
MSG-1-unsubscribe@yahoogroups.com





------------------------------------------------------------------------------
Yahoo! Groups Links

a.. To visit your group on the web, go to:
http://groups.yahoo.com/group/MSG-1/

b.. To unsubscribe from this group, send an email to:
MSG-1-unsubscribe@yahoogroups.com

c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.


----------

http://www.gordano.com - Messaging for educators.


Ian S Deans <ian@...>
 

David, device manager driver details shows version V4.2.1.9999 although the
disc supplied with the new version card says V4.2.2b. That should be ok and
let's be honest it worked perfectly for the first 3 months.
Thanks again.

Ian.

----- Original Message -----
From: "David Taylor" <david-taylor@blueyonder.co.uk>
To: <MSG-1@yahoogroups.com>
Sent: Wednesday, December 24, 2003 2:23 PM
Subject: [MSG-1] Re: Segment Loss


--- In MSG-1@yahoogroups.com, "Ian S Deans" <ian@i...> wrote:
[]
I am using the new version DVB card and actually had additional
fan fitted about 4 weeks ago when the chip fan packed up and had to
be replaced.

Driver version? V4.2.2 and not V4.2.8?


James Brown
 

In message <000d01c3ca30$bd7c5970$eeeea8c0@ian>, Ian S Deans <ian@isdeans.freeserve.co.uk> writes
David, device manager driver details shows version V4.2.1.9999 although the
disc supplied with the new version card says V4.2.2b. That should be ok and
let's be honest it worked perfectly for the first 3 months.
Thanks again.
Ian I don't suppose there has been a loss of IRQ steering anywhere down the line? Does the DVB card have it's own? Any change in MB BIOS settings for the IRQ's?

Have you lost any RAM, installed any TV s/ware for the card?

No viruses, trojans, or firewalls checking files or virus scanners wanting to do the same?

Any s/ware updates you have performed - is it possible to roll back to a month when things were OK.

Using the system analysis tool how much CPU usage do you get, and what shows up in the networking.

It sure is annoying - and I'd love to help, but it looks like you are trying everything.

Cheers,

James
--
James Brown


Ian S Deans <ian@...>
 

Thanks for the mail James and detailed suggestions. I have looked at a
number of things and all appear to be ok. My last day with no segment loss
problems was 17/12/03 and I did roll back using restore but to no avail. The
DVB card is on IRQ 11 and shared with NVIDIA. One thing I haven't tried
though is disabling Norton Internet Security and Norton AntiVirus, might be
worth a try.
CPU usage varies greatly between about 5% and 100 % and I know that using
one fairly low spec computer is pushing things to the limit, but
nevertheless it does not explain the sudden change ( Athlon 1200, 768mb ram,
40Gb 7200 drive and 32mb NVIDIA graphics )
It is really very difficult with so many variables, can only hope that the
problem disappears as quickly as it appeared but I am not holding my breath.

Thanks again for your help and a Happy Xmas to you and your family.

Regards

Ian.


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

--- In MSG-1@yahoogroups.com, "Ian S Deans" <ian@i...> wrote:
Thanks for the mail James and detailed suggestions. I have looked
at a number of things and all appear to be ok. My last day with no
segment loss problems was 17/12/03 and I did roll back using restore
but to no avail.

Ian,

Did you install a new version of (for example) my software after the
17th?

Have you checked the Tellique log file in detail around the time of
a segment loss? Perhaps it is trying to tell you something? A few
of us run with the verbose logging set which does give quite a lot
of information (but may slightly increase the missing segment
rate). In recv.ini:

[logging]

log_file_size=1000000
log_file_number=30
log_level=verbose

I will certainlty have log files going back a day or two if you want
to check. Name a time period (say 1 minute either side of a loss)
and I could e-mail you my log file for the same period...

Cheers,
David


Ian S Deans <ian@...>
 

Thanks for your mail David and your kind offer of help. I will run with
verbose logging set for a period and see what that shows.
Will not be up and running again until Boxing day ( Going to Dunblane for
Xmas ) but will get back to you.
Since 17/12/03 I have not added/amended any program not even updated
Norton.
Strangely from about 8.30 to 17.00 I lost about 30 HRIT segments today but
from 17.00 to 21.15 only 1.

Thanks again.

Ian.


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

--- In MSG-1@yahoogroups.com, "Ian S Deans" <ian@i...> wrote:
Thanks for your mail David and your kind offer of help. I will run
with verbose logging set for a period and see what that shows.
Will not be up and running again until Boxing day ( Going to
Dunblane for Xmas ) but will get back to you.

Ian,

It does occur to me that sometimes disk-intensive operations may
interfere with the Tellique software, just as CPU-intensive
operations might. It also occurs to me that during a spell of
operation, over some weeks or months say, you will build up more
files, and may therefore be more susceptible to such problems. This
is on a system where you have both the Tellique and the MSG Data
Manager software running.

I have therefore made an update to the MSG Data Manager that will
run the "purge old images" housekeeping task at a lower priority (if
you have the Drop Priority box checked), and this may help in cases
such as yours. If you want a beta copy, please let me know.

Cheers,
David


Ian S Deans <ian@...>
 

David, I would very much like to try a beta copy. I have spent some of the
last 36 hours attempting to try and establish when and under what conditions
I am losing segments -- not easy. I have found however that there has been a
slow improvement since Xmas day ( computer off all day for MSG ) and if I
leave the computer running only for MSG segment loss is down to about an
average of less than 10 in about 13/14 hours. However using the computer for
MSGAnimator, internet, mail and odds and ends increases segment loss quite
considerably, although certain uses make losses much worse than others.
Unfortunately I have to continue using my computer for more than MSG.
The strange thing is that when I started out with MSG over 3 months ago,
this present scenario is what I expected using only one fairly low spec.
computer, but in fact the first 3 months were the complete opposite, with
totally trouble free running -- the fun of computers !!

I very much appreciate your continuing efforts to help/resolve the problem I
and others have re segment loss.

Regards

Ian.

----- Original Message -----
From: "David Taylor" <david-taylor@blueyonder.co.uk>
To: <MSG-1@yahoogroups.com>
Sent: Saturday, December 27, 2003 12:06 PM
Subject: [MSG-1] Re: Segment Loss

I have therefore made an update to the MSG Data Manager that will
run the "purge old images" housekeeping task at a lower priority (if
you have the Drop Priority box checked), and this may help in cases
such as yours. If you want a beta copy, please let me know.

Cheers,
David


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

--- In MSG-1@yahoogroups.com, "Ian S Deans" <ian@i...> wrote:
David, I would very much like to try a beta copy.
It's in the post. The evidence would be in trying to correlate the
segment loss with the house-keeping activity of the MSG Data
Manager. Unfortunately, that isn't logged, but happens at 238
minute intervals, I believe.

However using the computer for MSGAnimator, internet, mail and
odds and ends increases segment loss quite considerably, although
certain uses make losses much worse than others.

Anything that will help towards the understanding of what exactly is
required for low segment loss would very much be appreciated. High
CPU load is definitely one thing. High disk load (e.g. asking
Explorer to display a folder of 1000 files) is bad. Network I/O
seems to be OK, perhaps the network card throttles the total
activity to a reasonable level? Possibly, heavy use of the TCP/IP
stack causes some segment loss.

Unfortunately I have to continue using my computer for more than
MSG.

.. as do many people. Being able to run on a single PC would enable
more people to enjoy the hobby so is something we should work
towards. Right now, if you get a second PC, nothing fancy
500MHz/256MB, that's one solution using your existing PC for
everything except the DVB card and Tellique software. Making your
single main PC 2GHz/1GB is another possible solution. Others will
have their own "single PC" configurations, and it would be good to
hear from them.

Cheers,
David


Roger Mawhinney <roger@...>
 

Ian
If you are running XP then try running the Performance monitor. Programs ->
Run -> perfmon. You can select which parameters to continuously monitor and
importantly, it allows scheduled logging to a file - this may help isolate
your problems. It may also show regular background operations you were not
aware off.

I tend to monitor %processor idle time and %logical disk idle time. Even
when recording data and running MSGDM to "catch-up" on the previous day, my
% logical disk idle time = 86% (so its not stressed) and my processor is
idle about 15% of the time (so it's not on a limit). I run a fairly high
performance single PC set-up and rarely suffer lost segments even when using
intensive programs like Photoshop.

Cheers
Roger


a_van_belle
 

Hello Ian,

I don't know if this has been discussed before (long thread) but based on
your problem I guess a fragmented disc could be the cause.
Noticed too last month on my PIII (1 GHz, 512 Mb) that performance went
down and lost segments went up.
After running for months, the disc had become so fragmented that every disc
action was taking longer to complete.

Only taking the system off-line, deleting the *.fsy files from tellique and
defragmenting 3 times solved this.
Defragmentation the first time took long and left a lot of fragments on the
disc, so I started it over and over until all was "blue"

And Windows default settings do worsen this problem as by default Windows
will resize it's disc swap space whenever needed.
On a very fragmented disc, this will cause a swap space consisting of many
small chunks, lowering performance even more.
At Windows install I configured the swap space on the second partition (D:)
and fixed the size to 3 times the amount of RAM.
This way you can be sure that this part of the disc is reserved and will
never become fragmented by other files.

If you don't have a second partition you can improve by first setting
swapspace fixed and to a minimum, reboot, defragment and then set the
swapspace fixed to the recommended size and reboot again! This way the
newly created space should be placed on a free, not-fragmented part of the
disc.


Greetings,
Arne van Belle


Ian S Deans <ian@...>
 

Roger,

Thanks for your mail and advice. I will certainly have a look at that and
see what shows up. From Boxing day there has been a little improvement and
if I leave the computer solely for MSG segment loss is not serious, probably
less than 10 in 12/14 hours. However I need to use the computer for
internet, mail etc and of course animations and segment loss increases
significantly then. Strangely enough being on line is not a major cause of
lost segments, but if I try to prepare mail, delete mail etc ( off line )
segment loss increases.
All very strange and frustrating given that up to the 17/12/03 I had little
or no problems for the first 3 months. I do accept though that an Athlon
1200 with 768mb of PC 100/133 ram ( although 7200 HD ) is probably pushing
things a bit when you consider the amount of data that is flying in.
Unfortunately we have to take everything that comes in on channel 1 and 2
even although David's program will quickly delete the data we are not
wanting.

Thanks again for your help --- very much appreciated.

Regards

Ian.


Ian S Deans <ian@...>
 

Hello Arne,

Many thanks for your mail. I do defrag. very regularly ( every second day at
the moment ) but even at that the drive gets very badly fragmented very
quickly. I too have fixed the swap space to approx. twice my ram.
I do not have a second partition, but I think I will take your advice in
your last paragraph re getting the swapspace on a free part of the drive.
If all fails I will need to get my Athlon 3200 on order.

Appreciate your help.

Regards

Ian.


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

--- In MSG-1@yahoogroups.com, Arne van Belle <a.van.belle@h...>
wrote:
Hello Ian,

I don't know if this has been discussed before (long thread) but
based on
your problem I guess a fragmented disc could be the cause.
My current recommendation is to schedule defragmentation once a week
for each disk, and to set the defrag to run at Low priority. My
recommended program is Executive Software's Diskeeper.


At Windows install I configured the swap space on the second
partition (D:) and fixed the size to 3 times the amount of RAM.
This way you can be sure that this part of the disc is reserved and
will never become fragmented by other files.

Agreed completely about fixing the pagefile, I sometimes use 2 x
RAM, and sometimes 3 x RAM depending on the applictions I expect to
run. It is best on a second physical spindle, just putting it on a
second partition may simply cause more head movement. However, it
should most certainly be contiguous for optimum system performance,
which means configuring it correctly very early on in the
installation sequence.

The other thing to watch out for is overfilling a disk. This can
cause fragmentation of the master file table (MFT) and an off-line,
boot time defragment is required to cure that. Much better not to
let it happen. If your partition gets above 85% full, you can get
problems.

Cheers,
David


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

--- In MSG-1@yahoogroups.com, "Ian S Deans" <ian@i...> wrote:
[]
Unfortunately we have to take everything that comes in on channel
1 and 2 even although David's program will quickly delete the data
we are not wanting.


That's channels 2 and 3, actually. But it raises a point I made
much earlier on. Would it be better for Eumetsat to split the data
into rather more than 2 channels? For example, anyone getting HRIT
doesn't need the LRIT MSG-1 data, but does want the MSG-1 foreign
satellite data. Equally, those not licenced for MDD don't need to
take it. Finally, it might be useful to have two HRIT data streams,
primary with the same channels as LRIT currently (say), and
secondary with the remaining channels, and perhaps HRV on its own
channel.

Whether this would exacerbate the segment and prolog ordering issues
we see, I don't know. Is this even possible, I don't know. I would
certainly like to see finer-grained control over the data you need
to ingest.

Cheers,
David


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

--- In MSG-1@yahoogroups.com, "Ian S Deans" <ian@i...> wrote:
[]
If all fails I will need to get my Athlon 3200 on order.
Ian, rather than waste money on a fatser processor, perhaps get a
second-hand 500MHz/256MB PC (main box only) for reception only, and
network it with the present system? There will be plenty of people
with such a spare system around (I have one - make me an offer!)
Your existing keyboard, monitor and mouse can be used (KVM switch)
and you can get advice from Douglas on networking!

Cheers,
David