Date   

Single-machine statistics

Alan Sewards <alan.sewards@...>
 

Hi David,
As you know, I have been persevering with a single-machine setup, but
recently, I have been forced to make some changes. I thought you would be
interested in the results. I am running the 2.3.0a version of the
SkyStar2/Tellique software, MDM 1.2.4.255 and Animator 1.1.8.33.

I have been aware for some time that the wxsat machine has been
sluggish, with waits for the menus to pop up, jerky animations, etc. but it
all got too much last week and I checked out the Performance Monitor and
found to my horror that the pages writes per second were up in the hundreds
if not thousands! The Average Disk Queue was getting pretty long too but the
CPU was still handling things OK. So I decided to order another 512 MB RAM.
I expected that would arrive within a couple of days, but when it did not I
decided to connect my systems to run as a two-machine setup, with one PC for
the receive function and MDM, Animator, etc on the second. This proved to
have surprisingly little effect, the machine with MDM (but only 512 MB RAM)
was almost as bogged down as the single-machine had been, and as this
machine is my daily workhorse, this was not acceptable! Typical stats for
this (two-machine) setup are: Pages/sec: Avg 880, Disk Q: 2.5 Avg, CPU: 28
Avg. After a few days with still no new RAM, I changed back to the single
machine again (with 512 MB RAM). Stats for this were: Pages/sec: 1247 Avg,
Disk Q: 0.1 Avg, and CPU: 18 Avg. Finally the RAM arrived and I installed
it with these results: Pages/sec: 0.4 Avg, Disk Q: 0.009 Avg, CPU: 23 Avg. I
was surprised to find that the Commit Charge with 1GB RAM was still as high
as 404 MB, as the sum of memory needs for all processes did not seem to come
anywhere near 1024 MB. Snapshots later and following morning gave the
following: Pages: 3 Avg, Disk Q: 0.2 Avg, CPU 35 Avg; and pages: 0.4, Disk
Q: 0.005, CPU 18. Later I ran the Animator and the following resulted:
Pages: 0.4, Disk Q: 0.011, CPU: 27. Commit Charge went up to 490 MB. A bit
later (just after the big save at the end of a cycle) I got the following:
Pages: 578, Disk Q: 3, CPU: 78, but within a few seconds things were back to
the more normal figures quoted above.

My comments on these figures are as follows:
1. There is a HUGE difference between a machine with 512 MB and 1024 MB when
it comes to running MDM. With only (!) 512 MB RAM, the machine quickly
becomes bogged down in paging to disk, not perhaps to the point of losing
data, but certainly to the point of becoming a pain to use!
2. It seems essential that the machine running MDM has 1 GB of RAM. If it
does, and is fast enough (mine is Athlon XP2400+), it can handle as well the
receive function without noticing.
3. I don't understand why with 1 GB RAM, such a large Commit Charge is
required. Where is all this memory being used? MDM is shown as taking up
200-350 MB depending on when you look, but the sum total of all the other
processes running does not reach that figure.

Sorry this is rather long, but I felt the data might be useful and of
general interest.

Best regards - Alan

Alan Sewards
email: alan.sewards@...
web site: http://asewards.free.fr


Re: Missing HRV Sector 17

Peter Benney <tugboat@...>
 

In all of my recent testing I have both the Upper and Lower boxes
checked for the Centre HRV window option on the Advanced setup.
It might be interesting to know what you have set, and if setting those
options removes the problem.
Upper and lower boxes checked.

If it's not the MSG Data Manager - do you have a process running
every 15 minutes on your PC which might be interfering with
reception.
I have Zone Alarm running on the RX PC.

If it _was_ the MSG Data Manager, I would have thought
that other people would have seen the problem as well.
Definately not the Data Manager David as the Tellique log on the RX PC shows that the files were not received.On the 0615 cycle two files were reported missing or incomplete and I would have expected to see this on the other cycles.

Peter


Re: Missing HRV Sector 17

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

--- In MSG-1@..., "Peter Benney" <tugboat@g...> wrote:
I seem to have a problem receiving this HRV sector. This morning
it is missing from the 0530, 0600, 0615 and 0700 cycles. The
Tellique log appears normal except for the 0615 cycle when a missing
or incomplete file warning is logged between the reception of
sectors 16 and 18. No clues in the DM log. This is happening quite
regularly with missing sector 17's on most days.

Any ideas ?
Peter,

That sector is typically the one where the switch is made from the
lower window of the scan to the upper. In all of my recent testing
I have both the Upper and Lower boxes checked for the Centre HRV
window option on the Advanced setup. It might be interesting to
know what you have set, and if setting those options removes the
problem.

If it's not the MSG Data Manager - do you have a process running
every 15 minutes on your PC which might be interfering with
reception. If it _was_ the MSG Data Manager, I would have thought
that other people would have seen the problem as well.

Cheers,
David


Missing HRV Sector 17

Peter Benney <tugboat@...>
 

I seem to have a problem receiving this HRV sector. This morning it is missing from the 0530, 0600, 0615 and 0700 cycles. The Tellique log appears normal except for the 0615 cycle when a missing or incomplete file warning is logged between the reception of sectors 16 and 18. No clues in the DM log. This is happening quite regularly with missing sector 17's on most days.

Any ideas ?

Peter


resolution

Guy Martin <agm@...>
 

I may have missed the discussion about this but when animating HRV, as the terminator moves across and the pictures descends into darkness the resolution goes to 'pot', square edged chunks of dark grey and black. I can see this effect on some of the LRIT images as well. Is this an effect of the lossy compression or the fact that graphics cards only use 8 bits even for a greyscale.

regards, Guy

----------

http://www.gordano.com - Messaging for educators.


Re: IMPF

Luca Bertagnolio <lucaberta@...>
 

Actually what happened was, they skipped all the 7th HRIT segments
and HRV segments 19 thru 23, so the final segments were all
disseminated, and MDM recorded the image normally.

Bye, Luca

--- In MSG-1@..., "Douglas Deans" <douglas@d...> wrote:
As advised the IMPF changed occurred at 13.00. As suggested the
data is
definitely earlier, and to allow for that the 12.45 UTC scan was
cleverly
ended early by, I think, sending blank final sectors. So David's
program
did not need to wait.
Looks good.

Douglas.


Re: IMPF

Luca Bertagnolio <lucaberta@...>
 

--- In MSG-1@..., "David Taylor" <david-taylor@b...>
wrote:
Luca, that usually means that the OS has created the file, but
nothing has yet been written to it. Unusual, yes. Perhaps
Tellique
took its time writing the file data?
sure, that's what I thought, but the odd thing was that the 0k file
was surrounded by other files with reasonable lentgh, and there were
some three 0k files in a couple of minutes time.

We'll see later, when I turn on MDM, if those files are OK or not.
Running the usual 8% signal strength today... ;-)

Bye, Luca


Re: Tellique software...........

John Parkins <kvp@...>
 

Hello Alan,

No the log didn't help at all on one line it was downloading a file
the next line was when I restarted it! Oh well at least I know it's
not just me. In fact I had been rather impressed with that machine, it
had been running for nearly a month since it's last re-boot.

Tuesday, October 28, 2003, 3:39:25 PM, you wrote:

AS> John,
AS> I have had this happen twice, both times I reported it here, the second
AS> email is quoted below FYI (17/10/2003).

AS> Hi all,
AS> This morning at 0653 UTC, the tellique software shut itself down after
AS> having problems with the child process, almost identical to that which I had
AS> a few days ago and reported here. I did not notice this until a couple of
AS> hours later, and tried to restart tellique but it would not restart. I had
AS> to shut down the computer and restart it, whereupon tqrecv would start and
AS> run and everything else could be got back running again. I wish I knew the
AS> cause of this, but whatever the cause it does show that tqrecv is not
AS> robust. Perhaps the newer version (is it officially approved by Eumetsat
AS> yet?) may be better.

AS> Oh, and HRV is back over Europe, thank you!

AS> Best regards - Alan
AS> BTW: the Tellique log showed the shutting down of tqrecv when it had lost
AS> its child process. what did your log say?

AS> Best of luck - Alan

--
Best regards,
John mailto:kvp@...


Re: Tellique software...........

Alan Sewards <alan.sewards@...>
 

John,
I have had this happen twice, both times I reported it here, the second
email is quoted below FYI (17/10/2003).

Hi all,
This morning at 0653 UTC, the tellique software shut itself down after
having problems with the child process, almost identical to that which I had
a few days ago and reported here. I did not notice this until a couple of
hours later, and tried to restart tellique but it would not restart. I had
to shut down the computer and restart it, whereupon tqrecv would start and
run and everything else could be got back running again. I wish I knew the
cause of this, but whatever the cause it does show that tqrecv is not
robust. Perhaps the newer version (is it officially approved by Eumetsat
yet?) may be better.

Oh, and HRV is back over Europe, thank you!

Best regards - Alan
BTW: the Tellique log showed the shutting down of tqrecv when it had lost
its child process. what did your log say?

Best of luck - Alan


----- Original Message -----
From: "John Parkins" <kvp@...>
To: "MSG-1 Mailing List" <MSG-1@...>
Sent: Tuesday, October 28, 2003 4:17 PM
Subject: [MSG-1] Tellique software...........


Hello,

Erm.... just noticed that for some reason the Tellique software just
stopped at 23:56 last night. It's running on it's own computer
(1800+AMD WinXP) and hasn't done this before. Anyone else had the
software just stop? No indications in the log and the computer and
Skystar card are working fine.



--
Best regards,
John mailto:kvp@...





To unsubscribe from this group, send an email to:
MSG-1-unsubscribe@...



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/




Re: IMPF

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

hmmm... how strange, I had a bunch of files with 0k length, but I
just checked now and they are of proper length. Just hit F5 and
their correct size appeared. Never seen this behaviour before.

FYI, I monitor my receiving PC at home via VNC over my VPN link
between the router at home and the office, but as I said I had
never
seen a 0k file before.
Luca, that usually means that the OS has created the file, but
nothing has yet been written to it. Unusual, yes. Perhaps Tellique
took its time writing the file data?

Ciao,
David


Tellique software...........

John Parkins <kvp@...>
 

Hello,

Erm.... just noticed that for some reason the Tellique software just
stopped at 23:56 last night. It's running on it's own computer
(1800+AMD WinXP) and hasn't done this before. Anyone else had the
software just stop? No indications in the log and the computer and
Skystar card are working fine.



--
Best regards,
John mailto:kvp@...


Re: IMPF

Douglas Deans <douglas@...>
 

----- Original Message -----
From: "David Taylor" <david-taylor@...>
To: <MSG-1@...>
Sent: Tuesday, October 28, 2003 2:44 PM
Subject: [MSG-1] Re: IMPF


Channel 6 HRIT also looks a lot better. The interpolation
artefacts are almost gone now but at the expense of quite a loss on
the west side.

Presumably, you don't have the "Calibrate thermal" set for channel 6?
Oops you are right !!

Hadn't noticed the west side loss.
Still a nice balanced disc now.
HRV outlines now spot on.
The change has certainly rounded off the few remaining sharp edges...
excellent.

Regards
Douglas.


Re: IMPF

Luca Bertagnolio <lucaberta@...>
 

--- In MSG-1@..., "David Taylor" <david-taylor@b...>
wrote:
No, I didn't see that. What were the file names? (There should
never be zero length files, as all files should have a header).
hmmm... how strange, I had a bunch of files with 0k length, but I
just checked now and they are of proper length. Just hit F5 and
their correct size appeared. Never seen this behaviour before.

FYI, I monitor my receiving PC at home via VNC over my VPN link
between the router at home and the office, but as I said I had never
seen a 0k file before.

Bye, Luca


Re: IMPF

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

I am not running the decode PC now, so I am just monitoring the
downloaded data. Did you noticed to 0k files in both HRV and HRIT
channels around 14:12?

Bye, Luca
No, I didn't see that. What were the file names? (There should
never be zero length files, as all files should have a header).

Cheers,
David


Re: IMPF

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

Channel 6 HRIT also looks a lot better. The interpolation
artefacts are almost gone now but at the expense of quite a loss on
the west side.

Presumably, you don't have the "Calibrate thermal" set for channel 6?

Hadn't noticed the west side loss.

Cheers,
David


Re: IMPF

Luca Bertagnolio <lucaberta@...>
 

The scans seem to now fit well into the 15 minutes slot, the last
file of the 1400 slot was sent at 14:13, so the EPI file is now
really an epilogue and not a mix between two adjacent slots, finally!

I am not running the decode PC now, so I am just monitoring the
downloaded data. Did you noticed to 0k files in both HRV and HRIT
channels around 14:12?

Bye, Luca

--- In MSG-1@..., "Douglas Deans" <douglas@d...> wrote:
As advised the IMPF changed occurred at 13.00. As suggested the
data is
definitely earlier, and to allow for that the 12.45 UTC scan was
cleverly
ended early by, I think, sending blank final sectors. So David's
program
did not need to wait.


HRV channel experiment - was: Re: Re: IMPF

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

Will there be a temp to let us see the trial HRV with wavelet lossless,
David.

Regards
Douglas.
It should just be automatic, I think. The section that handles the
decoding of segments knows nothing about whether it's dealing with an
HRIT, LRIT or whatever channel, it just decodes wavelet, JPEG etc. to an
image segment. I'm planning on keeping some raw data just in case.

Cheers,
David


Re: IMPF

Douglas Deans <douglas@...>
 

----- Original Message -----
From: "David Taylor" <david-taylor@...>
To: <MSG-1@...>
Sent: Tuesday, October 28, 2003 1:31 PM
Subject: [MSG-1] Re: IMPF


--- In MSG-1@..., "Douglas Deans" <douglas@d...> wrote:
As advised the IMPF changed occurred at 13.00. As suggested the
data is definitely earlier, and to allow for that the 12.45 UTC scan
was cleverly ended early by, I think, sending blank final sectors.
So David's program did not need to wait.
Looks good.

Douglas.
Well I just checked the spatial accuracy of HRV, and it's much
better than before. The water vapour gains have been adjusted as
well, making the LRIT water vapour image (channel 5) much more
usable. I think the interpolation artefacts at the SE and NW limbs
are reduced as well.... Still waiting for the first foreign
satellite data, although I don't know if it will affected by the
change.

Hurrah! Let me know if you do see any issues....

Cheers,
David
Channel 6 HRIT also looks a lot better. The interpolation artefacts are
almost gone now but at the expense of quite a loss on the west side.
The data is definitely earlier and there now appears to be a gap between
scans perhaps resolving for once and for all the prologue/last sector issue.
All looking very good. Tomorrow (HRV test) will also be interesting.
Will there be a temp to let us see the trial HRV with wavelet lossless,
David.

Regards
Douglas.


Re: IMPF

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

--- In MSG-1@..., "Douglas Deans" <douglas@d...> wrote:
As advised the IMPF changed occurred at 13.00. As suggested the
data is definitely earlier, and to allow for that the 12.45 UTC scan
was cleverly ended early by, I think, sending blank final sectors.
So David's program did not need to wait.
Looks good.

Douglas.
Well I just checked the spatial accuracy of HRV, and it's much
better than before. The water vapour gains have been adjusted as
well, making the LRIT water vapour image (channel 5) much more
usable. I think the interpolation artefacts at the SE and NW limbs
are reduced as well.... Still waiting for the first foreign
satellite data, although I don't know if it will affected by the
change.

Hurrah! Let me know if you do see any issues....

Cheers,
David


IMPF

Douglas Deans <douglas@...>
 

As advised the IMPF changed occurred at 13.00. As suggested the data is
definitely earlier, and to allow for that the 12.45 UTC scan was cleverly
ended early by, I think, sending blank final sectors. So David's program
did not need to wait.
Looks good.

Douglas.