Segment transmission order (long repost)


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

The following has been discussed on the SatSignal list. Although it
is arguably an issue with specific software, it may be of more
general interest.

David
------------------------------------------


Hi all,

I have just noticed something I had not seen before, which turned
out to be an issue.

Basically, every 8th HRIT segment and the last 3 HRV segments were
delayed in the 1215 scan, so the 1st segments of the next scan came
before, and MDM started to decode the new scan, and only erased the
8th segments of the previous scan when they appeared in the received
directory, instead of decoding them before erasing.

I hate to say this, but since the 8th segments are the most useful
to us europeans, I sure would like to see some timer applied to the
next scan decoding, in order to allow some time should any 8th
segment be late in being delivered by EUMETCast.

How about if MDM would hold the new EPI and data segments for a
couple of minutes before decoding the new scan?

David, shoot! :-)

Bye, Luca ON4OVV
Brussels, Belgium
------------------------------------------


Luca,

This is something that has only recently happened. For many months
of these trials things have been normal, and the segments have been
sent in the order they have been processed, so I don't know what
change has recently been made which causes the problem (all recent
changes have been with the LRIT stream, and not the HRIT stream).
Perhaps we are actually simply seeing an error state rather than the
normal state. What I suspect _may_ have happened is that segment 1
is now being higher priority over the other segments, and may
therefore be sent earlier even though it is processed later.

Already, the MSG Data Manager does not use the timing of the prologue
information to determine when to start a new scan. It only uses the
change of cycle number on a per-channel basis to decide when to
clear old images. Of course, you can always uncheck the "Clear
images" setup option, which will leave you with image data for the
last segment, albeit slightly out of date data.

It would require a significant rework of the program to allow older
images to to updated after being saved. I am not saying it is
impossible, but it would mean, for example, reloading the JPEG
images, updating them, and saving the images back. A degradation
for each load/save cycle, at least when using JPEG. The program
would also need to remember the offset for each half of the channel
12 image for each back cycle that might, or might
not come in.

To me, it is now a Eumetcast problem to ensure that all segments
(for one particular channel) are transmitted at the same priority
and therefore in strict sequence. If there is any need to
prioritise the HRIT bandwidth, why not give channels 1, 3, 4, 5, 9
and 12 priority over the others? That way, there is some scope for
flexibility without ruining images.

The same issue arises in catch-up mode - Eumetsat insist on sending
the most recent data first, whereas if there is only one cycle or
less to catch up, most likely the channel has enough capacity to
take at least the higher priority channels (suggested above) in time
order, thus giving you more data than a "most recent first"
algorithm would.

I will take these issues up with Eumetsat and let us hope that
something can be done.

Cheers,
David
------------------------------------------

[Luca]
well, I already don't use "clear images". The thing struck me as I
had just turned on MDM in the middle of the scan, so I had received
segment 5,6 and 7, and when finally I would see what was over
Europe... nothing! Segment 1 of the next scan had already arrived,
and JPEGs already saved with the black band.
[]
how about not saving the images right away? Instead, hold the next
scan data in the received directory for maximum 2-3 minutes until
we're kind of sure that no data from the old scan would come, and
then once the timeout expires (or the 8th segment comes) save the
JPEG?

Would this require major rework of the code too?
[]
good point, maybe this will even get worse by the time DWDsat data
starts to flow (or maybe EUMETCast already sends that data and we
don't know about it).



------------------------------------------
[David]

I am very wary of tying in timeouts to try and suit how a particular
communications channel (Eumetcast) is behaving on one particular
day. I mean, this channel in general handles data in sequence,
doesn't it? Do movies come out scrambled? Of course not! So my
view is that MSG-1 data should be correctly sequenced as well!
[]
My guess, for what it's worth, is that it may be the link between
Eumetsat and the Eumetcast distribution point. Many of the
algorithms were designed not for DVB dissemination, but for a direct
dedicated 1Mb/s downlink from MSG-1. Already the HRIT bandwidth has
crept up to 1.7Mb/s, and yet some part of the link isn't behaving
properly. Why not up the bandwidth even more so that accumulated
back data can be sent in sequence as well?

I think that all of us are still learning about this novel form of
dissemination and how best to use it!
------------------------------------------


a_van_belle
 

I have just noticed something I had not seen before, which turned
out to be an issue.
Just some observations:

The missing segmens report does not mention these "out of order"
segments, so to see the impact on our images we have to browse them.

This way I noticed an odd one: 1315 HRV (ch12) does show full image
width Africa !

Don't know if the larger amount of data in this image started it but
scan 1330 and 1345 were heavily affected by out of order segments.

Scan 1900 does have missing segments over here.
So we can assume that work is in progress.

Greetings,
Arne van Belle


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

This way I noticed an odd one: 1315 HRV (ch12) does show full
image width Africa !

Which day was that, Arne?

Don't know if the larger amount of data in this image started it
but
scan 1330 and 1345 were heavily affected by out of order segments.

Scan 1900 does have missing segments over here.
So we can assume that work is in progress.
Certainly, an overflow of data could cause problems, but the HRIT
channel is now 60 -70% greater bandwidth than it would have been
direct from the satellite, and it still seems to have a one minute
data gap at the start of each 15 minute period, when nothing is sent.

Does anyone disagree that the problem was fixed after 1930 last
night?

Cheers,
David


fvalk <fvalk@...>
 

The first I have again was 19:45. Not 19:30 itself. But from there on all
seemed quite OK.

Concerning the full width Africa, I couldn't find it in my database, but it
sounds interesting. Do you have more info Arne?

Ferdinand

---------- Original Message -----------
From: "David Taylor" <david-taylor@blueyonder.co.uk>
To: MSG-1@yahoogroups.com
Sent: Sun, 05 Oct 2003 07:17:18 -0000
Subject: [MSG-1] Re: Segment transmission order (long repost)

This way I noticed an odd one: 1315 HRV (ch12) does show full
image width Africa !

Which day was that, Arne?

> Don't know if the larger amount of data in this image started it
but
> scan 1330 and 1345 were heavily affected by out of order segments.
>
> Scan 1900 does have missing segments over here.
> So we can assume that work is in progress.

Certainly, an overflow of data could cause problems, but the HRIT
channel is now 60 -70% greater bandwidth than it would have been
direct from the satellite, and it still seems to have a one minute
data gap at the start of each 15 minute period, when nothing is sent.

Does anyone disagree that the problem was fixed after 1930 last
night?

Cheers,
David


a_van_belle
 

--- In MSG-1@yahoogroups.com, "fvalk" <fvalk@f...> wrote:
Concerning the full width Africa, I couldn't find it in my
database, but it
sounds interesting. Do you have more info Arne?
Sorry for the delay folks, I do need a second operator for my MSG
station to do all the animations and the reporting !

On 4 October, scan 1315 CH12 (HRV).
It was only one slot.

The "out-of-order" segment problem seems to have started on 3 october
0930 already. The missing segments report does not detect these but
browsing through my images does show.

Greetings,
Arne van Belle


fvalk <fvalk@...>
 

Arne,

Maybe, as you participate in the dongle trial, you get access to different info
than the others. For the whole 4th of October I have only the standard Ch 12
view, including the 13:15 slot.

Ferdinand

---------- Original Message -----------
From: "a_van_belle" <a.van.belle@hccnet.nl>
To: MSG-1@yahoogroups.com
Sent: Sun, 05 Oct 2003 11:03:51 -0000
Subject: [MSG-1] Re: Segment transmission order (long repost)

--- In MSG-1@yahoogroups.com, "fvalk" <fvalk@f...> wrote:
> Concerning the full width Africa, I couldn't find it in my
database, but it
> sounds interesting. Do you have more info Arne?

Sorry for the delay folks, I do need a second operator for my MSG
station to do all the animations and the reporting !

On 4 October, scan 1315 CH12 (HRV).
It was only one slot.

The "out-of-order" segment problem seems to have started on 3 october
0930 already. The missing segments report does not detect these but
browsing through my images does show.

Greetings,
Arne van Belle




Yahoo! Groups Sponsor

ADVERTISEMENT


To unsubscribe from this group, send an email to:
MSG-1-unsubscribe@yahoogroups.com



Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
------- End of Original Message -------


a_van_belle
 

Folks,

The anomaly that I detected in the 4 October, HRV 1315 CH12 was
caused by me !

I had stopped processing in order to manual re-process some raw data
to double check the missing segments we had.
But this way I lost the Prologue, giving the next HRV no offset.

You see, even users are on trial here !

Now that I noticed the diference I have un-ticked the "Lower" Centre
HRV window in the advanced screen of MDM.
This way you can watch the Madagascar region in full HRV res.

As we can read in the admin messages: "The HRV scan region may
periodically be moved", so watch this channel.

Greetings,
Arne van Belle