|
MSG-1/2 via Datamanager v 2.5.2.554
Terence O'Hanlon Smith
Dear all,
if anyone has attempted anything like this, or knows what the problem might be, I would be grateful for some help or advice. I have adjusted the 'receive-ini' file, to David's recommendation, and my Rx machine now has two directories,' received' and 'received56' for MSG 1 and 2 respectively. The Tellicast log shows all 5 channels receiving and completing - either (finished) or (suspended), and the files are correctly directed into the appropriate directories. It all just about worked but, with MSG DM on the one machine coping with both sets of data (and my reticence with Ram drives), it was reporting many lost segments on that processing machine, so I networked in my travelling laptop with its MSG DM (same version) and set it up to process separately only the MSG2 files. The main MSG-1 processing machine is carrying on as normal, with no lost segments, but the second MSG-2 DM is not processing the files from the 'received56' directory. Watching the DM log and the directory in Explorer, at the same time, shows the MSG-2 files appearing and disappearing in the directory, (just as they do with MSG-1 on the other computer), but no processing taking place in DM, although the yellow "LED" does flicker occasionally. I can only think that this is a networking problem, although I have been very careful with the setting up of the source directory ( received56 ), and have even tried specifically mapping it. I am reasonably sure it's not a permissions thing, Explorer has no problem with the directory, but I can't think what else it might be and am presently at a loss. Perhaps someone has another notion? Thanks in anticipation, Terence.
|
|
|
|
Re: MSG-1/2 via Datamanager v 2.5.2.554
Terence O'Hanlon Smith wrote:
Dear all,Terence, If the files are disappearing from the 'received56' folder, something must be deleting them. Any chance it's the second MSG Data Manager deleting them? Any chance you left the second instance of the MSG Data Manager running in the first machine? I would be tempted to say you have the "Delete other satellite" box checked, but that shouldn't delete files in a different directory! I take it the log shows "MSG-2" and not "MSG-1"? Probably something obvious. There is no foreign satellite data being sent as MSG-2 right now, nor any MTP data (Met-5 or Met-7), just LRIT and HRIT. By the way: this would be more appropriate in the SatSignal group. Cheers, David -- SatSignal software - quality software written to your requirements Web: http://www.satsignal.net Email: davidtaylor@writeme.com ___________________________________________________________ Try the all-new Yahoo! Mail. "The New Version is radically easier to use" � The Wall Street Journal http://uk.docs.yahoo.com/nowyoucan.html
|
|
|
|
Re: Recent EUMETCast changes
Mads Olander Rasmussen wrote:
Hi David and listMads, As far as I can tell at the moment, we will alll need to change away from using the catch-all [*] section in recv-channels.ini, and add sections specific to the data we want. Otherwise it seems that the Announcement channel says something like: - there are files for Data Channel 3 on multicast address a.b.c.d:e and if you have the wildcard [*] set, the TelliCast software say to the DVB software: - please send me multicast data for address a.b.c.d:e and if the PID for that data isn't set, then the TelliCast client fails to connect, and hence registers an error. As far as I can understand, though, such an error doesn't cause data loss, so it simply gets recorded in the log file (causing more disk I/O). What you need to do, I believe, is to remove the section [*], and (for the data you want), replace it with sections like (where I made up the directory names): ----------------------------------------------------------- # (for the AVHRR data) [EUMETSAT Data Channel 1] target_directory=D:\Tellique\received\AHVRR\ # # (for MSG-2 HRIT data) [EUMETSAT Data Channel 5] target_directory=D:\Tellique\received\HRIT\ ----------------------------------------------------------- Indeed, that may be the full content of the recv-channels.ini file. In this way, when files for Data Channel 3 come along, the TelliCast client will know that you don't want them, as it can't find a match for "EUMETSAT Data Channel 1" and that it should not try to set up a data connection to: 224.223.222.23:nnnn (which wouldn't work as you are not taking PID 301). There is a list of channels and PIDs here, although I don't know the multicast address for DWDSAT, and if someone has this I could complete the table. http://www.david-taylor.myby.co.uk/wxsat/atovs/index.html#DataChannels Is that any clearer? Does it cure the flashing red icon problem? Cheers, David -- SatSignal software - quality software written to your requirements Web: http://www.satsignal.net Email: davidtaylor@writeme.com ___________________________________________________________ Try the all-new Yahoo! Mail. "The New Version is radically easier to use" � The Wall Street Journal http://uk.docs.yahoo.com/nowyoucan.html
|
|
|
|
Re: Segment Loss
a_van_belle
Hello Ulrich,
Using disks larger than 137 GB on W2k and older XP (before SP1) requires a registry setting and "mainboard compatibility" see www.48bitlba.com Without this you might be able to use the drive but will encounter complete corruption if you try to access beyond the 137 Gb "barrier" Good luck, Arne van Belle --- In MSG-1@yahoogroups.com, "Ulrich G. Kliegis" <Ulrich.Kliegis@...> wrote: ...
|
|
|
|
Re: Recent EUMETCast changes
Mads Olander Rasmussen <mor@...>
Hi David and list
after reading your email (3) David, I realized that I had to update the client software (I was running 2.4.0a). On that machine I only receive NOAA AVHRR data and the trial MSG2 data so I only have enabled PIDs 100 and 500. After updating the software my 'T' changed from consistently pink to changing between pink and red. It claims that I have a 'configuration error' as you also mention (1,b). Does anyone have an idea how to avoid this? As I am only receiving the NOAA AVHRR data and the MSG-2 data I have not edited recv-channels.ini to separate the data from the different channels (so I'm using the [*] wildcard option). Enabling the other PIDs make the 'Configuration Error' messages go away (and the pink 'T' reappears) - but it also increases the load on the machine which was why I disabled PIDs 300 and 301 in the first place! Cheers, Mads 1 - "wrong interface address" messages in the log file.gm8arv@yahoo.co.uk 01-06-2006 10:25 >>> (a) A few of these messages are to be expected when a higher priority transmission from another channel was started, and the opened channel was not used. These are annoying, but can be ignored. They don't signify data loss. (b) The channel has been disabled (PID not selected of channel not selected in recv-channels.ini, then happens frequently). To me (b) sounds like a configuration error. My guess is that we may all need to learn how to edit the recv-channels.ini file to filter in the data we want, rather than just getting everything. 3 - the upgrade of the client is mandatory, as future changes may cause even more problems for the old client software. Cheers, David -- SatSignal software - quality software written to your requirements Web: http://www.satsignal.net Email: davidtaylor@writeme.com
|
|
|
|
Re: Segment Loss
a_van_belle
Hello All,
On my standard RX PC I do not run a firewall nor a virusscanner and do not run defrag. This may not be advisable, yet the system has run from April 2003 in this config without any problem. Why ? My Speedtouch ADSL modem as a hardware firewall actived. I do not surf on the RX PC and disabled messenger service. The 40 GB disk is partitioned in 10 GB C: and 30 GB D:, swap space is set to a fixed size and on D: and all received data is stored on D: So the C drive is very static (apart from the log files and regular updates from David Taylor !) and does not fragment. All data from the D drive is moved to my process PC almost daily, so fragmentation is not an issue. Currently both PC's have been running without segment loss (from May 31 12:45 UTC up to this morning 06:00 UTC). The errors in TC240 recv.log that I have do not concern data that I take and don't cause missing segments. They do show that a client update is mandatory ! Greetings, Arne van Belle
|
|
|
|
Recent EUMETCast changes
Folks,
I've just had an e-mail from EUMETSAT which I may summarise as follows: 1 - "wrong interface address" messages in the log file. (a) A few of these messages are to be expected when a higher priority transmission from another channel was started, and the opened channel was not used. These are annoying, but can be ignored. They don't signify data loss. (b) The channel has been disabled (PID not selected of channel not selected in recv-channels.ini, then happens frequently). To me (b) sounds like a configuration error. My guess is that we may all need to learn how to edit the recv-channels.ini file to filter in the data we want, rather than just getting everything. 2 - increased missing segments If the system performance was borderline before, the new server and client software might push such a system over the limit. Using a RAMdisk for the file database location (i.e. the .FSY file) should resolve these problems. (Arne van Belle's article is here: http://www.david-taylor.myby.co.uk/wxsat/atovs/RAMdisk.pdf) 3 - the upgrade of the client is mandatory, as future changes may cause even more problems for the old client software. 4 - you are encouraged to report problems. The trouble-shooting guide section (A) lists the minimum information that EUMETSAT need. http://www.david-taylor.myby.co.uk/wxsat/atovs/EUMETCast-trouble-shooting-guide.html Cheers, David -- SatSignal software - quality software written to your requirements Web: http://www.satsignal.net Email: davidtaylor@writeme.com ___________________________________________________________ Now you can scan emails quickly with a reading pane. Get the new Yahoo! Mail. http://uk.docs.yahoo.com/nowyoucan.html
|
|
|
|
Re: Segment Loss
Alan Sewards <asewards@...>
Hello Arne,
toggle quoted messageShow quoted text
Thank you for your very useful comments. Responses below: You are right that the \receiving folder on the C: drive is a left over from before I implemented the RAMdisk - the data in it is over a year old. The Recycled folder on the RAMdisk is created by Norton Utilities. I have tried to stop it allotting space for recycling on the Z: drive but it tries again after a reboot! If it persists, I'll uninstall the lot. Its useful, but not if it gets in the way of doing what you want to do! I will keep 0.fsy on the RAMdisk with its existing directory. I increased the RAMdisk size to 64 MB to hold the update files and larger .fsy. I may increase it again if necessary. I have suppressed the \receiving/temp folder and specified z:\received for the received data. It all seems to be working OK. I'm still getting the ' Cannot join channel 1 - missing key messages' but some channel 1 data is getting through. Best regards - Alan a_van_belle wrote:
Hello Alan, --
Alan Sewards email: alan.sewards@free.fr b.sewards@libertysurf.fr web site: http://asewards.free.fr
|
|
|
|
Re: Segment Loss
ferdinand valk <fvalk@...>
I have been using Diskeeper 10 for about two months now on (amongst others)
a four disk machine running 24/7 (XP pro) using the smart defragmentation settings. When I look at the Volume Map of three of the drives, they always appear as fully defragmented whereas the most active disk where the EUMETCast files are stored appears as somewhat fragmented. When I do a manual defragmentation on that disk, the program runs for about a minute and then all is blue again. This means in my view that Diskeeper does do a good job in keeping things in order and that the visual presentation in the Volume Map looks worse than in reality it is. I have no secondary, or FAAST job set. I have never been able to associate missing segments with running the program. It seems that in my case there has been no increase in missing segments since I am using Diskeeper. Since I have no experience with predecessors of version 10 I can not judge if it is worth to upgrade to it, but this version does wonders to my systems. Cheers, Ferdinand _____ From: MSG-1@yahoogroups.com [mailto:MSG-1@yahoogroups.com] On Behalf Of seggins2000 Sent: Thursday, June 01, 2006 6:14 AM To: MSG-1@yahoogroups.com Subject: [MSG-1] Re: Segment Loss ==> To any one who has not already upgraded to Diskeeper10, my advice is don't bother if,like me, you run 24/7. I have put all this to the Diskeeper Corporation but they find themselves unable to help...or reply to my emails. Cheers John Say
|
|
|
|
Re: Segment Loss
Alan Sewards <asewards@...>
David,
toggle quoted messageShow quoted text
Thank you for your comments. I have Zone Alarm on the machine in question for the same reasons as you, but there is no AV as I do not get mail on the machine. I do, however, have Norton Utilities (an old one!) on there, and with it goes the Norton Protected Recycle Bin, and sure enough, it had got to the Z: drive and allotted space for recycling. I have now turned this off. I don't think any of the other Utilities affects the reception process, but I'll check further. It was smart of you to catch that one, thanks! BTW, the Channel 1 error messages continue. I have alerted OPS. Best regards - Alan David J Taylor wrote:
Alan Sewards wrote:David,[] --
Alan Sewards email: alan.sewards@free.fr b.sewards@libertysurf.fr web site: http://asewards.free.fr
|
|
|
|
Re: Segment Loss
seggins2000 <seggins@...>
By the way, on the main PC (which is a receive-only PC), I runDiskeeper to defrag each of the four partitions once a week, on differentdays, at different times. This can also cause the odd missing segment,perhaps one a month (rough guess). I can correlate the time and day of themissing segment with the defrag of the disk with the \receiving\directories. David, I use Diskeeper too. I have always been impressed by the program so I updated to Diskeeper 10. This version has many more sophisticated methods of defragmentation than previous versions. I find, if I try to implement its more 'thorough' methods (e.g. the recommended one)it completely fails to keep up. My main storage disk fragments very quickly as the files come in from MDM and it seemed a good idea to set up Diskeeper to smart-defragment the disk on its recommended setting. The result was that Diskeeper was running almost continuously, plodding about shifting bits and pieces while the disk was fragmenting more and more. I changed the setting away from 'smart' to a fixed time and the defragmentation method to 'simple' and it now keeps up. To any one who has not already upgraded to Diskeeper10, my advice is don't bother if,like me, you run 24/7. I have put all this to the Diskeeper Corporation but they find themselves unable to help...or reply to my emails. Cheers John Say
|
|
|
|
Re: Segment Loss
Peter Hayes wrote:
[] Does the Eumetcast machine ever need to go anywhere near the internetGood question. As my PCs sit on my small network, which is connected to the Internet via a router, I do protect them as a matter of routine with (in my case) CA's eTrust anti-virus and Zone Alarm. No Spyware scanners. I don't find any problems with Zone Alarm, and I configure the anti-virus not to touch any of the TelliCast related directories. No routine anti-virus scans, either. By the way, on the main PC (which is a receive-only PC), I run Diskeeper to defrag each of the four partitions once a week, on different days, at different times. This can also cause the odd missing segment, perhaps one a month (rough guess). I can correlate the time and day of the missing segment with the defrag of the disk with the \receiving\ directories. Cheers, David -- SatSignal software - quality software written to your requirements Web: http://www.satsignal.net Email: davidtaylor@writeme.com ___________________________________________________________ All New Yahoo! Mail � Tired of Vi@gr@! come-ons? Let our SpamGuard protect you. http://uk.docs.yahoo.com/nowyoucan.html
|
|
|
|
Re: Segment Loss
Peter Hayes <pedroh@...>
On 31 May 2006, at 22:43, David J Taylor wrote:
Alan Sewards wrote:Does the Eumetcast machine ever need to go anywhere near the internet - ie, does it need any security software in the first place?David,[] Peter
|
|
|
|
Re: Segment Loss
On 31 May 2006 at 19:51, Douglas Deans wrote:
Strange thing is David, the segment loss is always on the .30 scan andYou might try to monitor the network activities, preferably on the processing machine. Fetching data via the LAN also eats ressources in terms of disk access and -blockage. If you sync both computers to the same time server, you may get pretty explaining information on what's going on at the critical times. Running the monitoring process on the receiving machine may change its behaviour so much that the error gets damaged... :) Did anyone who is experiencing segment losses play with the priority parameter in rec-channels.ini? At least, I can report that most parts of my newly refurbished (now 1 GHz-) receiver work flawlessly now. Only the 250 GB disk refuses to cooperate, probably due to the FAT32 format under win2k. Yaddahyaddahyaddah, I know. But I will have to salvage the data that are still on it from its previous life before reformatting. Tschüß, U. -- ----------------------------------------------------------- Dr.med.Ulrich G. Kliegis Business: http://www.nordcom.de Phone (x49) 431 33 11 44 Fax (x49) 431 33 11 46 Don't flame me, I'm only the keyboard player...
|
|
|
|
Re: Segment Loss
Alan Sewards wrote:
David,[] Best regards - AlanAlan, I think Arne has answered your questions - the only oddity I spotted was "NPROTECT" - if that's something to do with Norton please keep /anything/ which could interfere with the clean flow of data /well away/ from anything to do with EUMETCast reception, be it anti-virus, Norton recycle bin, "RAM-boosters" etc. etc. Summarise your report to Ops, and try to include as much configuration data as possible. The more information they can get the better. Cheers, David -- SatSignal software - quality software written to your requirements Web: http://www.satsignal.net Email: davidtaylor@writeme.com Send instant messages to your online friends http://uk.messenger.yahoo.com
|
|
|
|
Re: Segment Loss
Douglas Deans wrote:
[] Oh yes, a little longer than most if you get my drift !Well, I suggest reporting the problem to OPS in as much detail as possible. It will get fed to the folk in EUMETSAT responsible for EUMETCast. Cheers, David -- SatSignal software - quality software written to your requirements Web: http://www.satsignal.net Email: davidtaylor@writeme.com ___________________________________________________________ The all-new Yahoo! Mail goes wherever you go - free your email address from your Internet provider. http://uk.docs.yahoo.com/nowyoucan.html
|
|
|
|
Re: Segment Loss
a_van_belle
Hello Alan,
The Z:\receiving\temp folder is created automatically by Tellicast. Your ....\receiving on the C: drive below T-Systems\BusinessTV-IP\ is a "left over" before you configured the RAMdisk and should contain old data (check the file dates). There should not be a recycled folder on RAMdisk. Some Windows process must have deleted something on Z:, causing the recycled to be created. After a system reboot it should be gone ! Only last week when we discussed the RAMdisk with Eumetsat a new "insight" came up. There is no doubt, the 0.FSY file should be on RAMdisk ! And the article on the RAMdisk does tell you that temp_directory is specified as =Z:\receiving/temp But when receiving the updates your RAMdisk should hold the larger 0.FSY and the temporary 25 Mb update file ! As David said after the file is received completely it has to be moved to the received folder. A move to a different drive is the same as a "copy and if successful then delete" in Windows ! Placing the receiving/temp on the same drive as the received folder simply turns a move command into a "rename path" and causes less IO. As the RAMdisk is very fast I don't think this does worsen performance , but you never know. And to complicate this matter even more, we don't really need the receiving/temp folder. Tellicast can write directly to the received folder. The temp folder is only to be used if you have a image decoder that tries to access the received file before it is completed. David's MDM does wait for the file being completed. At this moment (but we are in "Beta-phase" !) we can recommend to disable the temp_directory line(s) in your ini files. Please note that in TC244 this entry may be called tmp_directory Greetings, Arne van Belle --- In MSG-1@yahoogroups.com, Alan Sewards <asewards@...> wrote: OPS, I find that they started on 30 May at 0902 and there are now severalhave \receiving on the C: drive below T-Systems\BusinessTV-IP\, andand RECYCLED. The former contains a folder named temp (which is empty) andinto receiving until it has the complete segment, then it moves (or copies)went for it! Your comment on 'move' vs 'copy' presupposes that tqrecv issame drive depends on this. Should we be using a RAMdisk for both folders?
|
|
|
|
Re: Segment Loss
Alan Sewards <asewards@...>
David,
toggle quoted messageShow quoted text
Thanks for your comments. I will report the error messages to OPS, I find that they started on 30 May at 0902 and there are now several hundred of them! I am puzzled by your remarks on the \receiving file setup. I find I have \receiving on the C: drive below T-Systems\BusinessTV-IP\, and \receiving is on the RAMdrive, Z: Z contains two folders - receiving and RECYCLED. The former contains a folder named temp (which is empty) and the 0.fsy file. The latter (Recycled) contains a folder named NPROTECT, which in turn contains a file named NPROTECT.LOG, which is 632k and attempts to view it are met with the statement that it is in use. I did not tell the system to create this recycled folder or its contents, nor did I tell it to create the temp folder. My setup of this structure dates from about a year ago, when I edited recv.ini and recv-channels.ini on the basis of what I understood at the time. recv.ini specifies file_database_directory=Z:\received (that is the RAMdisk). Recv-channels.ini contains target_directory commented out, defaulting to ./received, as it actually is, as mentioned above. temp_directory is specified as =Z:\receiving/temp. [there seems to be a lot of confusion in this software between the UNIX / and the Windows\, with both appearing at times, as in the above specification!] As I understand it, tqrecv puts the files with bits of segments into receiving until it has the complete segment, then it moves (or copies) it to the received folder. The idea of using a RAMdisk seemed to arise over this collection process, but I am now puzzled as to why we all went for it! Your comment on 'move' vs 'copy' presupposes that tqrecv is smart enough to distinguish between the need for one or the other depending on the source and destination drives, which may be so but I doubt it. Your point that \received and \receiving should be on the same drive depends on this. Should we be using a RAMdisk for both folders? And will be it make any difference? Best regards - Alan The comments relating to David J Taylor wrote:
Alan Sewards wrote:David and Ian, --
Alan Sewards email: alan.sewards@free.fr b.sewards@libertysurf.fr web site: http://asewards.free.fr
|
|
|
|
Re: Segment Loss
Douglas Deans <dsdeans@...>
Douglas Deans wrote:Oh yes, a little longer than most if you get my drift !DouglasI have been up and running since 07.00 UTC this morning and so far Strange thing is David, the segment loss is always on the .30 scan and always only one HRIT segment. I have checked and nothing else is happening at that time (on the receive computer). Only started yesterday. Bit of a puzzle. Tomorrow I may go back to the 'other' data and see if things get worse. Thanks Douglas.
|
|
|
|
Re: Segment Loss
Alan Sewards wrote:
David and Ian, I would suggest you report those messages to EUMETSAT, Alan. I've already done so, but it may help to know that the older client software also produces similar errors. I've only seen one such message today. By the way: there is (probably) no need to use the "temp" files mechanism, even with a RAMdisk, when using my software, and quite possibly using "temp" files will simply result in a greater I/O load. If you need to use "temp", it should be on the same physical disk as the "received" files, so that the final step in the receiving operation becomes a "move" (i.e rename in disk directory), rather than a "copy", incurring extra I/O. Cheers, David -- SatSignal software - quality software written to your requirements Web: http://www.satsignal.net Email: davidtaylor@writeme.com ___________________________________________________________ All new Yahoo! Mail "The new Interface is stunning in its simplicity and ease of use." - PC Magazine http://uk.docs.yahoo.com/nowyoucan.html
|
|
|
