Topics

Some thoughts towards the future....


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

From the "Pytroll/SatPy for EUMETCast" thread.
~~~~~~~~~~~~~~~~~~~~

With all the attention on the above at the moment, spare a thought for those of us who can only rely on Windows programs for our hobby.

David, who has been the backbone of this hobby for many many years, has now retired and rightly pursuing his own hobby interests. This is leaving a gap for new programs for new data and new satellites, which I do not see being filled in the coming months/years and therefore will put the hobby at risk.

Let me make it quite clear that I for one would gladly pay for any new programs.

Regrettably I have zero knowledge of computer programming ( what is a Windows Editor and a command line ??? ] so that route is completely out of the question.

Regards
Ian.
======================================

Ian,

Some thoughts towards the future....

You are not alone! A lot of people ask for a "click-and-go" software. I use the free InnoSetup software for creating such a program, and it has a lot of scripting options (based on a Pascal-like language) which make it very versatile:

https://jrsoftware.org/isinfo.php

So rather than the user having to install and configure multiple unfamiliar packages perhaps a large proportion of the work could be handled by an automated InnoSetup script? Yes, it can call other installers. Perhaps folks like Douglas, Christian, Graham and Ernst who have the knowledge between them might look at creating an install script to ease the task?

Rob has already produced some helper programs to edit text scripts using a GUI and I could perhaps help with that if the requirements were known. I could attempt to make similar software open-source in Delphi so that it could also be compiled on Lazarus/FreePascal enabling Linux and Mac support.

BTW: I was interested to see that the Pytroll approach cannot as yet produce the results achieved with my GeoSatSignal program.

To clarify, all of my software is still available and support remains the same as ever. What I don't propose to do is to develop further software for the following generation of polar and geostationary birds as I may not be around to or capable of supporting it (2021-2041) so it would be unfair to users.

In any case, much of the interesting data is readily available from the Internet, so not much point in re-inventing the wheel? For Sentinel images, start here: https://twitter.com/Pierre_Markuse

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Ernst Lobsiger
 

On Wed, May 13, 2020 at 07:51 AM, David J Taylor wrote:
Some thoughts towards the future....
Ian,

you name the problem. It's a fact that this hobby has become more and more technical ever since
the change from DVB-S to DVB-S2 in 2014. If using an Editor (which is just right click the Desktop,
say New textfile and then doubleclick this file) or opening a CMD window (bottom left search for CMD)
is, as you put it, "completely out of the question", then indeed your EUMETCast hobby is at risk.


Though I never had a Windows EUMETCast receiver I certainly witnessed what David Taylor has done for years
for people running such receivers and maybe his software. This software will not stop working the very day
David will not be able to maintain it anymore. There is EUMETCastView but Hugo is not 20 years old either.
I'm also aware that running a full blown 2 PC receiver/processor chain under GNU/Linux is not commonplace.


If you look or know where EUMETCast is technically going e.g. with upcomming MTG satellites

https://www.eumetsat.int/website/home/Satellites/FutureSatellites/MeteosatThirdGeneration/EvolutionstoEUMETCastservicesfortheMTGera/index.html

and how EUMETSAT thinks we should already work today (going from David Taylor to Data Tailor !)

https://www.eumetsat.int/website/home/Data/DataDelivery/NewPilotDataServices/TheDataTailor/index.html

then learning a few things to get some available Pytroll/Satpy scripts running is indeed "childs play".

With all that processing software already available, I doubt that anybody will be interested in producing new
professional EUMETCast point and click software for an acceptable price in the future. Well, "Et it wi et it!".


Best Regards,
Ernst


Alan Banks <alanjamesbanks@...>
 

On 13/05/2020 17:06, Ernst Lobsiger via groups.io wrote:
On Wed, May 13, 2020 at 07:51 AM, David J Taylor wrote:
Some thoughts towards the future....
Ian,

you name the problem. It's a fact that this hobby has become more and more technical ever since
the change from DVB-S to DVB-S2 in 2014.


Ernst,

Not only are the software developers no longer spring chickens neither are the users. Since moving to this house 8 yrs ago, my dish cabling and LNB remain unchanged. I was lucky with the move to DVB-S2 in that I had a reasonable signal straight away.

Recent weather indicates I need to 'house keep' the system. This involves getting to the dish which is inaccessible without a scaffold tower.

I don't know whether to bother as I can't do it and my interest in the images has certainly waned. If the Ayecka were to die I would probably quit altogether in spite of having an unused spare.

My current interests of model railways, electronics for model railways and 3D printing keep me fully occupied.

I do have three instances of MSG Data Manager running 24/7 on my 4K screen, but rarely do I investigate further. I only take Basic Service as my Link Margin is too low for HVS. I'd need a bigger dish (currently 80cm) and as its on the front of the house I'm not risking upsetting my wife by putting a 120cm one there!

Any major investment in the future is to my mind better spent on an O Gauge Locomotive!

So one can see that if anything breaks in the automatic 'chain' that runs itself, I'm likely to quit the hobby.

Regards

Alan



Hugo
 

Hi Ernst, David and Ian,

I am indeed not 20 years old Ernst ... :) 
I want to draw the attention to the concept of open source. Even when the creator of the software is not involved anymore in the development in his/hers software,
others can take it over. For many projects, it can even result in a group effort like Pytroll. And that is the real advantage of open source projects. (also in Windows !)
For commercial applications, income can be generated by providing support.
My two cents, ....

Hugo


Douglas Deans
 

On 13/05/2020 18:06, Ernst Lobsiger via groups.io wrote:
On Wed, May 13, 2020 at 07:51 AM, David J Taylor wrote:
Some thoughts towards the future....
Ian,
you name the problem. It's a fact that this hobby has become more and more technical ever since
the change from DVB-S to DVB-S2 in 2014. If using an Editor (which is just right click the Desktop,
say New textfile and then doubleclick this file) or opening a CMD window (bottom left search for CMD)
is, as you put it, "completely out of the question", then indeed your EUMETCast hobby is at risk.
Though I never had a Windows EUMETCast receiver I certainly witnessed what David Taylor has done for years
for people running such receivers and maybe his software. This software will not stop working the very day
David will not be able to maintain it anymore. There is EUMETCastView but Hugo is not 20 years old either.
I'm also aware that running a full blown 2 PC receiver/processor chain under GNU/Linux is not commonplace.
If you look or know where EUMETCast is technically going e.g. with upcomming MTG satellites
https://www.eumetsat.int/website/home/Satellites/FutureSatellites/MeteosatThirdGeneration/EvolutionstoEUMETCastservicesfortheMTGera/index.html
and how EUMETSAT thinks we should already work today (going from David Taylor to Data Tailor !)
https://www.eumetsat.int/website/home/Data/DataDelivery/NewPilotDataServices/TheDataTailor/index.html
then learning a few things to get some available Pytroll/Satpy scripts running is indeed "childs play".
With all that processing software already available, I doubt that anybody will be interested in producing new
professional EUMETCast point and click software for an acceptable price in the future. Well, "Et it wi et it!".
Best Regards,
Ernst
========================================================================================================

Ernst you are right.
I am very similar to Ian (actually his identical twin in real life !) in that I have no experience with programming in any way and 60 + years ago, at school, domestic computers did not even exist.

But recognising that I may have to get future new data in other ways (ex FY-3D which I now have) I decided to give the pytroll/satpy a go. I have degrees in electrical engineering and maths and that has helped my application in new learning even at my age but for many it would be very difficult.
I still mainly use a nice simple GUI based weather satellite system. Actually two, David's and Hugo's but I now feel more flexible and prepared for the future with my slowly increasing knowledge of Pytroll/satpy. Not to mention personal satisfaction.

Regards,
Douglas.


geojohnt@...
 

Alan, David, Ian, Ernst and All,

Yes, that's right.
I'm now 77 and if 'it' isn't 'click (plug) and play' - is that the phrase? - I'm lost.
Command line, bios, registry edit et al ????
Having to go beyond just clicking and option or setting up simple parameters is a no-no.   

Sad but true.
As a teenager on the south coast I used to modify valve TV's and tuner units to receive continental DX TV.
In 1980 whilst based in New York working for BBC TV, I purchased all the equipment - receiver, downconverter, LNA and wave-guide for C-band satellite TV and purchased a 2 m dish in the UK.
This was to receive the Soviet Ghorizont satellite TV transmissions in SECAM - requiring a multi standard colour TV.

A few years later I graduated to TV reception from the European ECS Ku-band satellite - 1.8 m dish, professional head-end receiver and LNA - ~ £4,000 - in those days.

In 1986 I purchased a Microwave modules Meteosat and NOAA complete receiving system - WEFAX and APT which cost £2,360 - in those days.
This system used a Framestore and a phased array radiator 1.6 GHz antenna for Meteosat with GaASFET amplifiers for both Meteosat and NOAA 137 MHz APT with a turnstile antenna.

Not long afterwards I 'found' RIG, purchased one of their 1 m dishes with coffee can and built Les Currington's LOK board for the framestore.
Then built the Ray Godden 137 MHz receiver.

Then bought the Dartcom 137 MHz receiver board and RF amplifier,  wired and boxed them and purchased a 368 computer and the Dartcom weather satellite data processing software.

Became secretary of RIG in the 90's and the rest is history.
Though DVB-S EUMETCast with David's excellent software was not a problem.
DVB-S2 added a bit of a new learning curve.

Reading all the post's about  Pytroll/SatPy for EUMETCast just leaves me cold, full of fear for the future, but full of wonder for those experts who have been experimenting and developing it.

However, if the future isn't 'plug and play' then I'll be turning my dish into a bird bath. Relaxed


Regards,
John.

.
+++++++++++++++++++++++++



-----Original Message-----
From: Alan Banks via groups.io <alanjamesbanks@...>
To: MSG-1@groups.io
Sent: Wed, 13 May 2020 19:26
Subject: Re: [MSG-1] Some thoughts towards the future....

On 13/05/2020 17:06, Ernst Lobsiger via groups.io wrote:
On Wed, May 13, 2020 at 07:51 AM, David J Taylor wrote:
Some thoughts towards the future....
Ian,

you name the problem. It's a fact that this hobby has become more and more technical ever since
the change from DVB-S to DVB-S2 in 2014.

Ernst,
Not only are the software developers no longer spring chickens neither are the users.
_._,_._,_


Ernst Lobsiger
 

On Thu, May 14, 2020 at 09:47 AM, <geojohnt@...> wrote:
However, if the future isn't 'plug and play' then I'll be turning my dish into a bird bath. 
Dear All

Just out of curiosity I made my first MTG-1 image. With PyTroll/Satpy it's possible already.
Of course the data is pure test data that has been available since spring 2019 from EUMETSAT.

Data for one repeat cicle of the fdhsi instrument (decompressed *.nc) has 6'274'072'576 Bytes!
The raw full earth scan image has 11136 x 11136 pixels (my monitor is just Full HD 1920 x 1080)
The raw *.png has 42'592'637 Bytes while my with Image Magick (IM) annotated *.jpg still has
40'110.414 Bytes. When I wanted to reduce it (to attach it below) paint under Windows 10 could
not even read it (PC has 16GB RAM). So I resized it to 5% x 5% using IM convert under GNU/Linux.

EUMETSAT is talking about a 10 fold data stream via EUMETCast with several transponders on
possibly different downlink satellites. They also recommend QUATTRO LNBs plus multi swiches.
This means 4 cables to each of up to 2 dishes. And of course it means 3 or more DVB routers.

https://www.eumetsat.int/website/home/Satellites/FutureSatellites/MeteosatThirdGeneration/EvolutionstoEUMETCastservicesfortheMTGera/index.html

TBH I do not yet know whether I will follow this. I started with NOAA APT via a crossed dipole
and Meteosat SDUS with a 2.5m dish from WW2. The reception HW/SW was UKW-Technik on MS-DOS 3.3.

Unless you want to invest very heavily in HW you better stay with what you have right now and hope
Meteosat-11 will work for a while. Then follow John ".. I'll be turning my dish into a bird bath.".

The only "affordable" solution I can imagine right now is the new generation of DVB-S2X cards as

https://www.tbsdtv.com/products/tbs6909-x-dvb-s-s2-s2x-octa-tuner-pcie-card.html

and

https://digitaldevices.de/produkte/dvb-komponenten/max-sx8/

These cards have the multiswich built in and take up to 8 transponders. But the new ST chip
used is *VERY* complex and AFAIK there are *NO* drivers that work with EUMETCast up to now.
So *DON'T* buy such a card unless someone has definitely shown it works fine for EUMETCast.
Needless to say that such a solution for EUMETCast reception will never be "plug and pray".

That said only very few of us will probably follow where full EUMETCast is going. But the
Basic Service has already much to offer. And there is still Hugo's proposed club to join :-).


Regards,
Ernst

P.S. It's still possible to build a good Basic Service only EUMETCast receiver with a second
hand SkyStar 2 eXpress HD PCIe card for almost nothing. But it's not "plug and pray" either.


Christian Peters
 

Ernst and all,

I hope you all will not retire not any time soon. We need the "old" experts to get the new interesting times working.
I see it's a lot of data which will come with MTG...but also interesting new challenges.
We will get "HR" on all channels and there will be e.g. a lightning imager which will give us new data
and nice combined composites and movies with it.
But I see that the the data accumulation is really big and to buy 4x NOVRA receivers is not an option
for amateurs. So the only way to go would be the new ST chip build in in the two cards Ernst mentioned.
I hope you will stay on board and we will get a working PCIe card which could handle the three or maybe four
transponders with one card and we will get a setup for Linux and Win.
I even start with APT reception and a Turnstile antenna some years ago....just a little time before wxtoimg support was dropped.
So I had the luck to get into it before it was gone. So I google for sat images I discovered Davids site and started with his programs.
I was happy that so many of them for all use cases and satellites are available to get wonderful sat pictures in such an easy way. Thanks again David!
As I even like to try new things I first switched to a Linux TC receiver (thanks to Ernst again!) and then started to do some basic Pytroll coding and I think it's
the way to go as the new data will be in the future not be processed by Davids software. Maybe Hugo will update his ECV software!?
So in the end I had lucky times to get into it and to get the good times to start with BAS, HVS-1 and HVS-2 and the experts here!

Maybe Ernst as an hardware expert or some of the distinguished other experts could push TBS and digitaldevices to get the card working with EUMETCAST and even Eumetsat to think about their EUMETCAST amateur users to develop a solution for their enthusiastic Eumetcast users...?! 

So you all can join Hugo's proposed club ... but please don't turn your dish into a birds bath but stay nosy and don't leave the ship in the next 10 years...!  :-D

Regards,

Christian


Am 17.05.20 um 14:57 schrieb Ernst Lobsiger via groups.io:

On Thu, May 14, 2020 at 09:47 AM, <geojohnt@...> wrote:
However, if the future isn't 'plug and play' then I'll be turning my dish into a bird bath. 
Dear All

Just out of curiosity I made my first MTG-1 image. With PyTroll/Satpy it's possible already.
Of course the data is pure test data that has been available since spring 2019 from EUMETSAT.

Data for one repeat cicle of the fdhsi instrument (decompressed *.nc) has 6'274'072'576 Bytes!
The raw full earth scan image has 11136 x 11136 pixels (my monitor is just Full HD 1920 x 1080)
The raw *.png has 42'592'637 Bytes while my with Image Magick (IM) annotated *.jpg still has
40'110.414 Bytes. When I wanted to reduce it (to attach it below) paint under Windows 10 could
not even read it (PC has 16GB RAM). So I resized it to 5% x 5% using IM convert under GNU/Linux.

EUMETSAT is talking about a 10 fold data stream via EUMETCast with several transponders on
possibly different downlink satellites. They also recommend QUATTRO LNBs plus multi swiches.
This means 4 cables to each of up to 2 dishes. And of course it means 3 or more DVB routers.

https://www.eumetsat.int/website/home/Satellites/FutureSatellites/MeteosatThirdGeneration/EvolutionstoEUMETCastservicesfortheMTGera/index.html

TBH I do not yet know whether I will follow this. I started with NOAA APT via a crossed dipole
and Meteosat SDUS with a 2.5m dish from WW2. The reception HW/SW was UKW-Technik on MS-DOS 3.3.

Unless you want to invest very heavily in HW you better stay with what you have right now and hope
Meteosat-11 will work for a while. Then follow John ".. I'll be turning my dish into a bird bath.".

The only "affordable" solution I can imagine right now is the new generation of DVB-S2X cards as

https://www.tbsdtv.com/products/tbs6909-x-dvb-s-s2-s2x-octa-tuner-pcie-card.html

and

https://digitaldevices.de/produkte/dvb-komponenten/max-sx8/

These cards have the multiswich built in and take up to 8 transponders. But the new ST chip
used is *VERY* complex and AFAIK there are *NO* drivers that work with EUMETCast up to now.
So *DON'T* buy such a card unless someone has definitely shown it works fine for EUMETCast.
Needless to say that such a solution for EUMETCast reception will never be "plug and pray".

That said only very few of us will probably follow where full EUMETCast is going. But the
Basic Service has already much to offer. And there is still Hugo's proposed club to join :-).


Regards,
Ernst

P.S. It's still possible to build a good Basic Service only EUMETCast receiver with a second
hand SkyStar 2 eXpress HD PCIe card for almost nothing. But it's not "plug and pray" either.


stevens312
 

Hi all, quite strange Ernst you mention TBS-6909X Tuner, I started to experiment with same in co-operation with TBS about 18 months ago, but due to other priorities we did not complete all testing, I will talk with Bob at TBS and see if we can reignite his interest in perfecting that particular Tuner, if we are to stay with this hobby in the future, I am now 80 years young so my brain is also getting tired but let's see what we can do regarding this Tuner. Best Regards to all. Mike G4CFZ.


geojohnt@...
 

Hello Ernst,

Thanks for your very interesting and informative email.
And the three timely links you included.

Well, never mind the complexity of the new RF requirements, there is the incredible amount of data to be dealt with.
Add to that Metop 2nd generation and ones going to need a super-computer - and an awful lot of time to actually sort through all the data/imaging.

Of course we had to cancel this years GEO visit to EUMETSAT - but actually a visit next year may be far more useful when more operational requirements may be in place.  
Particularly new transponders on other satellite?

I do like your phrase 'plug and pray.'

That just about sums up my feelings about what lies ahead. Relaxed

Best wishes,
Stay safe,
John.

++++++++++++++++++++++++++


-----Original Message-----
From: Ernst Lobsiger via groups.io <ernst.lobsiger@...>
To: MSG-1@groups.io
Sent: Sun, 17 May 2020 13:57
Subject: Re: [MSG-1] Some thoughts towards the future....

On Thu, May 14, 2020 at 09:47 AM, <geojohnt@...> wrote:
However, if the future isn't 'plug and play' then I'll be turning my dish into a bird bath. 
Dear All

Just out of curiosity I made my first MTG-1 image. With PyTroll/Satpy it's possible already.
Of course the data is pure test data that has been available since spring 2019 from EUMETSAT.

Data for one repeat cicle of the fdhsi instrument (decompressed *.nc) has 6'274'072'576 Bytes!
The raw full earth scan image has 11136 x 11136 pixels (my monitor is just Full HD 1920 x 1080)
The raw *.png has 42'592'637 Bytes while my with Image Magick (IM) annotated *.jpg still has
40'110.414 Bytes. When I wanted to reduce it (to attach it below) paint under Windows 10 could
not even read it (PC has 16GB RAM). So I resized it to 5% x 5% using IM convert under GNU/Linux.

EUMETSAT is talking about a 10 fold data stream via EUMETCast with several transponders on
possibly different downlink satellites. They also recommend QUATTRO LNBs plus multi swiches.
This means 4 cables to each of up to 2 dishes. And of course it means 3 or more DVB routers.

https://www.eumetsat.int/website/home/Satellites/FutureSatellites/MeteosatThirdGeneration/EvolutionstoEUMETCastservicesfortheMTGera/index.html

TBH I do not yet know whether I will follow this. I started with NOAA APT via a crossed dipole
and Meteosat SDUS with a 2.5m dish from WW2. The reception HW/SW was UKW-Technik on MS-DOS 3.3.

Unless you want to invest very heavily in HW you better stay with what you have right now and hope
Meteosat-11 will work for a while. Then follow John ".. I'll be turning my dish into a bird bath.".

The only "affordable" solution I can imagine right now is the new generation of DVB-S2X cards as

https://www.tbsdtv.com/products/tbs6909-x-dvb-s-s2-s2x-octa-tuner-pcie-card.html

and

https://digitaldevices.de/produkte/dvb-komponenten/max-sx8/

These cards have the multiswich built in and take up to 8 transponders. But the new ST chip
used is *VERY* complex and AFAIK there are *NO* drivers that work with EUMETCast up to now.
So *DON'T* buy such a card unless someone has definitely shown it works fine for EUMETCast.
Needless to say that such a solution for EUMETCast reception will never be "plug and pray".

That said only very few of us will probably follow where full EUMETCast is going. But the
Basic Service has already much to offer. And there is still Hugo's proposed club to join :-).



Regards,
Ernst


P.S. It's still possible to build a good Basic Service only EUMETCast receiver with a second
hand SkyStar 2 eXpress HD PCIe card for almost nothing. But it's not "plug and pray" either.


Ernst Lobsiger
 

On Mon, May 18, 2020 at 12:26 PM, <geojohnt@...> wrote:
Add to that Metop 2nd generation and ones going to need a super-computer - and an awful lot of time to actually sort through all the data/imaging.
John,

that's a minor problem. Here is "SuperTux 3", the third Beowulf Cluster I built.
Used for aircraft noise calculations so far. But with Corona no aircraft traffic
means no noise,no contracts. So it's free for MTG-1,  Metop 2nd generation ...

Cheers,
Ernst


Graham Woolf
 

Hi Ernst

Where can I find the link for the python scripts you used for MTG-1

I would quite like to have a go

Kind Regards

Graham


Ernst Lobsiger
 

On Fri, May 22, 2020 at 07:21 AM, Graham Woolf wrote:
I would quite like to have a go
Graham,

I did it under GNU/Linux with some Quick&Dirty changes to an advanced msg4.py script.
Your fastest track under Windows 10 is to start with a working copy of your msg4.py script.

Download a full uncompressed 10 Minutes RepeatCicle (RC) of test data here (RC 72 is about 12:00)

ftp://ftp.eumetsat.int/pub/OPS/out/test-data/Test-data-for-External-Users/MTG_FCI_Test-Data/FCI_L1C_24hr_Test_Data_for_Users/1.1/UNCOMPRESSED/

Take this RC directory as segdir. With this directory you directly make a "files = glob(segdir  +  "/*")  to get all these *.nc in the filelist files[].

global_scene = Scene(filenames = files, reader = 'fci_l1c_fdhsi')

Instead of IM "-rotate 180" you have to use "-flip -equalize". Not sure IM under Windows 10 will do that: I had to allow IM for more
memory usage under GNU/Linux. In any case you should get the (flipped) *.png without annotation in the respective tmp directory.

Good Luck
Ernst


geojohnt@...
 

Hello Ernst,

Gosh, that is impressive.
Is it 15 computers?

I live around 20 km from Heathrow Airport on the east side - one of Europe's busiest airports.
Very few planes flying and very little noise.
Add to that very little blown in pollution from the airport + very little local traffic pollution - clean air for all!

No wonder the birds are singing so loudly.

Best wishes,
John.

+++++++++++++++++  


-----Original Message-----
From: Ernst Lobsiger via groups.io <ernst.lobsiger@...>
To: MSG-1@groups.io
Sent: Mon, 18 May 2020 22:06
Subject: Re: [MSG-1] Some thoughts towards the future....

On Mon, May 18, 2020 at 12:26 PM, <geojohnt@...> wrote:
Add to that Metop 2nd generation and ones going to need a super-computer - and an awful lot of time to actually sort through all the data/imaging.
John,

that's a minor problem. Here is "SuperTux 3", the third Beowulf Cluster I built.
Used for aircraft noise calculations so far. But with Corona no aircraft traffic
means no noise,no contracts. So it's free for MTG-1,  Metop 2nd generation ...

Cheers,
Ernst
_._,_._,_



Ernst Lobsiger
 

On Fri, May 22, 2020 at 07:21 AM, Graham Woolf wrote:
Where can I find the link for the python scripts you used for MTG-1
Graham,

Here is a quick and dirty script that makes an MTG-1 image. IM cannot -equalize it under Windows
and the picture is rather dark. Don't know how to give IM portable convert.exe enough memory ...

Regards
Ernst


Graham Woolf
 

Hi Ernst

Thanks

I will download the data and give it a go

Kind Regards

Graham