Pytroll/Satpy and Environment Variable


rnosborne@...
 

Hello to the Group.
 
I recently installed Pytroll/Satpy on Windows 10 following the instructions given in SatPy_HOWTO_V30.txt kindly provided in this
forum. The Satpy version is 0.29. Everything appears to be working with one exception - the Python scripts do not appear to be 
responding the environment variable PPP_CONFIG_DIR=C:/EMCtools/pppconfig and cannot find the areas.yaml file at this location.
 
To begin, running a script produces the warning 'PPP_CONFIG_DIR' is deprecated. Please use 'SATPY_CONFIG_PATH' instead. I replaced
'PPP_CONFIG_DIR' with 'SATPY_CONFIG_PATH' but it had no effect. Although I am not a programmer by any means, I decided to dig deeper to 
see if I could identify the problem.
 
I consulted the Configuration section of the Satpy documentation but found this extremely difficult to understand. However it did suggest
the use of two runtime methods called satpy.config.get and satpy.config.set which I decided to try. First, I used
satpy.config.get('config_path') on its own and it returned a result of ['C', '/EMCtools/pppconfig'] which looked odd. I then used 
satpy.config.set(config_path='C:/EMCtools/pppconfig') followed by satpy.config.get('config_path') which then returned the result
'C:/EMCtools/pppconfig'. The latter 'set' sequence also allowed the script to correctly locate the areas.yaml file in the pppconfig directory.
 
Have I missed something obvious in setting the environment variable?
 
Richard
 


Ernst Lobsiger
 

Richard,

Some versions back the *name* of the relevant variable was changed from PPP_CONFIG_DIR to SATPY_CONFIG_PATH.
That's all. Before starting a script in the black window this variable must be known. Under GNU/Linux I issue the command
echo $SATPY_CONFIG_PATH
and it says
/home/eumetcast/EMCtools/pppconfig

Under Windows the command (with default install as in the HOWTO 3.0)
echo %SATPY_CONFIG_PATH%
shoud return
C:/EMCtools/pppconfig

I do not see why my instructions should not work for the new name of this variable under Windows.
Just configure an environment variable with this name (and delete the old environment variable).

With satpy.config.set() and satpy.config.get() you leave the comfort zone of most Windows users.

Ernst


rnosborne@...
 

Ernst,

Thanks for you reply. I confirmed that SATPY_CONFIG_PATH was set up correctly but the environment variable was still not being read by Satpy.
I have since made a very small change to an underlying Satpy module and this appears to have cured the problem for me.

Regards,
Richard


Ernst Lobsiger
 

Richard,

my version of Satpy is 0.27 with a Miniconda install under GNU/Linux. It works as expected with SATPY_CONFIG_PATH.
If this is not the case for Satpy 0.29 under Windows you should report that to the maintainers (e.g. on the google list).

Best regards,
Ernst


Ernst Lobsiger
 

Richard,

The last few days I worked with PyTROLL/SatPy v0.30 under Windows 10 PRO. It seems I can reproduce your problem. With default install C:\EMCtools and D:\EMCdata and the environment variable SATPY_CONFIG_PATH properly set to "C:/EMCtools/pppconfig" it seems that SatPy is looking under D:\EMCtools\pppconfig (wrong drive!) for areas.yaml. So private area definitions are not found. As a crude workaround a D:\EMCtools\pppconfig\areas.yaml can be generated.

Can someone (maybe Ferdinand or Graham) confirm that they have the same problem with SatPy v0.30 or maybe v0.29 or even an earlier version? This could be a fallout from the change PPP_CONFIG_DIR to SATPY_CONFIG_PATH?

Thanks and best regards,
Ernst


rnosborne@...
 

Ernst,

I believe that the change from PPP_CONFIG_DIR to SATPY_CONFIG_PATH has caused the problem. I will describe my understanding but do not interpret this as correct as I have almost no Python experience.

The current SatPy documentation for environmental variables states:

If setting an environment variable then it must be a colon-separated string and must be set before calling/importing Satpy. If the environment variable is a single path,
it will be converted to a list when Satpy is imported.

An example is given as:
export SATPY_CONFIG_PATH="/path/custom1:/path/custom2"

A full Windows pathname will contain a colon as this is part of the drive identifier e.g. C:  When Satpy processes the environmental variable pathname under Windows, it interprets the colon as a separator and splits the pathname, rendering it useless as a pointer to the wanted folder.

My own fix was to change the underlying Satpy code to look for a different separator that was not present in the pathname. I used a semicolon and this fixed the problem for me.

SatPy code prior to the introduction of 
SATPY_CONFIG_PATH uses a different method an so the problem does not appear.

Best regards,

Richard




Ernst Lobsiger
 

On Mon, Sep 27, 2021 at 04:37 AM, <rnosborne@...> wrote:
A full Windows pathname will contain a colon as this is part of the drive identifier e.g. C:  When Satpy processes the environmental variable pathname under Windows, it interprets the colon as a separator and splits the pathname, rendering it useless as a pointer to the wanted folder.
Richard,

that seems to be the point. Most PyTROLL/Satpy developers work under GNU/Linux that has no drive letters and most of the few PyTROLL/Satpy Windows users will not change from drive C: so everything seems to work.
I tried to escape the drive colon in the environment variable SATPY_CONFIG_PATH to no avail. I will send a note to the PyTROLL Google list. This brakes my EUMETCast PyTROLL/Satpy Starter Kit under Windows 10 :-(.

Best regards,
Ernst


Ernst Lobsiger
 

Richard and All,

Users of my EUMETCast PyTROLL/SatPy Starter Kit 3.0 (or earlier) that run into the problem described in this thread can simply change SATPY_CONFIG_PATH in the Windows env app like:

If your    SATPY_CONFIG_PATH    has been set before as    C:/EMCtools/pppconfig    then set it to    ['C:/EMCtools/pppconfig']     which skipps the problematic code of recent SatPy versions.

SatPy is still in heavy development, you have been warned in my HOWTO.Thanks go to Richard for the heads up and to developer Dave. See  https://groups.google.com/g/pytroll/c/3fXHZxEJiDo

Happy hacking,
Ernst


Ernst Lobsiger
 

Richard and All,

Dave has updated Satpy to version 0.30.1. This has solved all problems. Everything works as before. If you want to use the new feature of more than one config directory you can set SATPY_CONFIG_PATH to a *semicolon* separated string like C:/here;D:/there  . It's now a colon under *nix OSs and a semicolon under Windows. Even the deprecated PPP_CONFIG_DIR still works but reminds you, that you should slowly change to the new name SATPY_CONFIG_PATH. Thanks @Dave.

Regards,
Ernst