Re: Linux Disk Performance

Thomas Scheelen
 


Olivier, 

i use Windows, so i can't really say to much about Linux, but it could help anyway. 

I have also a small ssd as boot device and a much larger normal HDD as storage. 

The way it realized that is a tool named file juggler. 
That tool has many automation capabilities. 
Basically it looks into a given folder and looks for new files. If it has been found those files, it moves them to the new location on the other drive. 

You can configure many criteria as filter like date, age, size, fragments of filename. 

Since the files are already moved by telecast, it makes no problems to move them. 

I also sort out files which i don't want to move and delete them automatically with file juggler. 

It works perfectly fast and smooth. 

I am quite sure that you will find a similar tool for linux. 

Regards 
Thomas 

Am 17. November 2019 21:03:54 schrieb hb9erg.kd0hop@...:

Good evening everyone

I hope I'm posting this question on the correct location... :-)

** My Setup **
I'm running Linux Mint 19.2, using an internal SSD (500 GB, ext4).
Tellicast client, EUMETCastView run perfectly fine using default values.

** What I'm intending **
I'd like to use my external 2TB HDD (ext4) as target for e.g. EUMETCast BAS files:
/eumetcast/bas/default

My idea is to have only the OS and EUMETCastView running on the SSD, while the data is stored on my "big" external HDD.

I named that external 2TB HDD "EUMETCast", so it appears as:
/media/ops/EUMETCast --> It's mounted as sdb1.

As described in EUMETSAT TD15, the tmp-files database AND the target MUST be on the *same* filesystem:
"During reception, the file fragments are immediately written to the temporary location on the diskusing a temporary file name and control information is held in a database, which is resident inmemory – the file database. Once a file is completely received, it is moved from the temporaryto the target location on the disk and renamed to the original filename, and the timestamp ofthe file is set to the original time in the EUMETCast platform. It is important that the temporarylocation and target location are on the same file system, otherwise the move operation will notbe possible (in Linux) or result in a copy operation (Windows) which takes more time andresources. The move is an “atomic” and fast operation because it consists of just a change inthe file allocation table - the file itself is not touched."

If it's not, I get, as expected, the following log-file "ERROR":
Invalid cross-device link
Cannot rename file "/media/ops/EUMETCast/ramdisk/data/tmp/bas/5dd156ce00e26b3d.tmp" to "/media/ops/EUMETCast/data/eumetcast/bas/default/H-000-MSG3__-MSG3_RSS____-IR_097___-000007___-201911171415-C_" (Invalid cross-device link)
** What I tried **
So I was setting up a RAM DISK, whose destination target is on my external 2TB HDD:
sudo mount -t tmpfs -o size=2048M tmpfs /media/ops/EUMETCast/ramdisk

At file cast-client-channels_bas.ini, I typed:
target_directory=/media/ops/EUMETCast/data/eumetcast/bas/default
tmp_directory=/media/ops/EUMETCast/ramdisk/data/tmp/bas

** What I hoped for **
Now that RAMDISK (used for tmp-files) AND target directory are *presumably* on the SAME filesystem, it should work.

But: it doesn't.

** My questions **
I'm new to Linux but really would like to learn more about it and get it to work.

1) What's the correct path to my external HDD that tellicast client "understoods"? Is it
a) /media/ops/EUMETCast/
b) /dev/sdb1
c) or via /dev/disk/, e.g. by-partuuid: /dev/disk/by-partuuid/4d878bfa-5680-4e32-bef4-a9d3e003286e

2) Is it SOMEHOW possible to
- have the tmp-files saved on the RAM DISK (-or- even to the internal SSD)
AND
- the target files saved on the external HDD?

Or doesn't it make sense what I'm aiming for...? :-P


Thanks very much in advance for your help!

Vy 73 de Olivier, HB9ERG, Basel


Join MSG-1@groups.io to automatically receive all group messages.