Howto monitor/graph Tellicast/receiver device in Linux?


Christian Peters
 

Dear group members,

I set up a receiver PC running Linux Ubuntu 14.04 32bit and Tellicst 2.3.4a with a TBS 5925 USB box.
It was a real challenge! But: it's working fine now and I start to think about monitoring some data.
I set up munin (a monitor solution I know and which was quite easy) to get a website with graphs for CPU, memory, disk, network etc. But I would like to monitor even S/N, lost/missed/recovere packets, link margin etc. There are solutions for mrtg and win on the great website of D.Taylor but I didn't find any information for linux on the web. I found a nice script at Dundee Satellite Receiving Station http://www.sat.dundee.ac.uk named recv.log file analyser http://www.sat.dundee.ac.uk/%7Earb/tellicast/dvbs_filter_recv_log.pl which give me some ideas what could be possible. Even there I found some nice code to extract the missed packets from the tellicast website:

#!/usr/bin/perl
# Use links to get tellicast Missed and recovered packets
open(CMD, "links -dump http://localhost:2517/www/tsl/receiver/tsl_receiver_statistics.html |") || die "cannot run links localhost";
while(<CMD>)
{
chomp;
@f=split(/[ :]+/);
if (/Missed Data Packets before FEC:/)
{
$tc_missed = $f[6];
}
if (/Recovered Data Packets:/)
{
$tc_recovered = $f[4];
}
}
close(CMD);
printf "Tellicast missed %s packets before FEC %s recovered&#92;n", $tc_missed, $tc_recovered;

But I am not really a programming guy so if somone have some tips or scripts, or ideas howto get that data monitored (e.g. from the TBS device or the Tellicast website) it would be very helpfull! Or if this is a real challenge it was even hepful to let me know if you think it's to hard to setup such monitoring solution without high knowledge of linux and scripting.

Regards,

Christian


http://www.sat.dundee.ac.uk

Dundee Satellite Receiving Station http://www.sat.dundee.ac.uk Dundee Satellite Receiving Station, Dundee University, UK, maintains an up-to-date archive of images from NOAA, SeaStar, Terra and Aqua polar orbiting ...



View on www.sat.dundee.ac.uk http://www.sat.dundee.ac.uk
Preview by Yahoo



(


Ernst Lobsiger
 

Christian


Welcome to the lonly rangers that use GNU/Linux for Eumetcast. I use RRDtool and eLuna to monitor my receivers. For log analysis I wrote TClogSummary. There are versions for Windows and Linux. I had a look at munin but it seems to be an overkill. But of course it works with RRDtool whitch is pretty much standard today.


You can see what can be done with RRDtool/eLuna and TClogSummary here



Old TC Client / Receiver Io :
http://5.153.123.95:84 http://5.153.123.95:84/

New TC Client / Receiver Ganymed:
http://5.153.123.95:83 http://5.153.123.95:83/


New TC Client / Receiver Kallisto:
http://5.153.123.95:82 http://5.153.123.95:82


You will have to learn how to write or adapt very small scripts using perl, bash and awk (usually less than 10 lines, I have no WEB-Site anymore but I could make all my update scripts available somehow) to produce the numbers that go to the RRDs. This should be rather easy after you got your GNU/Linux receiver going ...


Cheers,
Ernst


P.S.
My receivers are a work in progress and might be down sometimes. I experiment with the new TC Client.


fox91fox@...
 

Hello Ernst,
I think the thing that is holding back from making use of GNU/Linux for Eumetcast is the fact that there are no (or at least I have not found) comprehensive guides and updated.
The few that are found are of 2000 or even earlier.
Same thing for the software to be used.

I've just installed Ubuntu Eumetcast 14.04x64, but the reception gives me a problem (I have to figure out if the problems are in the antenna or in the PC configuration with TelliCast).

For monitoring the TBS-6983 I use Cacti (relies on RRDtool).

I think it would be very interesting if you could post (for example on Github) a guide to install on Debian.

Thank you,
Andrew


Christian Peters
 

Ernst,

yes....I think there are really only some lonely rangers using Eumetcast
on Linux! :-O
Yes...Munin will be overkill but was easy to setup.
Maybe RRDtoll is the way to go! Your eLuna/RDDtool combination on your
website is really very nice and what I am dreaming of!
i will have a look into RDDTool. How do you get the data/informations? I
discovered femon which gives me this output:

status SCVYL | signal 60% | snr 70% | ber 0 | unc 0 | FE_HAS_LOCK

but signal strength and snr are in percent, not dB? Did you use this to
got data from the TBS device?
How did you got the missed/recovered packtes from Tellicast monitored!?
Where is that information available!? Is there a log file which contains
missed/recovered packet or did you extract this from the tellicast
website!? How did you spilt this data into 5 min sections!? Did you
reset the values on the website after 5 min? I've no idea so far...!? So
if you have some ideas or code snippets as example it would be great!?
Maybe a git login would be great to share some ideas or code if you
would like, would help a lot!

@Andrew: I have choosen 14.04 Ubuntu as I saw there was a install script
for the TBS 5925 and Ubuntu 14.04 32bit on the Eumetcast-DVD Linux
folder. At the end I did exact what was described in that script. I even
ran into the errors they described and and did was they wrote.
Only problem was the ramdkisk which wasn't created in the right way
(only 64mb size). But in the Eumetsat FAQ the situation was described
and now I use /run/shm and all works well. I didn't really understand
all at the time but some things become more clear now.
Yes, an up-to-date install howto for Linux would be really great! I'm
just worrying whether I have to switch to the new tc-client 2.12.
version but I think as long as it's not released and no howtos found on
the Eumetsat website I hope it's the right choice to stay at the 2.4.3
version which I got on the DVD?

Regards,

Christian


Am 19.02.16 um 09:40 schrieb ernst.lobsiger@... [MSG-1]:

Chri


Welcome to the lonly rangers that use GNU/Linux for Eumetcast. I use
RRDtool and eLuna to monitor my receivers. For log analysis I wrote
TClogSummary. There are versions for Windows and Linux. I had a look
at munin but it seems to be an overkill. But of course it works with
RRDtool whitch is pretty much standard today.


You can see what can be done with RRDtool/eLuna and TClogSummary here



Old TC Client / Receiver Io :
http://5.153.123.95:84 http://5.153.123.95:84/

New TC Client / Receiver Ganymed:
http://5.153.123.95:83 http://5.153.123.95:83/


New TC Client / Receiver Kallisto:
http://5.153.123.95:82 http://5.153.123.95:82


You will have to learn how to write or adapt very small scripts using
perl, bash and awk (usually less than 10 lines, I have no WEB-Site
anymore but I could make all my update scripts available somehow) to
produce the numbers that go to the RRDs. This should be rather easy
after you got your GNU/Linux receiver going ...


Cheers,
Ernst


P.S.
My receivers are a work in progress and might be down sometimes. I
experiment with the new TC Client.



[Non-text portions of this message have been removed]



[Non-text portions of this message have been removed]


Francis Greaves
 

Hi Christian
Also think of Cacti which is a good 'front end' for RRD Tool
All the best
Francis


fox91fox@...
 

Hi Christian,
I have the TBS6983 board and I followed this guide http://www.eumetsat.int/website/home/TechnicalBulletins/EUMETCast/DAT_2082113.html#device.
The drivers I downloaded them from here http://www.tbsdtv.com/download/ http://www.tbsdtv.com/download/ (v150429 version for TBS6983).
I have not tried the last v160219 driver.

This is my Cacti:
sat signal http://imagizer.imageshack.us/a/img922/9728/0DTCDo.png sat quality http://imagizer.imageshack.us/a/img922/4153/oNBiBW.png dummy0 http://imagizer.imageshack.us/a/img924/1106/fGiPM0.png
Andrew


Ernst Lobsiger
 

Hi All


Well, you can use munin, Cacti or eLuna. It always boils down to 3 things:


1 What and how (timeline) variable/s should be monitored? Create an RRD accordingly.
2 How should the graph look like? Constuct the RRD command to produce this graph.
3 Write a small update script that fills e.g. every 5 minutes the RRD under cron control.


I came to use eLuna because it is two small perl scripts only and the 3 steps are well documented for beginners. The downside is that there is no single package to install with one mouse click. but there are packages for apache2, RRDtool and the 2 perl modules that must be added for eLuna to work ...


Cheers
Ernst



---In MSG-1@..., <francis@...> wrote :

Hi Christian
Also think of Cacti which is a good 'front end' for RRD Tool
All the best
Francis


Christian Peters
 

Hi Andrew,

I installed v160126 driver.
I used the same .pdf! ;-)

Nice graphs...but I would really now howto get not only '%' but 'dB'
values at signal levels and s/nr and even hotwo comute the link margin!?
It would be nice to have a comparison to other stations!?

Regards,

Christian

Am 19.02.16 um 11:15 schrieb fox91fox@... [MSG-1]:

Hi Christian,
I have the TBS6983 board and I followed this guide
http://www.eumetsat.int/website/home/TechnicalBulletins/EUMETCast/DAT_2082113.html#device.
The drivers I downloaded them from here
http://www.tbsdtv.com/download/ http://www.tbsdtv.com/download/
(v150429 version for TBS6983).
I have not tried the last v160219 driver.

This is my Cacti:
sat signal http://imagizer.imageshack.us/a/img922/9728/0DTCDo.png sat
quality http://imagizer.imageshack.us/a/img922/4153/oNBiBW.png dummy0
http://imagizer.imageshack.us/a/img924/1106/fGiPM0.png
Andrew

[Non-text portions of this message have been removed]



[Non-text portions of this message have been removed]


Ernst Lobsiger
 

Hi Christian and Andrew


To get S/N in dB and Power in dBm from the TBS-5925 you must set module parameters esno and dbm for module stv090x (at the command line type modinfo stv090x). The parameters are set in /etc/moprobe where you set the issy too ... Then you have to use femon without the -H switch and get the values from there.


@Andrew
You see module parameters for TBS-6983 with modinfo tbsfe. Unfortunately the TBS-6983 driver does not give Power values back. The 85% is a fixed fake,value. Thats why your graph has been stable for weeks now.


Cheers
Ernst


fox91fox@...
 

Hi Ernst,
thanks for the explanation.

This is the output of modinfo tbsfe:

filename: /lib/modules/3.19.0-43-generic/kernel/drivers/media/dvb/frontends/tbsfe.ko
license: GPL
author: Konstantin Dimitrov <kosio.dimitrov@...>
description: TurboSight TBS FE module
srcversion: C3C15775628776C31F9EB78
depends:
vermagic: 3.19.0-43-generic SMP mod_unload modversions
parm: pne:Phase noise enhancements 0:Off, 1:On (default:0) (int)
parm: esno:SNR is reported in 0:Percentage, 1:(EsNo dB)*10 (default:0) (int)
parm: dbm:Signal is reported in 0:Percentage, 1:-1*dBm (default:0) (int)
parm: occ:0:outer code correction is disabled, 1: outer code correction is enabled(default:1) (int)

As I wrote in a previous post (https://groups.yahoo.com/neo/groups/MSG-1/conversations/topics/22069), if the quality falls below 65%, TelliCast has 100% of lost.
It's normal?


Ernst Lobsiger
 

Andrew


As I said, your signal power plot is fixed to 85% because the driver gives back a stable fake value.


Your quality plot is real but nowadays you should use EsNo (S/N) in dB (see the plots on Davids page).
To do so you have to edit a simple text file /etc/modprobe.d/tbsfe.conf with on line in it:


options tbsfe esno=1 dbm=1


With the next boot your module tbsfe will be loaded with these parameters and give back (still no Signal Power)
EsNo in dB. Then you have to adapt Cacty to display EsNo in dB. When EsNo goes below 9.5dB your TBS card will run into problems th decode HVS packets. This will harm your receiver even if you only take the Basic Service.That's probably where you have the 65% quality indication now. Unfortunately there is still no general solution for MODCOD setting in GNU/Linux. By taking only 8psk 3/5 you can go down to almost EsNo 6dB.


Regards,
Ernst


Christian Peters
 

Hi Ernst,

I set the parameters and will reboot at Sunday! So I can use femon to
extract these values in dB-great.
I just installed eLuna, very nice. But got no data graphs/lines appear
inside the empty tables.....updating isn't working....no errors at
all....maybe perl is not working.....but still investigating what the
problem is. :-)

Thanks for that information!

Regards,

Christian

Am 19.02.16 um 12:20 schrieb ernst.lobsiger@... [MSG-1]:

Hi Christian and Andrew


To get S/N in dB and Power in dBm from the TBS-5925 you must set
module parameters esno and dbm for module stv090x (at the command line
type modinfo stv090x). The parameters are set in /etc/moprobe where
you set the issy too ... Then you have to use femon without the -H
switch and get the values from there.


@AndrewHi Ernst,

You see module parameters for TBS-6983 with modinfo tbsfe.
Unfortunately the TBS-6983 driver does not give Power values back. The
85% is a fixed fake,value. Thats why your graph has been stable for
weeks now.


Cheers
Ernst


[Non-text portions of this message have been removed]



[Non-text portions of this message have been removed]


Ernst Lobsiger
 

Christian


You are probably not updating the RRDs. You have to add a line in /etc/crontab to update all RRDs every 5 minutes. It's described in the eLuna help files. I use the following line, put it at the bottom of /etc/crontab


*/5 * * * * root /var/www/eluna/update.pl > /dev/null 2>&1


If you already see some graphs you have adapted the Apache conf file O.K. I'm not sure Ubuntu has the Apache web root in /var/www too. If not you have to adapt my cron line to point to update.pl. Then you just restart cron at the command line with "service cron restart". If this doesn't work try "/etc/init.d/cron restart" .


Another point is that all the eluna files have to belong to the user Apache2 runs under. It's www-data in Debian and probably in Ubuntu as well. So I normally change to Apache2s root directory /var/www and issue at the command line the command "chown -R www-data:www-data eluna" Then descending from the web root in directory eluna everything is owned by www-data for sure.


Good luck.


Regards,
Ernst


Christian Peters
 

Ernst,

eLuna is working, I see the page and even the graphs, I set up the
cronjob and the tables even updates every 5 min as I see the most recent
time at the bottom of each graph.
But the data seems not to be inserted in the graphs....? I changed the
'graphs' directory were the eLuna files live to www-data too. Strange...

Have to check further on...!

Thanks for your help.

Christian

Am 19.02.16 um 20:04 schrieb ernst.lobsiger@... [MSG-1]:

Christian


You are probably not updating the RRDs. You have to add a line in
/etc/crontab to update all RRDs every 5 minutes. It's described in the
eLuna help files. I use the following line, put it at the bottom of
/etc/crontab


*/5 * * * * root /var/www/eluna/update.pl > /dev/null 2>&1


If you already see some graphs you have adapted the Apache conf file
O.K. I'm not sure Ubuntu has the Apache web root in /var/www too. If
not you have to adapt my cron line to point to update.pl. Then you
just restart cron at the command line with "service cron restart". If
this doesn't work try "/etc/init.d/cron restart" .


Another point is that all the eluna files have to belong to the user
Apache2 runs under. It's www-data in Debian and probably in Ubuntu as
well. So I normally change to Apache2s root directory /var/www and
issue at the command line the command "chown -R www-data:www-data
eluna" Then descending from the web root in directory eluna everything
is owned by www-data for sure.


Good luck.


Regards,
Ernst


[Non-text portions of this message have been removed]



[Non-text portions of this message have been removed]


Ernst Lobsiger
 

Christian


Two more things:


I faintly remember that the eLuna files had some wired attributes: e.g. the RRDs should not be set executable but must be writable. Maybe this is fixed in the latest eluna version (I use and prefer an older version here) ...






If you are reluctant to reboot your system and if you are BRAVE you can change module parameters on the fly:


First check that your module is loaded and these parameters are actually in place (this is the easy part):


As root at the prompt issue the command:


cat /sys/module/stv090x/parameters/dbm


cat /sys/module/stv090x/parameters/esno


This probably will give you a "0" in both cases. You change on the fly with (the part you have to be brave):


echo 1 > /sys/module/stv090x/parameters/dbm
echo 1 > /sys/module/stv090x/parameters/esno


You have written directly to the kernel and femon should now show dBm and db as described in the module info. Do NOT use femon -H which gives you meaningless % stuff. Use just femon and the numbers are HEX.




Regards,
Ernst




P.S.
To make the changes permanent after the next boot edit the file in /etc/modprobe.d/stv090x.conf (your naming may vary) where you already have the line "options stv090x issy=0". Just add on the same line esno=1 dbm=1.








---In MSG-1@..., <hctpeters@...> wrote :

Hi Ernst,

I set the parameters and will reboot at Sunday! So I can use femon to
extract these values in dB-great.
I just installed eLuna, very nice. But got no data graphs/lines appear
inside the empty tables.....updating isn't working....no errors at
all....maybe perl is not working.....but still investigating what the
problem is. :-)

Thanks for that information!

Regards,

Christian


Ernst Lobsiger
 

Christian


The graph frames and times you see come from the eluna perl file. It does not mean that the RRDs are actually updated. To check the latter go e.g. to directory /var/www/eluna/rrd/01_load. You should see 4 files there and rhe file "load.rrd" should belong to www-data, be writable and be updated (and change it's file time accordingly) every 5 minutes. If this is not the case, check that the file /var/www/eluna/update.pl is executable. You can even execute it from the command line. This should definitely fill first data into all the RRDs.


Finally go to /var/log and check the demon, apache, cron, system and message logs for eLuna errors ...


Cheers,


Ernst



---In MSG-1@..., <hctpeters@...> wrote :


Ernst,

eLuna is working, I see the page and even the graphs, I set up the
cronjob and the tables even updates every 5 min as I see the most recent
time at the bottom of each graph.
But the data seems not to be inserted in the graphs....? I changed the
'graphs' directory were the eLuna files live to www-data too. Strange...

Have to check further on...!

Thanks for your help.

Christian


Ernst Lobsiger
 

Christian


My last hint: I installed eLuna in /var/www/eluna but I just noticed that the README.english of my eLuna version 1.07 proposes to install in /var/www/graphs. I changed the name because there is one more graphs directory below. Of course cron and Apache2 must be tolld the exact name of the directory you choose. And maybe my earlier chown command to give ownership of all eluna files to www-data must be adapted too ...


Good night


Ernst


Christian Peters
 

Ernst,

just inserted the the '1' into the kernel module, it's working! Thank
you! :-) If eLuna is working I will try to convert the two values from
hex to decimal and maybe start to insert a new graph in eLuna...let's
see...

Just solved the eLuna problem: i shouldn't copy/paste...even if it's
right from the README of eLuna! :-O
*/5 * * * * root /var/www/html/eluna/update.pl

didn't work but that:

*/5 * * * * /var/www/html/eluna/update.pl

I checked and could update inside each rrd subdirectory, cron was
running but no error!
But now it updates and it's running-graphs appear...! :-)

Thank you very much for your help!

Christian

Am 19.02.16 um 21:33 schrieb ernst.lobsiger@... [MSG-1]:

Christian


My last hint: I installed eLuna in /var/www/eluna but I just noticed
that the README.english of my eLuna version 1.07 proposes to install
in /var/www/graphs. I changed the name because there is one more
graphs directory below. Of course cron and Apache2 must be tolld the
exact name of the directory you choose. And maybe my earlier chown
command to give ownership of all eluna files to www-data must be
adapted too ...


Good night


Ernst






[Non-text portions of this message have been removed]


Ernst Lobsiger
 

Christian


Glad to hear the RRDtool/eLuna monitoring for your GNU/Linux Eumetcast receiver is up and running.


To add more graphs you can directly copy create.sh, update.sh, and graph.pm files from my receivers.
Just access them via a browser, you should know by now where you can find the rrd subdirectory ;-).


The web interfaces of old and new Tellicast Clients are somewhat different. As you still use 2.4.3a you should copy the missed/recovered files from my receiver io. Of course you will have to adapt some pathes and the like. A word to the TCP Interfaces: If eluna updates the traffic still via the "ifconfig" command you should have a look at my updates that work via /proc/net/dev. The difference is that my solution uses 64 Bit counters while via "ifconfig" you will get 32 Bits only and these will wrap from time to time messing up your traffic graphs :-(.


Regards,
Ernst


P.S.
For debugging update.sh files just type an "echo" command in front of the
"rrdtool update" line and execute them from the command line. Good luck.


Christian Peters
 

Ernst,

thank you very much! :-) I will take the offer and learn something more!
That's a good clue to check manual with an echo command included.
But now I have to look after my dish, receiption stalls at 6:00 in the
morning....I think a gust moved the dish!? :-(

Have a nice weekend!

Regards,

Christian

Am 20.02.16 um 09:12 schrieb ernst.lobsiger@... [MSG-1]:

Christian


Glad to hear the RRDtool/eLuna monitoring for your GNU/Linux Eumetcast
receiver is up and running.


To add more graphs you can directly copy create.sh, update.sh, and
graph.pm files from my receivers.
Just access them via a browser, you should know by now where you can
find the rrd subdirectory ;-).


The web interfaces of old and new Tellicast Clients are somewhat
different. As you still use 2.4.3a you should copy the
missed/recovered files from my receiver io. Of course you will have to
adapt some pathes and the like. A word to the TCP Interfaces: If eluna
updates the traffic still via the "ifconfig" command you should have a
look at my updates that work via /proc/net/dev. The difference is that
my solution uses 64 Bit counters while via "ifconfig" you will get 32
Bits only and these will wrap from time to time messing up your
traffic graphs :-(.


Regards,
Ernst


P.S.
For debugging update.sh files just type an "echo" command in front of the
"rrdtool update" line and execute them from the command line. Good luck.





[Non-text portions of this message have been removed]



[Non-text portions of this message have been removed]