SME server resources
Howto monitor your sme7 general performance using with RRDtool and collectd.
By Hans-Cees Speel (hanscees@<-remove this->hanscees.com
24 Oktober 2005
A newer version of this howto, but still untested by me, lives here:
sme7systemmonitoring.html.
Provided by Stig W. Jensen. Please provide feedback in the forums at www.contribs.org.
Another project I learned about recently that has performance statistics lives here. It is more easy to install and better integrated, but based on another manner to collect data.
They are discussed here.
Here is what you get: take a look! It is/was discussed here
What this howto will and will not produce:
- 1. This howto works in that it produces RRD-tool graphics of performance relevant data like cpu (load), memory usage, IO performance and interface statistics (traffic flow).
- 2. This howto is not yet finished/polished toward sme. Anyone is called upon to improve it. I will be willing to help if needed.
- 3. This howto should also generally work for centhos4 and redhat 4 enterprise, since sme7 is based upon that.
- In addition be advised that tools already on board of sme7 are "top" and "iptraf".
Special thanks to Florian Forster (of collectd ) for helping me out with some initial troubleshooting. And of course to the volunteers of sme7 (www.contribs.org) for making a great platform.
The buildingblocks for this howto are
collectd
RRdtool.
. I also used this guide to make some rpms
Collectd is a small daemon which collects system information every 10 seconds and writes the results in an RRD-file. RRD-tool is a framework which uses round-robin datbases and which has tools included to make graphical overviews. It is well supported in perl.
For your information, there are and will be fast developments on the basis of this first howto. There are newer rpm's for instance. You can find this discussed at contribs here.
However I will not update this howto with the latest and greatest everyday. Because the howto will be unstable now and then if I do that.
I will post, at the bottom, some newer rpm's you can try. But please don't complain if they do not work as advertised. But do help in making the stats better. If you are great in perl we can use your help!
The howto
###############################################
#Step a: get the rpm's and install them
###############################################
##rrdtool and perl-rrd and perl-libxml-dumper for rhel4
mkdir /root/tmp
cd /root/tmp
wget ftp://rpmfind.net/linux/dag/redhat/el4/en/i386/dag/RPMS/rrdtool-1.0.50-1.2.el4.rf.i386.rpm
wget ftp://fr.rpmfind.net/linux/dag/redhat/el4/en/i386/dag/RPMS/perl-rrdtool-1.0.50-1.2.el4.rf.i386.rpm
wget http://ftp.debian.org/debian/pool/main/libx/libxml-dumper-perl/libxml-dumper-perl_0.79.orig.tar.gz
#install RRD rpm's
rpm -ihv rrdtool-1.0.50-1.2.el4.rf.i386.rpm
rpm -ihv perl-rrdtool-1.0.50-1.2.el4.rf.i386.rpm
#install perl-xml-dumper
tar -zxvf libxml-dumper-perl_0.79.orig.tar.gz
cd XML-Dumper-0.79
perl Makefile.PL
make
make test
make install
##install collectd and sensors:
##this is not the latest version. Improvent is possible.
##I am not sure they work for al cpu's
cd /root/tmp
wget http://www.hanscees.com/sme7/collectd-3.0.0-2.i386.rpm
wget http://www.hanscees.com/sme7/collectd-sensors-3.0.0-2.i386.rpm
rpm -ihv --nodeps collectd-3.0.0-2.i386.rpm
rpm -ihv collectd-sensors-3.0.0-2.i386.rpm
##startup collectd
collectd
##you can also do: collectd -p www.nu.nl www.somehost.nl and collectd
will start pinging and
making graphs of ping performance. See man collectd.
##make sure collectd is started up after a reboot:
echo "collectd" >> /etc/rc.local
#see if rrd files are there:
ls /var/lib/collectd/
##You should see a bunch of RRD files there.
###############FYI####################
#collectd files:
#RRD files in : /var/lib/collectd/
#cgi scripts and readme for them are in /usr/share/doc/collectd-3.0.0/contrib/museum
# rrd png making example shellscripts are in /usr/share/doc/collectd-3.0.0/contrib/
# see man collectd
#############################
##################################################
##Step B: install an ibay and monitor statistics from there on demand
##################################################
## The purpose of this step is to have a website you can locally visit to see your statistics.
## You can see statistics like these by doing http://yourdomain/cgi-bin/collection.cgi form a local network
## Please DO NOT make your statistics site publicly available as cgi-bin scripts are famous to get your site hacked.
## I am not saying the script used is unsafe, but it just might turn out to be. I did not research its safety
# If you already have an internal website with cgi-bin support you can use that one of course.
##make a apache dir to call the stats:
This step is for SME server gateway 7 as described here. It will work for any cgi-bin apache site however.
-
#1. create a new ibay for the server manager
https://yourserver/server-manager -> ibay -> create ibay with values:
name stats; description several stats; group admin; write=group; read=everyone; local network (no password required). cgi enabled. -
#make a "fake" domain to which you can bind this ibay in server- manager:
#my choice of naming ("stats") is of course not wise. Choose your own.
https://yourserver/server-manager -> domains -> Add domain with values:
cpustats.com; ibay: several stats; content several stats; dns
internet -
##copy the cgi-script from collectd to your ibay:
##change "stats" to the domain you bound the ibay to..
cp /usr/share/doc/collectd-3.0.0/contrib/museum/* /home/e-smith/files/ibays/stats/cgi-bin/# make sure it is there?
ls -all /home/e-smith/files/ibays/stats/cgi-bin/collection.cgi -
If you want graphs of an hour/day/week/month you have to adjust collection.cgi, or you can copy it
and adjust the copy:
cp collection.cgi collection.all.cgi
Find these lines:
######################################
if (length ($inst)) {
print qq(<img src="$MySelf$RelDir/$type/$inst/day"/><br />\n);
}
else
{ print qq(<img src="$MySelf$RelDir/$type/day" /><br/>\n);
}
#########################################
and replace:
print qq(<img src="$MySelf$RelDir/$type/$inst/day"/><br />\n);
with:
print qq(<img src="$MySelf$RelDir/$type/$inst/3600"/><br />\n);
print qq(<img src="$MySelf$RelDir/$type/$inst/day"/><br />\n);
print qq(<img src="$MySelf$RelDir/$type/$inst/week"/><br />\n);
print qq(<img src="$MySelf$RelDir/$type/$inst/month"/><br />\n);
and that's it.
-
Optional: Make an html file called index.html that redirects to the collection.cgi:
copy and paste this to your command prompt:
echo -E "<\!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0 Transitional//EN\">" > /home/e-smith/files/ibays/stats/html/index.html
echo "<html>" >> /home/e-smith/files/ibays/stats/html/index.html
echo "<head>" >> /home/e-smith/files/ibays/stats/html/index.html
echo "<title>SME statistics</title>" >> /home/e-smith/files/ibays/stats/html/index.html
echo -E "<meta http-equiv=\"REFRESH\" content=\"0;url=\"http://cpustats.com/cgi-bin/collection.cgi\"></HEAD>" >> /home/e-smith/files/ibays/stats/html/index.html
echo "<BODY>" >> /home/e-smith/files/ibays/stats/html/index.html
echo "</BODY>" >> /home/e-smith/files/ibays/stats/html/index.html
echo "</HTML>" >> /home/e-smith/files/ibays/stats/html/index.html
Open the file index.html and het rid of the "\" before !Doc on line 1. -
##put the domain you used in your local dns, or in your host file on your workstation:
-
##linux
##change the ip-address to your sme7 internal ip address
echo "ipaddress cpustats.com" >> /etc/hosts
-
On windows you hosts file is
$windows\winnt\system32\drivers\etc\hosts
put in it on a new line:
ipaddress cpustats.com
#(it may not be there, then you must make it: make sure it is not called hosts.txt!)
-
-
#########################################################
##call the stats:
#########################################################http://cpustats.com/cgi-bin/collection.cgi
Or if you took the optional step to make a index.html:
http://cpustats.com/
##that's it!
############
##Improvement
############
# 1. make a dns structure without host files
# 2. make rpm's for i586 and i686 and so on.
# 3. make a script that uses RRD shellscripts for emailing stats
# 4 place collectd in /service scripts on sme7
###################
## Where are the files
###################
If the packages are not on the internet anymore here are some
backups:
wget http://www.hanscees.com/sme7/collectd-3.0.0-2.i386.rpm
wget http://www.hanscees.com/sme7/collectd-sensors-3.0.0-2.i386.rpm
wget http://www.hanscees.com/sme7/rrdtool-1.0.50-1.2.el4.rf.i386.rpm
wget http://www.hanscees.com/sme7/perl-rrdtool-1.0.50-1.2.el4.rf.i386.rpm wget
http://www.hanscees.com/sme7/libxml-dumper-perl_0.79.orig.tar.gz wget
http://www.hanscees.com/sme7/collectd-3.0.0-2.src.rpm
###################latest rpm's (not stable):
http://www.hanscees.com/sme7/collectd-3.2.0-1.i386.rpm
http://www.hanscees.com/sme7/collectd-sensors-3.2.0-1.i386.rpm
http://www.hanscees.com/sme7/collectd-hddtemp-3.2.0-1.i386.rpm
ftp://ftp.nluug.nl/pub/os/Linux/distr/CentOS/4.2/os/alpha/CentOS/RPMS/perl-XML-Dumper-0.71-2.noarch.rpm
rpm -Uvh perl-XML-Dumper-0.71-2.noarch.rpm
http://www.hanscees.com/sme7/hddtemp-0.3-0.beta12.2.2.el4.rf.i386.rpm
#####################not stable
Everything below is not yet stable, you can try it, but only for testing.
There is a part on hddtemp and on lm_sensors
-
##########################hddtemp for collectd 3.2.0.1:
getting hddtmp to work: get hddtemp
cd /root/tmp
wget http://mirrors.ircam.fr/pub/dag/dries/packages/hddtemp/el4-i386/hddtemp-0.3-0.beta12.2.2.el4.rf.i386.rpm
rpm -ihv hddtemp-0.3-0.beta12.2.2.el4.rf.i386.rpm
hddtemp -d /dev/hda
telnet localhost 7634
Should give something back like:
|/dev/hda|WDC WD400LB-00DNA0|35|C|Connection closed by foreign host.
If it does add "hddtemp -d /dev/hda" to rc.local BEFORE!! collectd is started!:
vi /etc/rc.local
#################################
#########if it does not work:
#mine sais:
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
|/dev/hda|WDC WD400LB-00DNA0|UNK|*|Connection closed by foreign host.
So my drive is unknown. man hddtemp gives some options. Do this:
hddtemp -D /dev/hda
gives on line 194 the temperature
Add the drive to the database:
vi /usr/share/hddtemp/hddtemp.db
added line
"WDC WD400LB-00DNA0" 194 C "Western Digital 400LB-00DNA)"
and now it does work:
# hddtemp -d /dev/hda
# telnet localhost 7634
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
|/dev/hda|WDC WD400LB-00DNA0|35|C|Connection closed by foreign host.
###################end of hddtemp config. -
##############################################
getting lm_sensors to work
collectd 3.2.0.1 can use lm_sensors
using lm_sensors:
- Ian Wells, very active contribs member (Thanks Ian!), has done a lot on lm_sensors see here.
This will tell you how to find and activate the sensors in your PC as used by collected.
Note: SME7 does include a working version of lm_sensors - a new rpm install should not be needed ...
If you have trouble try this one: wget ftp://ftp.nluug.nl/pub/os/Linux/distr/CentOS/4.2/os/i386/CentOS/RPMS/lm_sensors-devel-2.8.7-2.i386.rpm rpm -Uvh lm_sensors-devel-2.8*
Since lm_sensors is installed we can start to detect:
/usr/sbin/sensors-detect
follow the things said...
Mine gives something like this back:
-
Driver `eeprom' (should be inserted):
Detects correctly:
* Bus `SMBus Via Pro adapter at 0500' (Algorithm unavailable)
Busdriver `i2c-viapro', I2C address 0x50
Chip `SPD EEPROM' (confidence: 8)
Driver `vt1211' (should be inserted):
Detects correctly:
* ISA bus address 0x6000 (Busdriver `i2c-isa')
Chip `VT1211 Super IO Sensors' (confidence: 9)
-
To make the sensors modules behave correctly, add these lines to
/etc/modules.conf: (sme7: /etc/modprobe.conf )
#----cut here----
# I2C module options
alias char-major-89 i2c-dev
#----cut here----
To load everything that is needed, add this to some /etc/rc* file:
#----cut here----
# I2C adapter drivers
modprobe i2c-viapro
#modprobe i2c-isa ##if vt12 does not work, this is not neccesary either
# I2C chip drivers
modprobe eeprom
#modprobe vt1211 #not found: fatal
# sleep 2 # optional
/usr/bin/sensors -s # recommended
#----cut here----
-
Do you want to generate /etc/sysconfig/lm_sensors? (YES/no): y
Copy prog/init/lm_sensors.init to /etc/rc.d/init.d/lm_sensors
for initialization at boot time.
/etc/sysconfig/lm_sensors
which is a config file for /etc/rc.d/init.d/lm_sensors
so nothing is to be done.
check to see if anything works:
sensors
Mine did not work.
See later for if I do get it to work.
-
Driver `eeprom' (should be inserted):
#####################
# Further reading
#####################
Rrd-tool:
A round robin database structure, where the database does not grow
indefinitly:
http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/doc/index.en.html
On sme see also man rrdtool
Collectd:
http://verplant.org/collectd/#description
mailing list: http://mailman.verplant.org/listinfo/collectd
On sme see man collectd
Collectd was chosen from a great many RRD-filling schemes because:
- It is small and c-based
- Therefore it can collect every 10 seconds,which is much better than cron-based things
- It does not install the bloated and relatively insecure snmp suites.
- It does what I wanted: performance and not much more.
- It is easy to install
does not work