smokeping 默认用sendmail 发邮件告警,也可以直接调用外部程序进行报警。smokeping的alert设置有点复杂,但是却很好用,设置很灵活,考虑得很周全。

一、配置邮件支持

默认情况下sendmail 是不通过163、qq之类的邮件服务器发送的,不过我们略微修改下配置即可以支持。首先需要安装 Authen::SASL 模块(auth 需要用的),然后修改 smokeping/lib/Smokeping.pm 。

如下:

 1#头上加
 2use Authen::SASL;
 3#定位到sendmail函数,默认新版本sendmail函数的定义的已经是下面这样了,如果不是改成下面这样
 4sub sendmail ($$$){
 5    my $from = shift;
 6    my $to = shift;
 7    $to = $1 if $to =~ /<(.*?)>/;
 8    my $body = shift;
 9    if ($cfg->{General}{mailhost} and
10        my $smtp = Net::SMTP->new([split /\s*,\s*/, $cfg->{General}{mailhost}],Timeout=>5) ){
11        $smtp->auth(split(/\s*,\s*/, $cfg->{General}{mailusr}),split(/\s*,\s*/, $cfg->{General}{mailpwd}));
12        $smtp->mail($from);
13        $smtp->to(split(/\s*,\s*/, $to));
14        $smtp->data();
15        $smtp->datasend($body);
16        $smtp->dataend();
17        $smtp->quit;
18    } elsif ($cfg->{General}{sendmail} or -x "/usr/lib/sendmail"){
19        open (M, "|-") || exec (($cfg->{General}{sendmail} || "/usr/lib/sendmail"),"-f",$from,$to);
20        print M $body;
21        close M;
22    } else {
23        warn "ERROR: not sending mail to $to, as all methodes failed\n";
24    }
25}
26#找到  '_vars =>' ,把 mailusr mailpwd  加进去。不然不能启动哦!General configuration values valid for the whole SmokePing setup.
27DOC
28_vars =>
29[ qw(owner imgcache imgurl datadir dyndir pagedir piddir sendmail offset
30smokemail cgiurl mailhost mailusr mailpwd snpphost contact display_name
31syslogfacility syslogpriority concurrentprobes changeprocessnames tmail
32changecgiprogramname linkstyle precreateperms ) ],

配置完成后修改配置文件/opt/smokeping/etc/config ,增加邮件服务器的配置:

1mailhost = smtp.361way.com
2mailusr = monitor@361way.com
3mailpwd = xxxxxxxxx

二、Alerts部分配置

1、自定义alerts告警策略

如下定义了几种告警策略,bigloss 、someloss等

 1*** Alerts ***
 2to = admin@361way.com
 3from = monitor@361way.com
 4+bigloss
 5type = loss
 6# in percent
 7pattern = ==0%,==0%,==0%,==0%,>0%,>0%,>0%
 8comment = suddenly there is packet loss
 9+someloss
10type = loss
11# in percent
12pattern = >0%,*12*,>0%,*12*,>0%
13comment = loss 3 times  in a row
14+startloss
15type = loss
16# in percent
17pattern = ==S,>0%,>0%,>0%
18comment = loss at startup
19+rttdetect
20type = rtt
21# in milli seconds
22pattern = <10,<10,<10,<10,<10,<100,>100,>100,>100
23comment = routing messed up again ?
24+hostdown
25type = loss
26# in percent
27pattern = ==0%,==0%,==0%, ==U
28comment = no reply
29+lossdetect
30type = loss
31# in percent
32pattern = ==0%,==0%,==0%,==0%,>20%,>20%,>20%
33comment = suddenly there is packet loss

以上几种告警,这里选取三种说明如下:

  • someloss: 如果在12次检查中出现了3次丢包的情况(不论丢多少个包),就进行alert;
  • rttbad: 如果连续出现两次50毫秒以上的延时,就进行alert;
  • rrtdetect: 之前5次检查延时都少于10毫秒,前6次检查延时都少于100毫秒,第7次开始连续3次检查延时都大于100毫秒的话,就进行alert。

2、策略应用

在target里面加上相应的策略配置即可,如下:

1++ 361way
2menu = 361way_host
3title =361way.com
4host = www.361way.com
5alerts = someloss,hostdown

主机一旦出现告警,就会通过邮件发送到我们事先配置的邮箱中,类似下图:

smokeping-alert
smokeping-alert

上图中的邮件告警内容也可以进行自定义格式输出和修改,这个还是修改Smokeping.pm文件,修改其中如下部分即可:

 1my $default_mail = <<DOC;
 2Subject: [SmokeAlert] <##ALERT##> <##WHAT##> on <##LINE##>
 3<##STAMP##>
 4Alert "<##ALERT##>" <##WHAT##> for <##URL##>
 5Pattern
 6-------
 7<##PAT##>
 8Data (old --> now)
 9------------------
10<##LOSS##>
11<##RTT##>
12Comment
13-------
14<##COMMENT##>
15DOC

3、pattern匹配

pattern匹配是编写alert规则中很最要的一部分,官方对这部分有说明,不过我查到了一个香港同胞总结的要好一些,这里摘录下,不做翻译了。

smokeping-pattern
smokeping-pattern

三、特殊调用

1、外部程序调用

调用外部告警程序,如IM、短信等的示例如下:

1*** Alerts ***
2to = |/usr/local/smokeping/bin/alert.sh
3from = joe@somehost

“to” 选项,默认是要填入一个email地址的,但是只要在”=”后面加上”|”,后面再跟你自定义的脚本的路径,就可以调用自己的脚本进行alert了。脚本会读入5或者6个参数:name-of-alert, target, loss-pattern, rtt-pattern, hostname,[raise]。自己选择使用哪些参数alert即可。

这里从老外站点上找到有一个示例如下:

config 配置

1to = |/etc/smokeping/config.d/trace_alert.sh 2> /tmp/trace.log

脚本内容:

 1########################################################
 2# Script to email a mtr report on alert from Smokeping #
 3########################################################
 4alertname=$1
 5target=$2
 6losspattern=$3
 7rtt=$4
 8hostname=$5
 9email="monitoring@email.com"
10smokename="BR-NYC-"
11if [$losspattern" = “loss: 0%" ];
12then
13  subject="Clear-${smokename}-Alert: $target host: ${hostname}"
14else
15  subject="${smokename}Alert: ${target}${hostname}"
16fi
17echo “MTR Report for hostname: ${hostname}" > /tmp/mtr.txt
18echo “" >> /tmp/mtr.txt
19echo “sudo mtr -n –report ${hostname} "
20sudo /usr/sbin/mtr -n –report ${hostname} >> /tmp/mtr.txt
21echo “" >> /tmp/mtr.txt
22echo “Name of Alert: " $alertname >> /tmp/mtr.txt
23echo “Target: " $target >> /tmp/mtr.txt
24echo “Loss Pattern: " $losspattern >> /tmp/mtr.txt
25echo “RTT Pattern: " $rtt >> /tmp/mtr.txt
26echo “Hostname: " $hostname >> /tmp/mtr.txt
27echo “" >> /tmp/mtr.txt
28echo “Full mtr command is: sudo /usr/sbin/mtr -n –report ${hostname}" >> /tmp/mtr.txt
29echo “subject: " $subject
30if [ -s /tmp/mtr.txt ] then
31 mailx -s “${subject}" $email
32fi

2、特定主机发送到特定用户

to 表示接受所有报警的邮箱,如果需要在特定的节点报警发送到特定的邮箱 则在该节点上增加alertee = testmonitor@139.com即可。具体可以参看/opt/smokeping/lib/Smokeping.pm源码,如下部分:

1foreach my $addr (map {$_ ? (split /\s*,\s*/,$_) : ()} $cfg->{Alerts}{to},$tree->{alertee},$alert->{to}){