Prometheus 使用 blackbox 监控 http tcp
2022-02-14 10:38:37

Prometheus http tcp 黑盒监控 blackbox_exporter


Blackbox_exporter

blackbox_exporter 是 Prometheus 拿来对 http/https、tcp、icmp、dns、进行的黑盒监控工具

什么是黑盒监控?以下介绍是抄 zhangguanzhang 的 ​​prometheus的黑盒监控​


常规的各种exporter都是和需要监控的机器一起安装的,如果需要监控一些tcp端口和七层应用层的状态呢,这个时候就需要黑盒监控了,不需要安装在目标机器上即可从外部去监控。



安装

二进制安装

wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.16.0/blackbox_exporter-0.16.0.linux-amd64.tar.gz
tar zxvf blackbox_exporter-0.16.0.linux-amd64.tar.gz
cd blackbox_exporter-0.16.0.linux-amd64
./blackbox_exporter <flags>

blackbox_exporter 有用的参数大概是如下几个

# ./blackbox_exporter --help
usage: blackbox_exporter [<flags>]

Flags:
-h, --help Show context-sensitive help (also try --help-long and --help-man).
--config.file="blackbox.yml"
Blackbox exporter configuration file.
--web.listen-address=":9115"
The address to listen on for HTTP requests.

--log.level=info Only log messages with the given severity or above. One of: [debug, info, warn, error]

启动

# 默认端口为9115
nohup ./blackbox_exporter --config.file="blackbox.yml" &

docker安装

由于 Prometheus 所有组件默认时区都使用的 UTC ,所以推荐用 Docker 去运行它

# 如果你不需要开 debug,请去掉最后的 --log.level=debug
docker run --rm -d -p 9115:9115 --name blackbox_exporter -v /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime:ro -v /data/prometheus/blackbox_exporter/blackbox.yml:/config/blackbox.yml prom/blackbox-exporter:master --config.file=/config/blackbox.yml --log.level=debug

blackbox.yml

# 官方默认的配置文件
modules:
http_2xx:
prober: http
http_post_2xx:
prober: http
http:
method: POST
tcp_connect:
prober: tcp
pop3s_banner:
prober: tcp
tcp:
query_response:
- expect: "^+OK"
tls: true
tls_config:
insecure_skip_verify: false
ssh_banner:
prober: tcp
tcp:
query_response:
- expect: "^SSH-2.0-"
irc_banner:
prober: tcp
tcp:
query_response:
- send: "NICK prober"
- send: "USER prober prober prober :prober"
- expect: "PING :([^ ]+)"
send: "PONG ${1}"
- expect: "^:[^ ]+ 001"
icmp:
prober: icmp

配置

prometheus.yml

HTTP 配置

scrape_configs:
- job_name: 'blackbox'
metrics_path: /probe
params:
module: [http_2xx] # 模块对应 blackbox.yml
static_configs:
- targets:
- http://baidu.com # http
- https://baidu.com # https
- http://xx.com:8080 # 8080端口的域名
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: 127.0.0.1:9115 # blackbox安装在哪台机器

TCP 配置

- job_name: blackbox_tcp
metrics_path: /probe
params:
module: [tcp_connect]
static_configs:
- targets:
- 192.168.1.2:280
- 192.168.1.2:7013

relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: 192.168.1.99:9115 # Blackbox exporter.

重启 Prometheus ,打开 targets ,即可看到。

Prometheus 使用 blackbox 监控 http tcp_域名解析

想查看有哪些指标,则点击 Endpoint 下面的 URL 即可:

Prometheus 使用 blackbox 监控 http tcp_黑盒_02

HTTP 比较值得关注的有如下几项:

# 返回的 http 状态码
probe_http_status_code 200
# https 证书过期时间,默认使用 unixtime
probe_ssl_earliest_cert_expiry 1.637745419e+09
# 如果探测成功则为 1,反之为 0
probe_success 1

告警规则

# 以下两条二选一
groups:
- name: http
rules:
- alert: xxx域名解析失败
expr: probe_success{instance="https://xx.com"} == 0
for: 1m
labels:
severity: "error"
annotations:
summary: "xxx域名解析失败"
- alert: xxx域名解析失败
expr: probe_http_status_code{instance="https://xx.com"} != 200
for: 5m
labels:
severity: "error"
annotations:
summary: "xxx域名解析失败"

自定义模块

有时可能对于某些 URL 需要带参数,如 header、body 之类的,就需要自定义一个模块,​​官方例子​​。

编辑 blackbox.yml

http_2xx_wxjj:
prober: http
timeout: 5s
http:
method: GET
headers:
Cookie: JSESSIONID=C123455dfdf
sid: 41c912344555-24rkjkffd
appid: 1221kj2h1k3hjk13hk
body: '{}'

编辑 Prometheus.yml

- job_name: 'blackbox_wxjl'
metrics_path: /probe
params:
module: [http_2xx_wxjj] # Look for a HTTP 200 response.
static_configs:
- targets:
- http://192.168.201.173:808/byxxxxx/41234456661f-4357c9?head=APP_GeList&user=%E9%BB%84%E5%AE%15
# Target to probe with http.

relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: 172.18.11.154:9115 # The blackbox exporter's real hostname:port.

开启 debug

当你觉得自己设置没错,http 状态码却返回不正确,想要调试一下,就需要打开debug。

targets 开启 debug 会比正常链接输出更多信息

Module configuration:
prober: http
timeout: 5s
http:
ip_protocol_fallback: true
method: GET
headers:
Cookie: JSESSIONID=C123455dfdf
appid: 41c912344555-24rkjkffd
sid: 1221kj2h1k3hjk13hk
body: '{}'
tcp:
ip_protocol_fallback: true
icmp:
ip_protocol_fallback: true
dns:
ip_protocol_fallback: true

FAQ

需要安装多少个 blackbox_exporter ?

理论上只安装一个即可,在特别的网络环境,比如政务云,和第三方对接的时候,只有几台机器开通了网络,那就需要在那几台机器中的其中一台部署 blackbox_exporter,同时 Prometheus.yml 里的 replacement 填上相应的 ip



本文摘自 :https://blog.51cto.com/u_10938529/5005479


更多科技新闻 ......