部署 prometheus + Grafana + node_exporter + alertmanager

prometheus 版本 2.25.0，Grafana 版本 7.5.0，node_exporter 1.1.2，alertmanager 0.22.0-rc.1 。在官网下载安装包即可。
https://prometheus.io/download/
https://grafana.com/grafana/download?pg=get&plcmt=selfmanaged-box1-cta1

node_exporter port:9100: 采集节点的 cpu，内存，磁盘等基本信息。
prometheus port:9090: 收集 node_exporter,redis_exporter 等数据。
Grafana port:3000: 从 prometheus,mysql,es 中拿到采集的数据进行图形展示。
alertmanager port:9093: 报警管理器。可以与钉钉结合将报警发给钉钉机器人。
百度网盘链接：
链接：https://pan.baidu.com/s/18ZNOyQlxXE52tlSD6uOLOA
提取码：r0gt

prometheus 部署步骤：

groupadd prometheus
useradd -g prometheus -M -s /sbin/nologin prometheus

tar -zxvf /opt/module/prometheus-2.25.0.linux-amd64.tar.gz -C /usr/local
cd /usr/local
mv /usr/local/prometheus-2.25.0.linux-amd64/ /usr/local/prometheus
mkdir -pv /usr/local/prometheus/data
chown -R prometheus.prometheus /usr/local/prometheus
chown -R prometheus.prometheus /data1/prometheus

vi /usr/lib/systemd/system/prometheus.service

prometheus [Unit]
Description=Prometheus Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network-online.target

[Service]
User=prometheus
Restart=on-failure
ExecStart=/usr/local/prometheus/prometheus \
  --config.file=/usr/local/prometheus/prometheus.yml \
  --storage.tsdb.path=/usr/local/prometheus/data
ExecReload=/bin/kill -HUP $MAINPID
[Install]
WantedBy=multi-user.target

vi /usr/local/prometheus/prometheus.yml(配置要监控的host和报警信息)

global:
	scrape_interval:     15s 
	evaluation_interval: 15s 

alerting:
	alertmanagers:
	- static_configs:
	- targets: ["localhost:9093"]

rule_files:
#- "alert.rules"
			  
scrape_configs:
	- job_name: 'prometheus'
	scrape_interval:     5s
	static_configs:
	- targets: ['localhost:9090']

启动服务
systemctl daemon-reload
systemctl start prometheus.service
systemctl enable prometheus.service
systemctl status prometheus.service
重启prometheus:
systemctl reload prometheus.service

访问 Web 页面

node_exporter 部署步骤：

groupadd prometheus
useradd -g prometheus -M -s /sbin/nologin prometheus

tar -zxvf /opt/module/node_exporter-1.1.2.linux-amd64.tar.gz -C /usr/local/
cd /usr/local
mv /usr/local/node_exporter-1.1.2.linux-amd64/ /usr/local/node_exporter
chown -R prometheus.prometheus /usr/local/node_exporter

vi /usr/lib/systemd/system/node_exporter.service

#Prometheus Node Exporter Upstart script
[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
ExecStart=/usr/local/node_exporter/node_exporter

[Install]
WantedBy=default.target

启动node_exporter(localhost:9100)
systemctl daemon-reload
systemctl enable node_exporter
systemctl start node_exporter
systemctl status node_exporter

如果可以正常获取到上游的指标后，我们可以将 node_exporter 集成到 prometheus

vim  /usr/local/prometheus/prometheus.yml(文件最后三行)

- job_name: 'node-01_agent1' # 取一个job名称来代 表被监控的机器
  static_configs:
  - targets: ['172.20.10.7:9100'] # 这里改成被监控机器 的IP，后面端口接9100 
设置完成后，保存，然后重新启动Prometheus
重启prometheus:
systemctl reload prometheus.service

可以看到 prometheus 已经获取到 job 了

alertmanager 部署步骤：

tar -zxvf alertmanager-0.22.0-rc.1.linux-amd64.tar.gz -C /usr/local/
mv /usr/local/alertmanager-0.22.0-rc.1.linux-amd64/ /usr/local/alertmanager

vi /usr/lib/systemd/system/alertmanager.service
[Unit]
Description=Alertmanager
After=network-online.target

[Service]
Restart=on-failure
ExecStart=/usr/local/alertmanager/alertmanager --config.file=/usr/local/alertmanager/alertmanager.yml

[Install]
WantedBy=multi-user.target

启动服务
systemctl daemon-reload
systemctl enable alertmanager
systemctl start alertmanager
查看状态
systemctl status alertmanager
netstat -nltup|grep 9093

grafana 部署步骤：

tar -zxvf grafana-7.5.0.linux-amd64.tar.gz -C /usr/local/
mv /usr/local/grafana-7.5.0/ /usr/local/grafana

vi /usr/lib/systemd/system/grafana-server.service
[Unit]
Description=Grafana Server
After=network-online.target
[Service]
Restart=on-failure
ExecStart=/usr/local/grafana/bin/grafana-server -config /usr/local/grafana/conf/defaults.ini -homepath /usr/local/grafana

[Install]
WantedBy=multi-user.target

启动服务
systemctl daemon-reload
systemctl enable grafana-server
systemctl start grafana-server
查看状态
systemctl status grafana-server
netstat -nltup|grep 3000

grafana 展示模板可直接导入网盘 JSON 文件：
进入 grafana Web 页面

添加 prometheus 数据源

选择 prometheus

添加数据源成功

导入 JSON 文件

面板示例