Using mongodb exporter + Prometheus + grafana to implement mongodb cluster monitoring

1 create Mongodb cluster

Directly use the following yaml file to create a mongodb cluster. In this environment, only two nodes of mongodb are started. Use hostPath for data storage and create storage directory

[root@master ~]# mkdir -p /home/storage/mongo/db/
[root@master mongo]# cat mongo.yaml
apiVersion: v1
kind: Service
metadata:
  name: mongo
  labels:
    app: mongo
spec:
  ports:
  - name: mongo
    port: 27017
    targetPort: 27017
  clusterIP: None
  selector:
    app: mongo
---
apiVersion: v1
kind: Service
metadata:
  name: mongo-service
  labels:
    app: mongo
spec:
  ports:
  - name: mongo-http
    port: 27017
  selector:
    app: mongo
  type: NodePort
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mongo
spec:
  selector:
    matchLabels:
      app: mongo
  serviceName: "mongo"
  replicas: 2
  podManagementPolicy: Parallel
  template:
    metadata:
      labels:
        app: mongo
    spec:
      terminationGracePeriodSeconds: 10
      affinity:
         podAntiAffinity:
           requiredDuringSchedulingIgnoredDuringExecution:
           - labelSelector:
               matchExpressions:
               - key: "app"
                 operator: In
                 values:
                 - mongo
             topologyKey: "kubernetes.io/hostname"
      containers:
      - name: mongo
        image: mongo:latest
        command:
        - mongod
        - "--bind_ip_all"
        - "--replSet"
        - rs0
        ports:
        - containerPort: 27017
        volumeMounts:
        - name: mongo-data
          mountPath: /data/db
      volumes:
      - name: mongo-data
        hostPath:
          path: /home/storage/mongo/db
[root@master mongo]# kubectl apply -f mongo.yaml
[root@master mongo]# kubectl get pod
NAME                              READY   STATUS    RESTARTS   AGE
details-v1-74f858558f-rckf4       2/2     Running   6          9d
**mongo-0                           2/2     Running   2          22h
mongo-1                           2/2     Running   2          22h**
productpage-v1-76589d9fdc-vp6h5   2/2     Running   6          9d
ratings-v1-7855f5bcb9-fd98w       2/2     Running   6          9d
reviews-v1-64bc5454b9-knhcs       2/2     Running   6          9d
reviews-v2-76c64d4bdf-wmsgq       2/2     Running   6          9d
reviews-v3-5545c7c78f-hsw5h       2/2     Running   6          9d
[root@master mongo]# kubectl get svc
NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)           AGE
details         ClusterIP   10.10.179.23    <none>        9080/TCP          9d
kubernetes      ClusterIP   10.10.0.1       <none>        443/TCP           14d
mongo           ClusterIP   None            <none>        27017/TCP         22h
mongo-service   NodePort    10.10.27.148    <none>        27017:31014/TCP   22h
productpage     ClusterIP   10.10.112.221   <none>        9080/TCP          9d
ratings         ClusterIP   10.10.183.94    <none>        9080/TCP          9d
reviews         ClusterIP   10.10.86.216    <none>        9080/TCP          9d
[root@master mongo]#

As shown above, mongodb cluster creation is completed

2 test whether the creation is successful

Using curl to test whether mongodb works properly

[root@master mongo]# curl http://192.168.122.7:31014
It looks like you are trying to access MongoDB over HTTP on the native driver port.

If the above prompt appears, mongodb starts successfully

3 prepare mongodb exporter image

Pull mongodb exporter image

[root@master mongo]# docker pull noenv/mongo-exporter:latest

4. Create mongodb exporter application, service, serviceaccount and servicemonitor

Use the mongodb exporter image to create a deployment. The author of this image specifies that the image of mongodb uri is 172.17.0.1 when creating this image. Therefore, when creating a deployment, you need to re specify the uri address of mongodb. Also, note that the deployment needs to be created in monitoring Under namespace, this namespace is the Kube Prometheus namespace. For the deployment mode of Kube Prometheus, the following article will show. Specific yaml documents are as follows:

[root@master mongo]# cat mongo-exporter.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongo-exporter
  namespace: monitoring
  labels:
    k8s-app: mongo-exporter
spec:
  selector:
    matchLabels:
      k8s-app: mongo-exporter
  template:
    metadata:
      labels:
        k8s-app: mongo-exporter
    spec:
      containers:
      - name: mongo-exporter
        image: noenv/mongo-exporter:latest
        args: ["--web.listen-address=:9104", "--mongodb.uri", "mongodb://192.168.122.7:31014"]
        ports:
        - containerPort: 9104
          name: http

Create deployment

[root@master mongo]# kubectl apply -f mongo-exporter.yaml
deployment.apps/mongo-exporter created
[root@master mongo]# kubectl get pod -n monitoring
NAME                                  READY   STATUS    RESTARTS   AGE
alertmanager-main-0                   2/2     Running   12         14d
alertmanager-main-1                   2/2     Running   12         14d
alertmanager-main-2                   2/2     Running   12         14d
grafana-77978cbbdc-szzbd              1/1     Running   6          14d
kube-state-metrics-7f6d7b46b4-wnkn6   3/3     Running   18         14d
**mongo-exporter-7557899dcc-xpzh5       1/1     Running   0          114s**
node-exporter-55259                   2/2     Running   12         14d
node-exporter-rr2bf                   2/2     Running   2          29h
prometheus-adapter-68698bc948-65blv   1/1     Running   6          14d
prometheus-k8s-0                      3/3     Running   19         14d
prometheus-k8s-1                      3/3     Running   19         14d
prometheus-operator-6685db5c6-7dbvj   1/1     Running   6          14d
[root@master mongo]#

Create service

[root@master mongo]# cat mongo-exporter-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: mongo-exporter
  name: mongo-exporter
  namespace: monitoring
spec:
  type: NodePort
  ports:
  - name: http
    port: 9104
    nodePort: 30017
    targetPort: http
  selector:
    k8s-app: mongo-exporter
[root@master mongo]# kubectl create -f mongo-exporter-service.yaml
service/mongo-exporter created
[root@master mongo]# kubectl describe svc -n monitoring mongo-exporter
Name:                     mongo-exporter
Namespace:                monitoring
Labels:                   k8s-app=mongo-exporter
Annotations:              <none>
Selector:                 k8s-app=mongo-exporter
Type:                     NodePort
IP:                       10.10.84.94
Port:                     http  9104/TCP
TargetPort:               http/TCP
NodePort:                 http  30017/TCP
Endpoints:                10.124.72.5:9104
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>
[root@master mongo]#

Create serviceacout

[root@master mongo]# kubectl create serviceaccount -n monitoring mongo-exporter
serviceaccount/mongo-exporter created

Create serviceMonitor

[root@master mongo]# cat mongo-exporter-serviceMonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    k8s-app: mongo-exporter
  name: mongo-exporter
  namespace: monitoring
spec:
  endpoints:
  - interval: 30s
    port: http
  jobLabel: k8s-app
  selector:
    matchLabels:
      k8s-app: mongo-exporter
[root@master mongo]# kubectl apply -f mongo-exporter-serviceMonitor.yaml
servicemonitor.monitoring.coreos.com/mongo-exporter created

5 view prometheus Targets

After all the above resources are created, check mongodb exporter targets in prometheus

At this time, prometheus can collect mongodb cluster performance data normally

6 create monitoring panel in grafana

Select the official mongodb template on the grafana page


Click Import to monitor mongodb

Why do some data not exist? Because: 1. There is no data stored in mongodb. 2. Some indicator names in the Prometheus mongodb exporter statement in the grafana template do not exist.

Tags: Database MongoDB Kubernetes curl Docker

Posted on Fri, 08 May 2020 07:40:46 -0700 by Neoraven456