Custom Exporter with Prometheus

0
1345
prometheus-monitoring
prometheus-monitoring

Prometheus is an excellent tool for collecting the metrics. In the previous article, I have explained the different data type of Prometheus. If you missed it please view this article here.

Sometimes we need to implement the custom exporter for Prometheus. For example, if I am having metrics already present in another system then I just need to export it. In such cases you might to SET the value for the counter. However, the counter is not designed for setting up the value. Gauge type will do that for you. There are numerous exporters are already available in Prometheus.

Prometheus provides custom collectors to solve these problems. You can write the exporters in supported client libraries.

I have written the custom exporter using the python client. You can deploy it using the Kubernetes as well.

In the exporter, we will have gauge and counter metrics exposed. For this, I ‘ll use the prometheus_client to import the libraries. I have the repository on Github you can check it here.

There are three steps:

  1. Writing the custom exporter in python for Prometheus.
  2. Dockerize the exporter
  3. Deploy on Kubernetes

You can add your custom logic with the exporter.

Lets check the first step:
/code/collector.py:

import time
from prometheus_client.core import GaugeMetricFamily, REGISTRY, CounterMetricFamily
from prometheus_client import start_http_server


class CustomCollector(object):
def __init__(self):
pass

def collect(self):
g = GaugeMetricFamily("MemoryUsage", 'Help text', labels=['instance'])
g.add_metric(["instance01.us.west.local"], 20)
yield g

c = CounterMetricFamily("HttpRequests", 'Help text', labels=['app'])
c.add_metric(["example"], 2000)
yield c


if __name__ == '__main__':
start_http_server(8000)
REGISTRY.register(CustomCollector())
while True:
time.sleep(1)

I am exposing the static gauge and counter metrics. But you can just modify it with your logic from another system.

In second step we will dockerize it:

Make sure you have pip-requirements:
/code/pip-requirements.txt

prometheus_client

Dockerfile:

FROM python:3.6

ADD code /code
RUN pip install -r /code/pip-requirements.txt

WORKDIR /code
ENV PYTHONPATH '/code/'

CMD ["python" , "/code/collector.py"]

In the last third step you can deploy the collector on Kubernetes.
/yaml/deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus-custom-collector-deployment
labels:
app: prometheus-custom-collector
spec:
replicas: 1
selector:
matchLabels:
app: prometheus-custom-collector
template:
metadata:
labels:
app: prometheus-custom-collector
spec:
containers:
- name: prometheus-custom-collector
image: kubejack/prometheus-custom-collector:0.2
ports:
- containerPort: 80

/yaml/service.yaml:

apiVersion: v1
kind: Service
metadata:
name: prometheus-custom-collector-service
spec:
selector:
app: prometheus-custom-collector
type: NodePort
ports:
- protocol: TCP
port: 80
targetPort: 8000

Final step will be deploying the collector on Kubernetes:

$kubectl create -f yaml/

Now you can easily add your endpoint to the prometheus and start collecting the metrics.

Example:

Please check following useful links for references:

Reference :

Github: https://github.com/jakirpatel/prometheus-custom-collector

Github: https://github.com/prometheus/client_python

Web: https://www.robustperception.io/setting-a-prometheus-counter

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments