Monitoring and Profiling

A Feldera instance and its pipelines can be monitored using various tools. This tutorial will guide you through setting up monitoring for your Feldera instance.

Metrics with Grafana and Prometheus

Metrics are helpful to check the health of your Feldera instance and to identify resource bottlenecks. Feldera exposes a metrics endpoint that can be scraped by Prometheus. Grafana is then used to visualize these metrics.

See Pipeline Metrics for a reference to the Prometheus metrics that Feldera exports.

Setup

1. Prometheus: You must have Prometheus installed.
2. Connect Prometheus to Feldera:
   • Add the following to your prometheus.yml configuration file, usually located in /etc/prometheus/prometheus.yml:

   - job_name: 'feldera'
     scrape_interval: 1s
     metrics_path: '/v0/metrics'
     static_configs:
       - targets: ['127.0.0.1:8080']

   • Replace 127.0.0.1:8080 with the address of your Feldera instance.
   • Restart Prometheus.
3. Grafana: You must have Grafana installed.
4. Add Prometheus To Grafana:
   • If you are using a local prometheus instance, the URL for the Prometheus data source will be http://127.0.0.1:9090.
   • Follow the steps in the Grafana documentation to add Prometheus as a data source.

Setup with Docker

Alternatively, with docker, you can avoid installing Prometheus and Grafana to your local machine. To run Feldera with both Prometheus and Grafana:

docker compose -f deploy/docker-compose.yml up grafana --force-recreate

This spins up:

1. Feldera
2. Prometheus
3. Grafana

If you want to run Prometheus without Grafana:

docker compose -f deploy/docker-compose.yml up prometheus --force-recreate

Set up the monitoring Dashboard

1. **Copy the JSON of the Feldera template dashboard **
2. Import the dashboard into Grafana
   • Under Dashboards, click the "New" icon, then click "Import Dashboard".
   • Paste the JSON copied from the template in the text-box and click "Load".

A Feldera Instance Monitoring dashboard will be created in Grafana. The dashboard is a template and may need to be adjusted to fit your specific needs. Look for the feldera_* metrics in Grafana to add more metrics to the dashboard.

DBSP Profiles

A DBSP profile is a graph of the pipeline's circuit where each node represents an operator and each edge represents a data flow between operators. The profile includes information about how much CPU time or memory each operator consumes.

The API endpoint /v0/<pipeline_name>/circuit_profile can be used to download the DBSP profile of a running pipeline. It returns a zip file containing multiple profiles (one for each worker) as .dot files, and a Makefile to transform the .dot files into .pdf files.

Alternatively, profile data can be browsed iteractively using the WEB UI.