Public Cloud Load Balancer Monitoring with Prometheus

Learn about the options to monitor your Octavia Load Balancer.

The Public Cloud Load Balancer provides two ways to monitor your Load Balancers: You can either query an API to get usage statistics or configure a Prometheus listener that can be scraped by your Prometheus infrastructure to gather metrics.

Requirements:

• a Load Balancer created

Instructions

Monitoring using CLI

The Load Balancer service aggregates these statistics and makes them available via the OpenStack API. Those statistics are available at the Load Balancer or listener level.

Load Balancer statistics can be queried using the OpenStack Client. They include metrics for all the listeners configured for the queried Load Balancer.

$ openstack loadbalancer stats show <lb id>
+--------------------+-----------+
| Field              | Value     |
+--------------------+-----------+
| active_connections | 0         |
| bytes_in           | 2236722   |
| bytes_out          | 100973832 |
| request_errors     | 0         |
| total_connections  | 3606      |
+--------------------+-----------+

Individual listener statistics can also be queried using the OpenStack Client.

$ openstack loadbalancer listener stats show <listener id>
+--------------------+-------+
| Field              | Value |
+--------------------+-------+
| active_connections | 0     |
| bytes_in           | 89    |
| bytes_out          | 237   |
| request_errors     | 0     |
| total_connections  | 1     |
+--------------------+-------+

Monitoring with Prometheus

To add a Prometheus endpoint on a Public Cloud Load Balancer, create a listener with a PROMETHEUS protocol. This will enable the /metrics endpoint which listens on all the interfaces from your load balancer by default. This means that if your load balancer is using a Floating IP, by default the /metrics endpoint will be opened to the internet. To avoid that, we advise setting the allowed_cidrs option to protect your metrics, for example with the CIDR of the subnet where the load balancer is spawned.

This listener type provides the same features as the "regular" listeners but does not support attaching pools or L7 policies. All metrics will be identified by the Octavia object ID (UUID) of the resources.

Create a Prometheus listener via the OVHcloud Control Panel

From the OVHcloud Control Panel:

1. Select Public Cloud from the top navigation bar.
2. In the left-hand menu under Network, select Load Balancer.
3. Select the name of the load balancer.

4. Select the Listeners tab.
5. Click + Add a listener.
6. Fill in a name, select Prometheus as the Protocol, and choose a Port (different from the already existing listener ports of your Load Balancer).
   The default pool is grayed out because the Prometheus listener does not need a pool, unlike the other types of listeners.

7. Click Add to trigger the listener creation.

The new listener is added to the listeners list.

Create a Prometheus listener via OpenStack GUI (Horizon)

Log in to Horizon using this guide.

Click Network and then Load Balancers. The Load Balancer list is displayed.

Click on the load balancer name. The load balancer details page is then displayed.

Click the Listeners tab and then Create Listener.

On the listener creation page, fill in the Name and switch the Protocol to PROMETHEUS. The port will be set to a default value that you can change if needed.

Click Create Listener.

Create a Prometheus listener via OpenStack CLI

To create a Prometheus endpoint on port 9100 for Load Balancer lb1, you would run the following command:

$ openstack loadbalancer listener create --name stats-listener --protocol PROMETHEUS --protocol-port 9100 lb1
+-----------------------------+--------------------------------------+
| Field                       | Value                                |
+-----------------------------+--------------------------------------+
| admin_state_up              | True                                 |
| connection_limit            | -1                                   |
| created_at                  | 2023-11-30T01:44:25                  |
| default_pool_id             | None                                 |
| default_tls_container_ref   | None                                 |
| description                 |                                      |
| id                          | fb57d764-470a-4b6b-8820-627452f55b96 |
| insert_headers              | None                                 |
| l7policies                  |                                      |
| loadbalancers               | b081ed89-f6f8-48cb-a498-5e12705e2cf9 |
| name                        | stats-listener                       |
| operating_status            | OFFLINE                              |
| project_id                  | 4c1caeee063747f8878f007d1a323b2f     |
| protocol                    | PROMETHEUS                           |
| protocol_port               | 9100                                 |
| provisioning_status         | PENDING_CREATE                       |
| sni_container_refs          | []                                   |
| timeout_client_data         | 50000                                |
| timeout_member_connect      | 5000                                 |
| timeout_member_data         | 50000                                |
| timeout_tcp_inspect         | 0                                    |
| updated_at                  | None                                 |
| client_ca_tls_container_ref | None                                 |
| client_authentication       | NONE                                 |
| client_crl_container_ref    | None                                 |
| allowed_cidrs               | None                                 |
| tls_ciphers                 | None                                 |
| tls_versions                | None                                 |
| alpn_protocols              | None                                 |
| tags                        |                                      |
+-----------------------------+--------------------------------------+

Note that you should add the --allowed-cidr option to filter the listener to a specific network.

Create a Prometheus listener via Terraform

The resource openstack_lb_listener_v2 from the OpenStack provider enables you to configure a Prometheus listener. The following snippet is extracted from a full example available on GitHub that you can adapt to your needs.

resource "openstack_lb_listener_v2" "prom_listener" {
 protocol = "PROMETHEUS"
 protocol_port = 9100
 loadbalancer_id = openstack_lb_loadbalancer_v2.tf_lb.id
 #restrict the access of the listener to the private network subnet
 allowed_cidrs = [openstack_networking_subnet_v2.tf_lb_subnet.cidr]
}

Configure Prometheus to collect your metrics

Once the PROMETHEUS listener is ACTIVE, you can configure Prometheus to collect metrics from the Load Balancer by updating the prometheus.yml file.

[scrape_configs]
- job_name: 'Octavia LB1'
  static_configs:
  - targets: ['loadbalancer-IP:9100']

global:
  scrape_interval:     15s # By default, scrape targets every 15 seconds.

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
    monitor: 'codelab-monitor'

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'
    
    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s 
   
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'Octavia LB1'
    static_configs:
      - targets: ['10.xxx.xxx.170:9100']

For more information on setting up Prometheus, see the Prometheus project website.

You can connect Grafana to the Prometheus instance to provide additional graphing and dashboard capabilities. A Grafana dashboard for Octavia Load Balancers is available here.

The metrics have the following naming format:

• octavia_loadbalancer_xxx or octavia_memory_pool_xxx metrics are global to the load balancer
• octavia_listener_xxx metrics are instantiated per listener using the label {listener=ID}
• octavia_pool_xxx metrics are instantiated per pool using the label {pool=ID}
• octavia_member_xxx metrics are instantiated per member using the label {member=ID}

Go further

If you want to automate this scenario with Terraform and our managed Grafana service, you can find a ready-to-run example on this GitHub repository.

For more information and tutorials, please see our other Public Cloud Networking support guides or explore the guides for other OVHcloud products and services.

If you need training or technical assistance to implement our solutions, contact your sales representative or click on this link to get a quote and ask our Professional Services experts for a custom analysis of your project.