Public Cloud Load Balancer - TCP / HTTP / HTTPS logs forwarding

Glossary

• Logs Data Platform: a fully managed and secured log management platform by OVHcloud. For more information, consult the Logs Data Platform service page.
• Data Stream: a logical partition of logs that you create in an LDP account and which you will use when ingesting, viewing, or querying your logs. Multiple sources can be stored in the same data stream, and it is the unit that can be used to define a log pipeline (retention policy, archiving, live streaming, etc.), access rights, and alert policies.
• Logs forwarding: a feature integrated into an OVHcloud product to ingest the logs of its services into a Data Stream of an LDP account in the same OVHcloud account. This feature must be activated by the customer and per service.
• Logs forwarding Subscription: when enabling the logs forwarding for a given OVHcloud service to a given LDP Data Stream, a Subscription is created and attached to the Data Stream for further management by the customer.

Requirements

• A Logs Data Platform account with at least one active Stream configured (see our Quick Start for Logs Data Platform guide for more information)

  If you are not familiar with all the LDP Stream configuration possibilities, we suggest you create a new one with the default options (indexing & WebSocket enabled, long-term storage disabled) for this guide.

• An up-and-running Public Cloud Load Balancer
• Both the LDP account and the Public Cloud Load Balancer account must belong to the same OVHcloud account.

Concepts & limits

What are the logs of a Public Cloud Load Balancer?

The forwarded logs are generated by HAProxy (the open-source component used for load balancing).

Log content for TCP-based listeners: TCP, HTTP, TERMINATED_HTTPS, HTTPS

Additional content for HTTP & TERMINATED_HTTPS listeners

Instructions

Note that the forwarding activation is free of charge, but you will be charged for the usage of the Logs Data Platform service as per the standard price plan. For LDP pricing, refer to this page.

Enabling Public Cloud Load Balancer Log Forwarding using the OVHcloud Control Panel

From the OVHcloud Control Panel, navigate to your Load Balancer via the Public Cloud menu.

Once there, click the Logs tab.

Logs access through LDP

From the Logs tab, you can Subscribe to an LDP data stream. Once the subscription is enabled, all the logs will be pushed to Logs Data Platform to archive generated logs and perform advanced searches, and create alerts or visualizations.

For more information, please refer to our Quick start for Logs Data Platform guide.

Enabling Audit Log Forwarding using APIs

You will have to define the targeted Stream of one of your LDP accounts to which you want your logs forwarded. The enablement of the forwarding will create a subscription for this stream ID.

You can retrieve the API specifications in the OVHcloud API Portal.

Step 1 - Retrieve your target Stream (and ID)

List data streams of your Logs Data Platform account (enter your LDP ID in the form ldp-xx-xxxx into the field "serviceName"):

Get the details of a data stream:

Step 2 - Create your subscription

Use the following API call to create a subscription:

You will need to replace:

• loadBalancerId: this is the Load Balancer ID, you can find it on the details page of your Load Balancer in the OVHcloud Control Panel or using the dedicated API call.
• regionName: the OpenStack region, for example, US-EAST-VA.
• serviceName: the Public Cloud Project ID. You can find it in the OVHcloud Control Panel under your project name or using the dedicated API call.

The POST request has a payload that requires:

• kind: haproxy is the only supported kind
• streamId: the target data stream of your LDP account where you want your Public Cloud Load Balancer logs to be forwarded.

{
  "kind": "string", // Currently the only supported value is 'haproxy'.
  "streamId": "18d602ec-af40-4000-8e59-41ecc8c23f80" // The streamID of the targeted Stream.
}

You will get in response an operationId:

{
  "operationId": "f550aa1c-89ab-4b1a-81ae-4fba4959966f",
  "serviceName": "ldp-xx-xxxxx"
}

You can use the operationId to retrieve the subscriptionId for further management purposes using the following api call:

Alternatively, once the operation is finished, the subscriptions can be retrieved using the following API call:

Once you have the subscriptionId, you can get the details using the following API call:

{
  "createdAt": "2024-01-31T15:45:25.286Z",
  "kind": "string",
  "resource": {
    "name": "string",
    "type": "string"
  },
  "serviceName": "string",
  "streamId": "string",
  "subscriptionId": "18d60324-b260-4000-83db-b484f4db6e80",
  "updatedAt": "2024-01-31T15:45:25.286Z"
}

How to use Public Cloud Load Balancer logs?

Now that your logs are ingested and stored in your Logs Data Platform data stream, you can query your logs and build dashboards to have a graphical representation of your logs using the web-based UI of Graylog.

• In the OVHcloud Control Panel, retrieve the LDP username (ex: logs-xxxx) and its password in your Logs Data Platform account home page. You can refer to our Quick Start guide for Logs Data Platform.
• Open the Graylog web-ui. You can retrieve the link in your account home page or using your Access point depending on your account region (for example: https://bh2.logs.ovh.us/).
• Log into Graylog using your Logs Data Platform Username and Password.
• Search through your logs across the data stream of your Logs Data Platform account. You can refer to Graylog writing search queries documentation for details on search syntax.

Refer to the following documentation: Logs Data Platform - Visualizing, Querying, and Exploiting Your Logs for more details about how to use your logs with Logs Data Platform, including how to:

• setup alerts
• view the logs in real time through a WebSocket
• build visualization with OpenSearch Dashboards
• integrate with OpenSearch API
• connect with Grafana

How to manage your subscriptions?

At any time, you can retrieve the subscriptions attached to your Logs Data Platform data stream and choose to disable the forwarding by cancelling your subscription on your stream, so that your Logs Data Platform stream no longer receives your audit logs.

Note that this does not delete the logs stored before the subscription was cancelled, as the data stored in a log stream is immutable unless you delete the entire stream.

To delete your subscription you can use the following API call:

Go further

For more information and tutorials, please see our other Public Cloud support guides or explore the guides for other OVHcloud products and services.

If you need training or technical assistance to implement our solutions, contact your sales representative or click on this link to get a quote and ask our Professional Services experts for a custom analysis of your project.