Skip to main content

Kafka® - Capabilities and limitations

Stephanie R.
Updated

This page provides the technical capabilities and limitations of the Cloud Analytics with Kafka® offer.

We continuously improve our offers. You can follow and submit ideas to add to our roadmap.

 

Capabilities and limitations

Supported regions

To view Cloud Analytics' availability, please see our regions and availability webpage.

Entire Analytics instances have to be in the same region.

 

Kafka® versions

The Cloud Analytics offer supports the following Kafka® versions:

  • Kafka® 3.8
  • Kafka® 3.9
  • Kafka® 4.0

Please refer to our Lifecycle policy guide for recommendations on version upgrades and end-of-life announcements of major versions. Additionally, you can follow the Kafka® Release Cycle on their official page.

 

Kafka® clients

You can use any of the Kafka®-recommended clients to access your cluster.

Kafka® Connect and Kafka® Mirrormaker 2 are available at OVHcloud.

 

Plans

The plans available are:

  • Business: 3 nodes
  • Enterprise: 6 nodes

Your choice of plan affects the number of nodes your cluster can run, the SLA, and a few other features, such as backup retention.

 

License type

Kafka® software is under the Apache 2 license, a liberal open-source license. More information can be found here.

 

Hardware resources

For more information about node types and their specifications, see our pricing page.

 

Flexible storage

You can increase your cluster's storage to the maximum allowed for the given reference. Please refer to our Resize your service storage guide for more information.

 

Node template upgrade

You can upgrade the node template of your cluster to scale your hardware resources up. This operation causes no interruption of service, but be aware that you will not be able to downgrade the node template afterwards.

 

Disk type

The type of storage available may vary according to the region your cluster lives in: see Availability of Public Cloud products for more information about block storage type availability depending on the region. Thus, your cluster may be backed by, e.g., High Speed or High Speed Gen2 block storage.

Also, the performance characteristics of the various storage offerings may vary depending on, e.g., the storage size your cluster uses: High Speed may offer better IOPS than High Speed Gen2 for some disk sizes. See our Block Storage webpage for more information about those performance characteristics.

Cloud Analytics will select the most efficient disk type for your cluster depending on your cluster parameters.

 

Effective storage

The disk size listed above is the total disk size of the underlying machine. However, a small part of it goes towards the OS install.

We try hard to avoid "disk full" situations that could be harmful to cluster health. Therefore:

  1. When reaching a concerning level of disk usage, a warning email is sent.
  2. When reaching a concerning level of disk usage, the service is moved to the "DISK_FULL" state, and "read-only" mode, meaning no more writes can be done.
  3. You can then upgrade to a higher service plan with more storage.

See our Handling «Disk Full» situations guide for more information.

 

Features

Network

Kafka® clusters are reachable through a random port, assigned during cluster creation. Once your cluster is in RUNNING status, the Service URI will display the port to use.

Public as well as private networking (vRack) can be used for all the offers.

Ingress and Egress traffic are included in the service plans and are unmetered.

The Analytics service's IP address is subject to change periodically. Thus, it is advised not to rely on these IPs for any configuration, such as connection or egress policy. Instead, utilize the provided DNS record and implement CIDR-based egress policies for more robust and flexible network management.

Private network considerations

Here are some considerations to take into account when using a private network:

  • Network ports are created in the private network of your choice. Thus, further operations on that network might be restricted - e.g., you won’t be able to delete the network if you didn’t stop the Cloud Analytics services first.
  • When connecting from an outside subnet, the OpenStack IP gateway must be enabled in the subnet used for the Analytics service. The customer is responsible for any other custom network setup.
  • Subnet sizing should include considerations for service nodes, other co-located services within the same subnet, and an allocation of additional available IP addresses for maintenance purposes. Failure to adequately size subnets could result in operational challenges and the malfunctioning of services.
  • OpenStack subnet routes announcement will not be applied to your services.
  • You can only create private network services if you are the original owner of the network. You cannot create private network services on a shared network.
Authorized IPs

Once your service is up and running, you will be able to specify IP addresses (or CIDR blocks) to authorize incoming traffic. Until then, your service will be unreachable.

 

Maximum simultaneous connections

The number of simultaneous connections in Cloud Analytics with Kafka® depends on the available total memory on the node. We allow approximately 100 connections per 4 GB of RAM, capped at a maximum of 1000 active connections.

So, for example, on a server with 7 GB of memory, you will get approximately 200 connections, and with 15 GB of memory, you will get 400 connections.

 

Schema registry

Schema registry is supported and can be activated on a running Kafka® service by setting the schemaRegistry to true with an API call:

PUT /cloud/project/{serviceName}/database/kafka/{clusterId}

 

Advanced parameters

You can further customize your Kafka® by using advanced parameters. See our Kafka® - Advanced parameters references guide for more information on the supported parameters.

 

Backups

Kafka® is a streaming tool. We don't back up Kafka® data.

 

Logs and metrics

Logs and metrics are available through the Control Panel and the API. Additionally, cross-service integration can be configured to leverage your logs and metrics in other Cloud Databases and Analytics services. You could then view your Kafka® logs in OpenSearch™ and metrics in Dashboards service (metrics have to be exported first in a time series-compatible engine such as PostgreSQL®). See our Set up a service integration guide for more information.

  • Logs retention: 1000 lines of logs
  • Metrics retention: 1 calendar month

Please note that if the analytics instance is deleted, logs and metrics are also automatically deleted.

 

Users and roles

The creation of users is allowed via the Control Panel and API.

You can specify a username for each user. By default, the role is admin.

 

Topics

The creation of topics is allowed via the Control Panel and API.

For each topic, you can specify:

  • A topic name;
  • The replication factor;
  • The number of partitions;
  • The size of partitions;
  • The retention time;
  • The minimum in-sync replication;
  • The deletion policy.

 

Data encryption

For detailed information about encryption, see our Security Overview guide.

 

Go further

For more information and tutorials, please see our other Managed Databases & Analytics or Platform as a Service guides. You can also explore the guides for other OVHcloud products and services.

OVHcloud Managed Databases and Analytics:

  • Grafana® is a registered trademark of Grafana Labs and is used with the permission of Grafana Labs. OVH SAS and its subsidiaries are not affiliated with or endorsed by Grafana Labs.
  • Kafka® is a registered trademark of The Apache Software Foundation and has been licensed for use by OVHcloud, who has no affiliation with and is not endorsed by The Apache Software Foundation.
  • MongoDB® is a registered trademark of MongoDB, Inc.
  • MySQL® is a registered trademark of Oracle and/or its affiliates.
  • PostgreSQL® is a registered trademark of the PostgreSQL Community Association of Canada.

Related articles