How to Customize CoreDNS on an OVHcloud Managed Kubernetes Cluster – Articles

Objective

The OVHcloud Managed Kubernetes service provides you with Kubernetes clusters without the hassle of installing or operating them.

The OVHcloud Managed Kubernetes clusters are using CoreDNS as a DNS server for Service Discovery.

At OVHcloud, we listen to our users and improve our products and services accordingly, which is why we give you the ability to customize the CoreDNS configuration, thanks to the expansion mechanism in a special Config Map which is not modified by a redeployment.

Requirements

An OVHcloud Managed Kubernetes cluster

Instructions

Displaying the CoreDNS default configuration

In an OVHcloud Managed Kubernetes cluster, DNS is handled by CoreDNS.

Take a look at the ConfigMap to see the default configuration.

kubectl get configmap coredns -n kube-system -o yaml

The result should be similar to the following:

$ kubectl get configmap coredns -n kube-system -o yaml
apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health {
            lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
            pods insecure
            fallthrough in-addr.arpa ip6.arpa
            ttl 30
        }
        prometheus :9153
        forward . /etc/resolv.conf {
          prefer_udp
          policy sequential
        }
        cache 30
        loop
        reload
        loadbalance
        import custom/*.include
        log
    }
    import custom/*.server
kind: ConfigMap
metadata:
  creationTimestamp: "2022-09-22T07:00:39Z"
  labels:
    addonmanager.kubernetes.io/mode: EnsureExists
  name: coredns
  namespace: kube-system
  resourceVersion: "5786561917"
  uid: a4e7310b-50b9-4976-a014-13e4bee12344

Warning: When the Kubernetes Control Plane is redeployed or when the cluster is upgrading to the latest patch, the default CoreDNS configuration is restored.

To understand what this CoreDNS configuration means, please read the CoreDNS official documentation and CoreDNS plugins documentation.

Customizing the CoreDNS configuration

If you want to edit and customize the CoreDNS configuration, it can be done by editing the coredns-custom Config Map, which will never be altered by OVHcloud services:

$ kubectl get cm coredns-custom -n kube-system -o yaml
apiVersion: v1
data:
  example.include: |
    # Include .:53 example
  example.server: |
    # Server example
kind: ConfigMap
metadata:
  creationTimestamp: "2022-10-10T09:14:03Z"
  labels:
    addonmanager.kubernetes.io/mode: EnsureExists
  name: coredns-custom
  namespace: kube-system
  resourceVersion: "5786561924"
  uid: 95c51a5d-c413-4520-b84d-a16892626620

You can add some new configuration in data.example.include and data.example.server blocks.

Edit the coredns-custom Config Map with the following command:

kubectl edit cm coredns-custom -n kube-system

For example, if you want to use a custom nameserver that will point to your private DNS, here are all the FQDN resolutions for *.myprivatedomain.com:

$ kubectl edit cm coredns-custom -n kube-system
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  example.include: |
    # Include .:53 example
  myprivatedomain.server: |
    myprivatedomain.com.:53 {
    forward . [my-custom-dns-server-private-ip]
  }
kind: ConfigMap
metadata:
  creationTimestamp: "2022-10-10T09:14:03Z"
  labels:
    addonmanager.kubernetes.io/mode: EnsureExists
  name: coredns-custom
  namespace: kube-system
  resourceVersion: "5786561924"
  uid: 95c51a5d-c413-4520-b84d-a16892626620

Check the logs to make sure that the customization has been applied:

kubectl logs -n kube-system -l k8s-app=kube-dns -f

Known limitations

Full private DNS resolution is not possible - only private domains can be resolved. This means that public FQDNs are resolved by the public OVHcloud DNS. For example, when resolving kubernetes.io from a Pod, the CoreDNS first resolves to .:53, which is the default nameserver. Then it resolves to the /etc/resolv.conf of the underlying worker node. Then it resolves to the public OVHcloud DNS (which is configured by the OVHcloud Managed Kubernetes).
When upgrading the cluster nodes (or when a node restarts), the CoreDNS pod can be restarted and takes on the new configuration. For this reason, you need to customize the configuration through the coredns-custom Config Map.
CoreDNS is configured to reload its configuration (Config Map here) every 30 seconds.
New Kubernetes clusters will have the new CoreDNS Deployment & Config Maps, whereas the old ones will need to update their control plane to the latest patch.

Articles in this section