How to Mitigate the Downfall Vulnerability – Articles

Introduction

On August 8th, 2023, Intel issued a security bulletin disclosing a vulnerability in its recent computer processor microarchitecture. Named "Gather Data Sampling (GDS)" by Intel, the vulnerability is also known as "Downfall".
Labeled CVE-2022-40982 and rated by Intel as Medium, the issue allows an attacker to potentially access sensitive information processed by the CPU in specific circumstances.
The issue affects all software running on the affected processors, including virtual machines, sandboxes, containers, and processes. Exploitation software has been released by researchers and shows how this vulnerability could be leveraged in a multi-user context.

At least the following microarchitectures are vulnerable:

Amber Lake
Cascade Lake
Coffee Lake
Cooper Lake
Ice Lake
Rocket Lake
Skylake
Tiger Lake
Tiger Lake Refresh
Whiskey Lake

Impacts on OVHcloud products

In response to that event, we immediately reviewed the security bulletin and technical information and determined the following potential impact on our products:

Range of products	Products	Impact
Public Cloud	All products	Mitigated by OVHcloud
Hosted Private Cloud	VMware on OVHcloud	Update in progress by OVHcloud
Hosted Private Cloud	SDDC / Essentials	Not impacted
Bare Metal Cloud	ADVANCE-1 Gen 2 ADVANCE-2 Gen 2 ADVANCE-6 Gen 2 GAME-1 GAME-2 HGR-AI-1 HGR-HCI-1 HGR-HCI-2 HGR-HCI-3 HGR-SDS-1 HGR-STOR-1 KS RISE-1 RISE-2 RISE-3 RISE-5 SCALE-4 SCALE-5 SCALE-6 SYS-2 SYS-3 SYS-4 SYS-4-SAT-32 SYS-4-SSD-16 SYS-4-SSD-32 SYS-5 SYS-5-SAT-32 SYS-5-SAT-64 SYS-5-SSD-32 SYS-5-SSD-64 SYS-GAME-1 SYS-GAME-2	Potentially impacted (check below for how to mitigate the vulnerability)
Bare Metal Cloud	Other commercial ranges of dedicated servers.	Not impacted

How to mitigate the vulnerability

Customer-initiated mitigation

Loading a patched microcode at boot with a firmware package update

This solution will trigger the update of the processor microcode through an operating system update (the Linux firmware package for instance). You might do it as soon as your OS editor or community distribute the updated package. This method is dependent on your distribution or Operating System editor and will only work if the appropriate microcode has been provided by Intel.

Mitigation with an updated kernel

When an update of the microcode is not available via a firmware update package, you may update the Kernel with a version that implements a way to shut off AVX instruction set support.

It can be achieved by adding the following kernel command line parameter:

gather_data_sampling=force

We recommend using this mitigation carefully since it may have a deep impact on the overall performance of the system.

OVHcloud-initiated mitigation

OVHcloud teams are working to implement transparent solutions that will ensure the patched microcode is updated in a transparent way for our customers. Those solutions will be deployed progressively on our servers using the solution detailed below.

Using OVHcloud iPXE

The microcode update may be loaded by the bootloader when the standard OVHcloud netboot is used by customers (the most common configuration). The feature is available so rebooting the server through the OVHcloud Control Panel will cause it to load the updated microcode before booting to disk, which will mitigate the vulnerability. However, if you are booting on disk without using the OVHcloud netboot system, the mitigation will not be applied and you should consider relying on the Operating-System-level mitigation.

REMINDER: The solution requires a public interface with a public IP and a UEFI boot to work. It is not compatible with legacy boot services.

Using UEFI

The UEFI firmware update may update the CPU microcode at boot. UEFI firmware updates including the patched microcode will likely be made available by motherboard manufacturers within the next months.

Once available, OVHcloud will include this patched microcode on the UEFI for any newly delivered server. Customers will then be able to request a UEFI firmware update by contacting support.

As an administrator of a potentially vulnerable server, what should I do?

The first action is to check if your server is impacted by the vulnerability. You might use the following tool (Linux-only) developed by our team and which has been updated to take into consideration CVE-2022-40982.

wget https://raw.githubusercontent.com/speed47/spectre-meltdown-checker/master/spectre-meltdown-checker.sh
# sh spectre-meltdown-checker.sh --variant downfall --explain

If the tool says NOT VULNERABLE then you are already safe and no further action is needed.

If the tool says VULNERABLE you should then evaluate your exposure to the threat.

It is necessary to determine if the server context allows to run code from an untrusted origin.
If the server is used to provide services to untrusted end-users that can execute code (VPS, Container, shared hosting, etc.), or is used as a desktop in the cloud browsing the Web (hence possibly running 3rd party JavaScript payloads), then your server might be at risk.
If the server is used only by trusted users and/or does not allow to run untrusted code, the risk of exploitation is probably quite low. Please note however that this vulnerability might allow an attacker to gain extra privilege in a chained attack, it could be used for persistence or lateral movement in a complex kill chain.

Based on this evaluation, you should determine the emergency to trigger mitigation and choose the most appropriate one.

What about performances?

The new microcodes have been deployed on our internal servers. Since performances are still a concern in such a situation, our teams closely monitored the impacts of the upgrades. So far, the conclusions are reassuring since we did not notice a deep impact on the overall performances.

What is OVHcloud working on?

Our technical and support teams are working to ensure the risk is lowered for each of our customers impacted by the vulnerability. We mostly focus on:

Informing impacted customers to ensure they take the risk into account in their operations and implement mitigation appropriately.
Developing and integrating updates in our automation to cover the risk in a transparent way for our customers.
Security watch of the vulnerability exploitation in the wild to define the appropriate extra mitigations we can implement to protect our customer infrastructures.