How to activate and use rescue mode – Support Guides

Learn how to reboot and use a server in rescue mode.

Rescue mode is a tool provided by OVHcloud that allows you to boot into a temporary operating system to diagnose and resolve issues on your server.

Usual tasks the rescue mode is appropriate for include:

Resetting your user password
Diagnosing network problems
Repairing a broken operating system
Fixing a software firewall misconfiguration
Testing disk performance
Testing CPU and RAM

Requirements

A dedicated server in your OVHcloud account
Access to the OVHcloud Control Panel

Topics

Activating rescue mode
Accessing your server in rescue mode via SSH
Mounting partitions to access your files
Link Aggregation configuration in rescue mode
Exiting rescue mode
How to add a default authentication key for rescue mode
Legacy Windows rescue system (WinPE rescue mode)
How to test Additional IP and vRack functionality in rescue mode

Instructions

To use rescue mode, you must edit the server's Netboot setting. Then the server has to be restarted.

Activating rescue mode

Log in to your OVHcloud Control Panel, open the Bare Metal Cloud section, then Dedicated servers, and select your server from the list.

In the General information box, click the more options ... button next Boot and select Edit.

DS_troubleshooting_rescueMode01*.png

1: Rescue mode options

On the page Change the netboot, select Boot in rescue mode.

The available options for rescue mode depend on the server type and the operating system installed.

Customer rescue system (always available)
Rescue system for Windows (available for Windows servers)
iPXE / ipxe-shell (external open-source tool, always available)
Legacy Windows rescue system (deprecated WinPE system, only relevant if your server does not meet the requirements for the current rescue system for Windows)

The instructions below will only cover the customer rescue system, the most commonly used option.

Please refer to our dedicated guide for a detailed explanation on using the rescue system for Windows.

Select Customer rescue system in the drop-down menu.

DS_troubleshooting_rescueMode02*.png

2: Authentication options

The next choice determines the method of authentication for the SSH connection to the rescue mode system. This is mainly a matter of convenience since each rescue mode session is meant to be transitory and will be discarded once you reboot the server from its disk.

Password authentication: You will be sent login credentials via email.
Key authentication: You can use a public authentication key of your choice (compatible formats: RSA, ECDSA, ED25519).

Click on the relevant tab for your connection method:

Click Password authentication.

The rescue mode notification email including its login details will be sent to the contact email address of your OVHcloud account. To use a different email address, enter it in the field Send new login details to the following email address.

Click Next.

DS_troubleshooting_rescueMode03*.png

Click Authentication via SSH key.

You have two options:

Select a key from the drop-down menu. You need to have at least one public key stored in the OVHcloud Control Panel already.
Manually copy the public key string and paste it into the field Your Public SSH key.

To find out more about this topic, consult our How to create and use keys for SSH authentication guide.

You can add a default public key for the customer rescue system to a server via the OVHcloud API. See the Go further section below for details.

Click Next.

DS_troubleshooting_rescueMode04*.png

3: Final steps to activate rescue mode

In the Summary step, click Confirm.

DS_troubleshooting_rescueMode05**.png

You should now have a notification regarding the Netboot setting in the General information tab.

The final step is to restart the server. Click the more options ... button next to "Status" in the Service status box, then select Restart. Click Confirm in the pop-up window.

This "hard reboot" will take a few minutes to complete. You can check the current status in the tab named Tasks.

After you have finished your actions in rescue mode, remember to change the Netboot setting back to Boot from the hard disk before restarting the server.

Accessing your server in rescue mode via SSH

Once you have received the email informing you that rescue mode is enabled, you can log on to the rescue mode system and access your server.

Your SSH client will normally block the connection at first due to a mismatch of the ECDSA fingerprint. This is normal because the rescue mode uses its own temporary SSH server. To resolve this, you need to edit the file known_hosts of your local .ssh folder.
You have two options:

Delete the fingerprint from the file. Your SSH client will then add a new fingerprint entry for the server when you are no longer using rescue mode. For a detailed explanation, refer to our SSH introduction guide.
Temporarily disable the fingerprint. Open the file known_hosts with a text editor and identify the fingerprint string of your server by its IP address. Add the character # at the start of the line. As a result, this line is now a "comment" and will be ignored by applications reading the file. Remember to revert this change before switching the Netboot back to the "normal" mode.

Click on the relevant tab for your selected connection method:

Open the command line application on your local device and enter the following command:

ssh root@SERVER_IP

Example:

ssh root@203.0.113.100

Enter the temporary rescue mode password when prompted.

root@ns9356771.ip-203-0-113.us's password:
root@rescue-customer-us (ns9356771.ip-203-0-113.us) ~ #

Find more information about SSH connections in our SSH introduction guide.

Open the command line application on your local device and enter the following command:

ssh -i USER_FOLDER/.ssh/KEY_FILE_NAME root@SERVER_IP

Example:

ssh -i ~/.ssh/MyAuthKey root@203.0.113.100

If prompted, enter your password to decrypt the private key file.

To learn more about this topic, consult our How to create and use keys for SSH authentication guide.

Mounting partitions to access your files

Unless you intend to configure the server's disks in a way that requires them to be detached (unmounted), you need to first mount the system partition in order to access your data from rescue mode.

Firstly, list all partitions to retrieve the name of the partition you need to mount:

lsblk

Output examples:

NAME      MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda         8:0    0  1.8T  0 disk
├─sda1      8:1    0  511M  0 part
├─sda2      8:2    0  1.8T  0 part
│ └─md127   9:127  0  1.8T  0 raid1
├─sda3      8:3    0  512M  0 part
└─sda4      8:4    0    2M  0 part
sdb         8:16   0  1.8T  0 disk
├─sdb1      8:17   0  511M  0 part
├─sdb2      8:18   0  1.8T  0 part
│ └─md127   9:127  0  1.8T  0 raid1
└─sdb3      8:19   0  512M  0 part

NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
nvme1n1     259:0    0 894.3G  0 disk
├─nvme1n1p1 259:2    0   511M  0 part
├─nvme1n1p2 259:3    0     1G  0 part
│ └─md2       9:2    0  1022M  0 raid1
├─nvme1n1p3 259:4    0 892.3G  0 part
│ └─md3       9:3    0 892.1G  0 raid1
└─nvme1n1p4 259:5    0   512M  0 part
nvme0n1     259:1    0 894.3G  0 disk
├─nvme0n1p1 259:6    0   511M  0 part
├─nvme0n1p2 259:7    0     1G  0 part
│ └─md2       9:2    0  1022M  0 raid1
├─nvme0n1p3 259:8    0 892.3G  0 part
│ └─md3       9:3    0 892.1G  0 raid1
├─nvme0n1p4 259:9    0   512M  0 part
└─nvme0n1p5 259:10   0     2M  0 part

Then mount the pertinent partition accordingly:

mount /dev/PARTITION_NAME /MOUNT_POINT/

The partition to mount should be easily identifiable by the SIZE indicated in the table (sda2 in the first example, nvme1n1p3 in the second). Using the folder name mnt as mount point, for the first example the mount command would therefore be as follows:

mount /dev/md127 /mnt/

Command to enter for the second example:

mount /dev/md3 /mnt/

NOTE: The examples above illustrate the necessary steps based on a typical server configuration. The information in the output table depends on your server's hardware and its partition scheme. When in doubt, consult the documentation of your operating system.

If you require professional assistance with server administration, consider the details in the Go further section of this guide.

To see more technical information about the server's disks and partitions, enter:

fdisk -l

Some tasks may require that disks or partitions be detached. To do this, use the unmount command:

umount /mnt

NOTE: For password reset instructions, see the guides below:

VMware - Mounting a datastore

You can mount a VMware datastore in a similar way as described in the previous segment.

Click here for the full instructions.

List your partitions in order to retrieve the name of the datastore partition:

lsblk

fdisk -l

Mount the partition with the following command, replacing sdbX with the value identified in the previous step:

vmfs-fuse /dev/sdbX /mnt

If you have VMFS 6 datastores, access the sbin folder and create the mount folder:

cd /usr/local/sbin/
mkdir /mnt/datastore

List your partitions to retrieve the name of the datastore partition:

lsblk

fdisk -l

Mount the partition with the following command, replacing sdbX with the value identified in the previous step:

vmfs6-fuse /dev/sdbX /mnt/datastore/

Once the mounting operation is done, you can access your files and carry out troubleshooting tasks inside the folder you have defined as mount point. Example:

cd /mnt

Certain operations on the file system (such as configuring user accounts) will require an additional step. Create a temporary chroot environment at the mount point with this command:

chroot /mnt/

You should now be able now apply all necessary changes to your system.

Link Aggregation configuration in rescue mode

Link Aggregation (LACP) is highly beneficial, as it increases your server's total bandwidth while providing network redundancy in case a network interface fails.

Although rescue mode is based on the Debian 12 operating system, its network configuration relies on the ifupdown utility, instead of Netplan.

If you have a server that supports link aggregation and you wish to configure it in rescue mode, please refer to this guide.

Exiting rescue mode

If relevant, return to the rescue mode login shell by entering:

exit

In your OVHcloud Control Panel, change the boot mode back to Boot from the hard disk and confirm.

DS_troubleshooting_rescueMode08*.png

You can now restart the server from the rescue mode shell:

reboot

Alternatively, use the Restart function in the OVHcloud Control Panel.

How to add a default authentication key for rescue mode

To speed up the process, you can add a public key as default for rescue mode SSH access to your server. This is only possible via the OVHcloud API.

Click here for the full instructions.

To do this in the API web console, open the following API endpoint:

PUT /dedicated/server/{serviceName}

Enter the internal name of your server (ns1111111.ip-203-0-113.us) in the appropriate field.

Then edit the text field below as follows:

{
  "rescueSshKey": "string"
}

Replace string with your full public key string.

The result should look as shown in the following example:

DS_troubleshooting_rescueMode09*.png

When you have entered your values correctly, click the button EXECUTE.

The field Your Public SSH key: will now be filled automatically with this key string when changing the Netboot mode.

Legacy Windows rescue system (WinPE rescue mode)

Once you have received the email informing you that rescue mode is enabled, you can log on to the rescue mode system and access your server.

Click here for the full instructions.

To use the Windows PE rescue mode GUI, you will need to download and install a VNC console or use the IPMI module (not available on all server models).

The following tools are already installed in this mode:

Tool	Description
Mozilla ULight	A web browser.
Memory Diagnostics Tool	A Windows tool to test the RAM.
Explorer_Q-Dir	An alternative file explorer.
GSmartControl	A tool to check HDDs and SSDs.
PhotoRec	A tool to recover possibly lost files from a disk.
SilverSHielD	A SSH2 and SFTP server.
System Recovery	The built-in Windows system restore and troubleshooting tool.
TestDisk	A powerful data recovery application. You can use it to recover and modify corrupted partitions, find lost partitions, repair a boot sector and even rebuild a defective MBR.
FileZilla	An open-source FTP client. It supports SSH and SSL protocols, and has a clear and intuitive drag-and-drop interface. You can use it to transfer your data to an FTP server, like the FTP backup service included with most OVHcloud server models.
7-Zip	A utility for compressing and archiving files, which reads the following formats: ARJ, CAB, CHM, CPIO, CramFS, DEB, DMG, FAT, HFS, ISO, LZH, LZMA, MBR, MSI, NSIS, NTFS, RAR, RPM, SquashFS, UDF, VHD, WIM, XAR and Z. You can also use it to create your own archives in the following formats: BZIP2, GZIP, TAR, WIM, XZ, Z and ZIP.

How to test Additional IP and vRack functionality in rescue mode

Testing Additional IPs

Configured as an alias (without vMAC)

To begin, put your server into rescue mode as described above. Once you are in rescue mode, use the following command to identify your NIC.

ip a

To find the correct NIC in your output, find the NIC that has your server's public IP address attached. In this article, we use the example interface eth0. We will add our additional IP to the interface using the following command. Be sure to replace our example values with yours.

ip addr add 203.0.113.1/32 dev eth0

To test that your additional IP is functioning properly, ping it from a separate machine. If you are concerned that you are having routing issues, you can perform an mtr (My traceroute) test. In our example, we use an OpenDNS server. Be sure to replace our example values with yours.

mtr -rbwz 208.67.222.222 -a 203.0.113.1

If you determine that you have routing issues, send the outputs of the following tests to OVHcloud US Support.

ip a
ip r
mtr -rbwz 208.67.222.222 -a 203.0.113.1

Bridge configuration (with vMAC)

For a bridge configuration, the above steps will work with one modification. To add our additional IP to the interface, we will use the following four commands instead of the ip addr add command used above.

In our example, we name our test link "testrescue." Be sure to replace our example values with yours. If you need help finding your Virtual MAC, check out the following article: How to Assign a Virtual MAC to an Additional IP

ip link add name testrescue link eth0 type macvlan
ip link set dev testrescue address 00:53:00:00:00:00
ip link set testrescue up
ip addr add 203.0.113.1 dev testrescue

Testing vRack

NOTE: Before testing your vRack connection, make sure both of the servers you use are on the same vRack.

To begin testing that your vRack connection works, put two servers into rescue mode.

Once you have gained access to rescue mode, we will configure both servers. We will use the same configuration on both servers, but with different private IP addresses. First, find your private interface using the following command:

ip a

Next, we will set our network configuration. To do so, open the interface file using the following command.

vim /etc/network/interfaces

Below is a sample configuration for this file. Make sure the private IPs you use on both servers are on the same subnet.

interfaces(5) file used by ifup(8) and ifdown(8)
Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d
 
auto eth1
iface eth1  inet static
address 192.168.70.5
netmask 255.255.255.0
gateway 192.168.70.254

Once you have configured both servers, ping one server from the other. If it works, you know you have a working vRack. If it does not, contact OVHcloud US support and send them the configuration files and results of your failed ping test.