How to activate and use rescue mode – Support Guides

Learn how to reboot and use a server in rescue mode.

Rescue mode is a tool provided by OVHcloud that allows you to boot into a temporary operating system to diagnose and resolve issues on your server.

Usual tasks the rescue mode is appropriate for include:

Resetting your user password
Diagnosing network problems
Repairing a broken operating system
Fixing a software firewall misconfiguration
Testing disk performance
Testing CPU and RAM

Requirements

A dedicated server in your OVHcloud account
Access to the OVHcloud Control Panel

Instructions

To use rescue mode, you must edit the server's Netboot setting. Then the server has to be restarted.

Activating rescue mode

Log in to your OVHcloud Control Panel, open the Bare Metal Cloud section, then Dedicated servers, and select your server from the list.

In the General information box, click the more options ... button next Boot and select Edit.

DS_troubleshooting_rescueMode01*.png

1: Rescue mode options

On the page Change the netboot, select Boot in rescue mode.

The available options for rescue mode depend on the server type and the operating system installed.

Customer rescue system (always available)
Rescue system for Windows (available for Windows servers)
iPXE / ipxe-shell (external open-source tool, always available)
Legacy Windows rescue system (deprecated WinPE system, only relevant if your server does not meet the requirements for the current rescue system for Windows)

The instructions below will only cover the customer rescue system, the most commonly used option.

Select Customer rescue system in the drop-down menu.

DS_troubleshooting_rescueMode02*.png

2: Authentication options

The next choice determines the method of authentication for the SSH connection to the rescue mode system. This is mainly a matter of convenience since each rescue mode session is meant to be transitory and will be discarded once you reboot the server from its disk.

Password authentication: You will be sent login credentials via email.
Key authentication: You can use a public authentication key of your choice (compatible formats: RSA, ECDSA, ED25519).

Click on the relevant tab for your connection method:

Click Password authentication.

The rescue mode notification email including its login details will be sent to the contact email address of your OVHcloud account. To use a different email address, enter it in the field Send new login details to the following email address.

Click Next.

DS_troubleshooting_rescueMode03*.png

Click Authentication via SSH key.

You have two options:

Select a key from the drop-down menu. You need to have at least one public key stored in the OVHcloud Control Panel already.
Manually copy the public key string and paste it into the field Your Public SSH key.

To find out more about this topic, consult our How to create and use keys for SSH authentication guide.

You can add a default public key for the customer rescue system to a server via the OVHcloud API. See the Go further section below for details.

Click Next.

DS_troubleshooting_rescueMode04*.png

3: Final steps to activate rescue mode

In the Summary step, click Confirm.

DS_troubleshooting_rescueMode05**.png

You should now have a notification regarding the Netboot setting in the General information tab.

The final step is to restart the server. Click the more options ... button next to "Status" in the Service status box, then select Restart. Click Confirm in the pop-up window.

This "hard reboot" will take a few minutes to complete. You can check the current status in the tab named Tasks.

After you have finished your actions in rescue mode, remember to change the Netboot setting back to Boot from the hard disk before restarting the server.

Accessing your server in rescue mode via SSH

Once you have received the email informing you that rescue mode is enabled, you can log on to the rescue mode system and access your server.

Your SSH client will normally block the connection at first due to a mismatch of the ECDSA fingerprint. This is normal because the rescue mode uses its own temporary SSH server. To resolve this, you need to edit the file known_hosts of your local .ssh folder.
You have two options:

Delete the fingerprint from the file. Your SSH client will then add a new fingerprint entry for the server when you are no longer using rescue mode. For a detailed explanation, refer to our SSH introduction guide.
Temporarily disable the fingerprint. Open the file known_hosts with a text editor and identify the fingerprint string of your server by its IP address. Add the character # at the start of the line. As a result, this line is now a "comment" and will be ignored by applications reading the file. Remember to revert this change before switching the Netboot back to the "normal" mode.

Click on the relevant tab for your selected connection method:

Open the command line application on your local device and enter the following command:

ssh root@SERVER_IP

Example:

ssh root@203.0.113.100

Enter the temporary rescue mode password when prompted.

root@ns9356771.ip-203-0-113.us's password:
root@rescue-customer-us (ns9356771.ip-203-0-113.us) ~ #

Find more information about SSH connections in our SSH introduction guide.

Open the command line application on your local device and enter the following command:

ssh -i USER_FOLDER/.ssh/KEY_FILE_NAME root@SERVER_IP

Example:

ssh -i ~/.ssh/MyAuthKey root@203.0.113.100

If prompted, enter your password to decrypt the private key file.

To learn more about this topic, consult our How to create and use keys for SSH authentication guide.

Mounting partitions to access your files

Unless you intend to configure the server's disks in a way that requires them to be detached (unmounted), you need to first mount the system partition in order to access your data from rescue mode.

Firstly, list all partitions to retrieve the name of the partition you need to mount:

lsblk

Output examples:

NAME      MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda         8:0    0  1.8T  0 disk
├─sda1      8:1    0  511M  0 part
├─sda2      8:2    0  1.8T  0 part
│ └─md127   9:127  0  1.8T  0 raid1
├─sda3      8:3    0  512M  0 part
└─sda4      8:4    0    2M  0 part
sdb         8:16   0  1.8T  0 disk
├─sdb1      8:17   0  511M  0 part
├─sdb2      8:18   0  1.8T  0 part
│ └─md127   9:127  0  1.8T  0 raid1
└─sdb3      8:19   0  512M  0 part

NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
nvme1n1     259:0    0 894.3G  0 disk
├─nvme1n1p1 259:2    0   511M  0 part
├─nvme1n1p2 259:3    0     1G  0 part
│ └─md2       9:2    0  1022M  0 raid1
├─nvme1n1p3 259:4    0 892.3G  0 part
│ └─md3       9:3    0 892.1G  0 raid1
└─nvme1n1p4 259:5    0   512M  0 part
nvme0n1     259:1    0 894.3G  0 disk
├─nvme0n1p1 259:6    0   511M  0 part
├─nvme0n1p2 259:7    0     1G  0 part
│ └─md2       9:2    0  1022M  0 raid1
├─nvme0n1p3 259:8    0 892.3G  0 part
│ └─md3       9:3    0 892.1G  0 raid1
├─nvme0n1p4 259:9    0   512M  0 part
└─nvme0n1p5 259:10   0     2M  0 part

Then mount the pertinent partition accordingly:

mount /dev/PARTITION_NAME /MOUNT_POINT/

The partition to mount should be easily identifiable by the SIZE indicated in the table (sda2 in the first example, nvme1n1p3 in the second). Using the folder name mnt as mount point, for the first example the mount command would therefore be as follows:

mount /dev/md127 /mnt/

Command to enter for the second example:

mount /dev/md3 /mnt/

NOTE: The examples above illustrate the necessary steps based on a typical server configuration. The information in the output table depends on your server's hardware and its partition scheme. When in doubt, consult the documentation of your operating system.

If you require professional assistance with server administration, consider the details in the Go further section of this guide.

To see more technical information about the server's disks and partitions, enter:

fdisk -l

Some tasks may require that disks or partitions be detached. To do this, use the unmount command:

umount /mnt

NOTE: For password reset instructions, see the guides below:

VMware - Mounting a datastore

You can mount a VMware datastore in a similar way as described in the previous segment.

Click here for the full instructions.

List your partitions in order to retrieve the name of the datastore partition:

lsblk

fdisk -l

Mount the partition with the following command, replacing sdbX with the value identified in the previous step:

vmfs-fuse /dev/sdbX /mnt

If you have VMFS 6 datastores, access the sbin folder and create the mount folder:

cd /usr/local/sbin/
mkdir /mnt/datastore

List your partitions to retrieve the name of the datastore partition:

lsblk

fdisk -l

Mount the partition with the following command, replacing sdbX with the value identified in the previous step:

vmfs6-fuse /dev/sdbX /mnt/datastore/

Once the mounting operation is done, you can access your files and carry out troubleshooting tasks inside the folder you have defined as mount point. Example:

cd /mnt

Certain operations on the file system (such as configuring user accounts) will require an additional step. Create a temporary chroot environment at the mount point with this command:

chroot /mnt/

You should now be able now apply all necessary changes to your system.

Exiting rescue mode

If relevant, return to the rescue mode login shell by entering:

exit

In your OVHcloud Control Panel, change the boot mode back to Boot from the hard disk and confirm.

DS_troubleshooting_rescueMode08*.png

You can now restart the server from the rescue mode shell:

reboot

Alternatively, use the Restart function in the OVHcloud Control Panel.

How to add a default authentication key for rescue mode

To speed up the process, you can add a public key as default for rescue mode SSH access to your server. This is only possible via the OVHcloud API.

Click here for the full instructions.

To do this in the API web console, open the following API endpoint:

PUT /dedicated/server/{serviceName}

Enter the internal name of your server (ns1111111.ip-203-0-113.us) in the appropriate field.

Then edit the text field below as follows:

{
  "rescueSshKey": "string"
}

Replace string with your full public key string.

The result should look as shown in the following example:

DS_troubleshooting_rescueMode09*.png

When you have entered your values correctly, click the button EXECUTE.

The field Your Public SSH key: will now be filled automatically with this key string when changing the Netboot mode.

Legacy Windows rescue system (WinPE rescue mode)

Once you have received the email informing you that rescue mode is enabled, you can log on to the rescue mode system and access your server.

Click here for the full instructions.

To use the Windows PE rescue mode GUI, you will need to download and install a VNC console or use the IPMI module (not available on all server models).

The following tools are already installed in this mode:

Tool	Description
Mozilla ULight	A web browser.
Memory Diagnostics Tool	A Windows tool to test the RAM.
Explorer_Q-Dir	An alternative file explorer.
GSmartControl	A tool to check HDDs and SSDs.
PhotoRec	A tool to recover possibly lost files from a disk.
SilverSHielD	A SSH2 and SFTP server.
System Recovery	The built-in Windows system restore and troubleshooting tool.
TestDisk	A powerful data recovery application. You can use it to recover and modify corrupted partitions, find lost partitions, repair a boot sector and even rebuild a defective MBR.
FileZilla	An open-source FTP client. It supports SSH and SSL protocols, and has a clear and intuitive drag-and-drop interface. You can use it to transfer your data to an FTP server, like the FTP backup service included with most OVHcloud server models.
7-Zip	A utility for compressing and archiving files, which reads the following formats: ARJ, CAB, CHM, CPIO, CramFS, DEB, DMG, FAT, HFS, ISO, LZH, LZMA, MBR, MSI, NSIS, NTFS, RAR, RPM, SquashFS, UDF, VHD, WIM, XAR and Z. You can also use it to create your own archives in the following formats: BZIP2, GZIP, TAR, WIM, XZ, Z and ZIP.