Skip to main content

Getting Started with Object Storage

Stephanie R.
Updated

Learn about managing containers and objects.

 

Requirements

 

Instructions

Using AWS CLI

Installation

Enter the following command depending on the method chosen:

user@host:~$ pip3 install awscli
user@host:~$ sudo apt install awscli
user@host:~$ sudo yum install awscli

Install the groff package if you want to use command line help.

 

Collect Credentials

  • To access your Object Storage resources, you will need your user's Access key and Secret key. You can retrieve this information from the OVHcloud Control Panel under Public Cloud > Object Storage > Object Storage users.
  • You will also need your url_endpoint and the region. If you have already created your container, you can access this information from the My containers tab in the details of your container. Otherwise, follow this guide.

 

Configuration

To use the interactive configuration, run the following command: aws --configure

The configuration file format in the aws client is as follows:

user@host:~$ cat ~/.aws/credentials

[default]
aws_access_key_id = <access_key>
aws_secret_access_key = <secret_key>
user@host:~$ cat ~/.aws/config

[profile default]
region = <region_in_lowercase>
endpoint_url = <url_endpoint>
s3 = 
  signature_version = s3v4

Here are the configuration values that you can specifically set:

Variable Type Value Definition
max_competitor_requests Integer Default: 10 The maximum number of simultaneous requests.
max_queue_size Integer Default: 1000 The maximum number of tasks in the task queue.
multipart_threshold Integer
String		 Default: 8MB The size threshold that the CLI uses for multipart transfers of individual files.
multipart_chunksize Integer
String		 Default: 8MB
Minimum for uploads: 5MB		 When using multipart transfers, this is the bit size that the CLI uses for multipart transfers of individual files.
max_bandwidth Integer Default: None The maximum bandwidth that will be used to load and download data to and from your buckets.
verify_ssl Boolean Default: true Enable / Disable SSL certificate verification

For a list of endpoints by region and storage class, refer to this page.

 

Usage

If you have more than one profile, add --profile <profile> to the command line.

Creating a bucket

aws s3 mb s3://<bucket_name>
aws --profile default s3 mb s3://<bucket_name>

 

Listing your buckets

aws s3 ls

 

Uploading your files as objects in your bucket

aws s3 cp /datas/test1 s3://<bucket_name>
The aws s3 cp command will use STANDARD as default storage class for uploading objects. To store objects in the High Performance tier, use the aws s3api put-object command instead, as aws s3 cp does not support the EXPRESS_ONEZONE storage class which is used to map the High Performance storage tier. To learn more about the storage class mapping between OVHcloud storage tiers and AWS storage classes, you can check our documentation here.
# upload an object to High Performance tier
aws s3api put-object --bucket <bucket_name> --key <object_name> --body /datas/test1 --storage-class EXPRESS_ONEZONE
# upload an object to Standard tier
aws s3api put-object --bucket <bucket_name> --key <object_name> --body /datas/test1 --storage-class STANDARD

 

By default, objects are named after files, but can be renamed

aws s3 cp /datas/test1 s3://<bucket_name>/other-filename

 

Downloading an object from a bucket

aws s3 cp s3://<bucket_name>/test1 .

 

Uploading an object from one bucket to another bucket

aws s3 cp s3://<bucket_name>/test1 s3://<bucket_name_2>

 

Downloading or uploading an entire bucket to the host/bucket

aws s3 cp s3://<bucket_name> . --recursive
aws s3 cp s3://<bucket_name> s3://<bucket_name_2> --recursive

 

Synchronizing buckets

aws s3 sync . s3://<bucket_name>
aws s3 sync s3://<bucket_name> s3://<bucket_name_2>

 

Deleting objects and buckets

To delete an object:

aws s3 rm s3://<bucket_name>/test1

To remove all objects from a bucket:

aws s3 rm s3://<bucket_name> --recursive

To delete a bucket, it must be empty:

aws s3 rb s3://<bucket_name>

If the bucket is not removed, you can use the same command with the --force option. This deletes all objects from the bucket, and then deletes the bucket.

aws s3 rb s3://<bucket_name> --force

 

Deleting objects and buckets with versioning enabled

If versioning is enabled, a simple delete operation on your objects will not permanently remove them.

In order to permanently delete an object, you must specify a version id:

aws s3api delete-object --bucket <NAME> --key <KEY> --version-id <VERSION_ID>

To list all objects and all version ID, you can use the following command:

aws s3api list-object-versions --bucket <NAME>

With the previous delete-object command, you will have to iterate over all your object versions. Alternatively, you can use the following one-liner to empty your bucket:

aws s3api delete-objects --bucket <NAME> --delete "$(aws s3api list-object-versions --bucket <NAME> --query='{Objects: Versions[].{Key:Key,VersionId:VersionId}}')"
If your bucket has Object Lock enabled, you will not be able to permanently delete your objects. See our documentation to learn more about Object Lock. If you use Object Lock in GOVERNANCE mode and have the permission to bypass GOVERNANCE mode, you will have to add the --bypass-governance-retention option to your delete commands.

 

Setting tags on a bucket

aws s3api put-bucket-tagging --bucket <bucket_name> --tagging 'TagSet=[{Key=myKey,Value=myKeyValue}]'
aws s3api get-bucket-tagging --bucket <bucket_name>
{
  "TagSet": [
    {
    "Value": "myKeyValue",
    "Key": "myKey"
    }
  ]
}

 

Deleting tags on a bucket

aws s3api s3api delete-bucket-tagging --bucket <bucket_name>

 

Setting tags on an object

aws s3api put-object-tagging --bucket <bucket_name> --key test1 --tagging 'TagSet=[{Key=myKey,Value=myKeyValue}]'
aws s3api get-bucket-tagging --bucket <bucket_name>
{
  "TagSet": [
    {
    "Value": "myKeyValue",
    "Key": "myKey"
    }
  ]
}

 

Deleting tags on an object

aws s3api s3api delete-object-tagging --bucket <bucket_name> --key test1

 

Go further

For more information and tutorials, please see our other Object Storage support guides or explore the guides for other OVHcloud products and services.

*S3 is a trademark filed by Amazon Technologies, Inc. OVHcloud's service is not sponsored by, endorsed by, or otherwise affiliated with Amazon Technologies, Inc.

Related articles