Skip to main content

Use Object Storage with S3cmd

Stephanie R.
Updated

Learn how to configure S3cmd to manage your containers and objects.

S3cmd is a free command line tool and client for managing data in storage spaces that use the Amazon S3®* protocol, such as Amazon S3® Object Storage, Google Cloud Storage, or DreamHost DreamObjects.

NOTE: OVHcloud provides services for which you are responsible for their configuration and management. You are therefore responsible for their proper functioning.

This tutorial is designed to help you as much as possible with common tasks. If you are having difficulty performing these actions, please contact a specialized service provider. OVHcloud can't provide you with technical support in this regard.

 

Requirements

  • a container (see our Getting Started with Object Storage guide)
  • a user with the required access rights on the container
  • your Object Storage credentials (access_key and secret_access_key)
  • S3cmd installed
To identify the endpoint corresponding to your storage class, please refer to our Endpoints and Object Storage Geoavailability guide.

 

Instructions

To configure s3cmd, run the following command:

$ s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: `<access_key>`
Secret Key: `<secret_key>`
Default Region [US]: `<region_in_lowercase>`

Use "s3.amazonaws.com" for S3 Endpoint and do not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]: `s3.<region_in_lowercase>.io.cloud.ovh.us`

Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: `<bucket>.s3.<region_in_lowercase>.io.cloud.ovh.us`

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: `<passphrase>`      
Path to GPG program [/usr/bin/gpg]:
When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]:

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can\'t connect to S3 directly
HTTP Proxy server name:

New settings:
  Access Key: ACCESS_KEY
  Secret Key: SECRET_KEY
  Default Region: <region_in_lowercase>
  S3 Endpoint: s3.<region_in_lowercase>.io.cloud.ovh.us
  DNS-style bucket+hostname:port template for accessing a bucket: hp-bucket.s3.<region_in_lowercase>.io.cloud.ovh.us
  Encryption password: passphrase
  Path to GPG program: /usr/bin/gpg
  Use HTTPS protocol: True
  HTTP Proxy server name:
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n]
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Success. Encryption and decryption worked fine :-)

Save settings? [y/N] y
Configuration saved to '/home/user/.s3cfg'

S3cmd is now ready to be used.

 

Command examples

List all buckets:

$ s3cmd ls

Create a new bucket:

$ s3cmd mb s3://BUCKET

List the contents of a bucket:

$ s3cmd ls s3://BUCKET[/PREFIX]

Synchronise /home/user/documents to a bucket

$ s3cmd sync /home/user/documents s3://BUCKET[/PREFIX]

Copy a file /home/user/file.txt into a bucket:

$ s3cmd put FILE [FILE...] s3://BUCKET[/PREFIX]

Download a file file.txt from a bucket:

$ s3cmd get s3://BUCKET/OBJECT LOCAL_FILE

You will find a detailed documentation of the possible actions on the official S3cmd documentation.

 

Go further

For more information and tutorials, please see our other Object Storage support guides or explore the guides for other OVHcloud products and services.

If you need training or technical assistance to implement our solutions, contact your sales representative or click on this link to get a quote and ask our Professional Services experts for a custom analysis of your project.

*S3 is a trademark filed by Amazon Technologies, Inc. OVHcloud's service is not sponsored by, endorsed by, or otherwise affiliated with Amazon Technologies, Inc.

Related articles