Configure MSR for a cloud storage provider (S3)

You can configure MSR to store Docker images on Amazon S3 or on any other file servers with an S3-compatible API.

All S3-compatible services store files in “buckets”, to which you can authorize users to read, write, and delete files. Whenever you integrate MSR with such a service, MSR sends all read and write operations to the S3 bucket where the images then persist.

Note

The instructions offered below pertain specifically to the configuration of MSR to Amazon S3. They can, however, also serve as a guide for how to configure MSR to other available cloud storage providers.

Create a bucket on Amazon S3

Before you configure MSR you must first create a bucket on Amazon S3. To optimize pulls and pushes, Mirantis suggests that you create the S3 bucket in the AWS region that is physically closest to the servers on which MSR is set to run.

  1. Create an S3 bucket.

  2. Create a new IAM user for the MSR integration.

  3. Apply an IAM policy that has the following limited user permissions:

    • Access to the newly-created bucket

    • Ability to read, write, and delete files

    Example user policy
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": "s3:ListAllMyBuckets",
                "Resource": "arn:aws:s3:::*"
            },
            {
                "Effect": "Allow",
                "Action": [
                    "s3:ListBucket",
                    "s3:GetBucketLocation",
                    "s3:ListBucketMultipartUploads"
                ],
                "Resource": "arn:aws:s3:::<bucket-name>"
            },
            {
                "Effect": "Allow",
                "Action": [
                    "s3:PutObject",
                    "s3:GetObject",
                    "s3:DeleteObject",
                    "s3:ListBucketMultipartUploads"
                ],
                "Resource": "arn:aws:s3:::<bucket-name>/*"
            }
        ]
    }
    

Configure MSR on Amazon S3

Kubernetes deployments

  1. Add the following values to the custom resource manifest. If you are using IAM role authentication, do not include the lines for accesskey and secretkey. Running Kubernetes on AWS requires that you include v4auth: true.

    spec:
      registry:
        storage:
          backend: "s3"
          s3:
            region: <region>
            bucket: <bucket-name>
            accesskey: <access-key>
            secretkey: <secret-key>
            v4auth: true
          persistentVolume:
            size: <size>
    
  2. Apply the changes to the custom resource:

    kubectl apply -f cr-sample-manifest.yaml
    
  3. Verify completion of the reconciliation process for the custom resource:

    kubectl get msrs.msr.mirantis.com
    kubectl get rethinkdbs.rethinkdb.com
    
  1. Set registry.storage.backend to s3.

  2. Specify registry.storage.s3.region and registry.storage.s3.bucket.

  3. If you are not using IAM role authentication, you must also set registry.storage.s3.accesskey and registry.storage.s3.secretkey.

  4. To activate the new storage configuration settings, issue the helm upgrade command.

Example configuration command at install time:

helm install msr msrofficial/msr \
--version 1.0.0 \
--set registry.storage.backend=s3 \
--set registry.storage.s3.accesskey=<> \
--set registry.storage.s3.secretkey=<> \
--set registry.storage.s3.region=us-east-2 \
--set registry.storage.s3.bucket=testing-msr-kube

Example configuration command at time of upgrade:

helm upgrade msr msrofficial/msr \
--version 1.0.0 \
--set registry.storage.backend=s3 \
--set registry.storage.s3.accesskey=<> \
--set registry.storage.s3.secretkey=<> \
--set registry.storage.s3.region=us-east-2 \
--set registry.storage.s3.bucket=testing-msr-kube

Swarm deployments

  1. Update your values.yaml file to include the following values.

    Note

    If you are using IAM role authentication, do not include the lines that set the accesskey and secretkey values.

    registry:
      storage:
        backend: 's3'
        s3:
          region: <region>
          bucket: <bucket-name>
          accesskey: <access-key>
          secretkey: <secret-key>
    
  2. Install or upgrade your deployment, as needed.

The following parameters are available for configuration in the registry.storage.s3 section of the custom resource manifest, Helm chart, or Swarm cluster values.yaml file:

Amazon S3

Field

Description

Level

accesskey

AWS Access Key.

Standard

secretkey

AWS Secret key.

Standard

region

The AWS region in which your bucket exists.

Standard

regionendpoint

The endpoint for S3 compatible storage services.

Standard

bucket

The name of the bucket in which image data is stored.

Standard

encrypt

Indicates whether images are stored in encrypted format.

Advanced

keyid

The KMS key ID to use for encryption of images.

Advanced

secure

Indicates whether to use HTTPS for data transfers to the bucket.

Advanced

v4auth

Indicates whether to use AWS Signature Version 4 to authenticate requests.

Advanced

chunksize

The default part size for multipart uploads.

Advanced

rootdirectory

A prefix that is applied to all object keys to allow you to segment data in your bucket if necessary.

Advanced

storageclass

The S3 storage class applied to each registry file. Valid options are “STANDARD” and “REDUCED_REDUNDANCY”.

Advanced

MSR supports the following S3 regions:

us-east-1

us-east-2

us-west-1

us-west-2

eu-west-1

eu-west-2

eu-central-1

ap-south-1

ap-southeast-1

ap-southeast-2

ap-northeast-1

ap-northeast-2

sa-east-1

cn-north-1

us-gov-west-1

ca-central-1

Restore MSR with your previous settings

Restore MSR with S3 settings

To restore MSR using your previously configured S3 settings, use restore.

Restore MSR with non-S3 cloud storage provider settings

For S3-compatible cloud storage providers other than Amazon S3, configure the following parameters in the registry.storage section of the custom resource manifest, Helm chart, or Swarm cluster values.yaml file:

Microsoft Azure

Field

Description

Level

accountname

The name of the Azure Storage Account.

Standard

accountkey

The Primary or Secondary Key for the Storage Account.

Standard

container

The name of the Azure root storage container in which image data is stored.

Standard

realm

The domain name suffix for the Storage API endpoint.

Advanced

OpenStack Swift

Field

Description

Level

authurl

OpenStack user name.

Standard

username

OpenStack user name.

Standard

password

OpenStack password.

Standard

container

The name of the Swift container in which to store the registry images.

Standard

region

The contents of a service account private key file in JSON format that is used for Service Account Authentication.

Advanced

tenant

OpenStack tenant name.

Advanced

tenantid

OpenStack tenant ID.

Advanced

domain

OpenStack domain name for Identity v3 API.

Advanced

domainid

OpenStack domain id for Identity v3 API.

Advanced

trustid

OpenStack trust ID for Identity v3 API.

Advanced

insecureskipverify

Skips TLS server certificate verification.

Advanced

chunksize

Data segments for the Swift Dynamic Large Objects.

Advanced

prefix

A prefix that is applied to all Swift object keys that allows you to segment data in your container, if necessary.

Advanced

secretkey

The secret key used to generate temporary URLs.

Advanced

accesskey

The access key to generate temporary URLs.

Advanced

authversion

Specifies the OpenStack Auth version.

Advanced

endpointtype

The endpoint type used when connecting to Swift.

Advanced

Google Cloud Storage

Field

Description

Level

bucket

The name of the Google Cloud Storage bucket in which image data is stored.

Standard

credentials

The contents of a service account private key file in JSON format that is used for Service Account Authentication.

Advanced

rootdirectory

The root directory tree in which all registry files are stored. The prefix is applied to all Google Cloud Storage keys, to allow you to segment data in your bucket as necessary.

Advanced

chunksize

The chunk size used for uploading large blobs.

Advanced

Alibaba Cloud Object Storage Service

Field

Description

Level

accesskeyid

Access key ID.

Standard

accesskeysecret

Access key secret.

Standard

region

The ID of the OSS region in which you would like to store objects.

Standard

bucket

The name of the OSS bucket in which to store objects.

Standard

endpoint

The endpoint domain name for accessing OSS.

Advanced

internal

Indicates whether to use the internal endpoint instead of the public endpoint, for OSS access.

Advanced

encrypt

Indicates whether to encrypt your data on the server side.

Advanced

secure

Indicates whether to transfer data to the bucket over HTTPS.

Advanced

chunksize

The default part size for multipart uploads.

Advanced

rootdirectory

A prefix that is applied to all object keys that allows you to segment data in your bucket, if necessary.

Advanced