Configure MSR for a cloud storage provider (S3)

You can configure MSR to store Docker images on Amazon S3 or on any other file servers with an S3-compatible API.

All S3-compatible services store files in “buckets”, to which you can authorize users to read, write, and delete files. Whenever you integrate MSR with such a service, MSR sends all read and write operations to the S3 bucket where the images then persist.

Note

The instructions offered below pertain specifically to the configuration of MSR to Amazon S3. They can, however, also serve as a guide for how to configure MSR to other available cloud storage providers.

Create a bucket on Amazon S3

Before you configure MSR you must first create a bucket on Amazon S3. To optimize pulls and pushes, Mirantis suggests that you create the S3 bucket in the AWS region that is physically closest to the servers on which MSR is set to run.

1. Create an S3 bucket.

2. Create a new IAM user for the MSR integration.

3. Apply an IAM policy that has the following limited user permissions:

   • Access to the newly-created bucket

   • Ability to read, write, and delete files

   Example user policy

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": "s3:ListAllMyBuckets",
               "Resource": "arn:aws:s3:::*"
           },
           {
               "Effect": "Allow",
               "Action": [
                   "s3:ListBucket",
                   "s3:GetBucketLocation",
                   "s3:ListBucketMultipartUploads"
               ],
               "Resource": "arn:aws:s3:::<bucket-name>"
           },
           {
               "Effect": "Allow",
               "Action": [
                   "s3:PutObject",
                   "s3:GetObject",
                   "s3:DeleteObject",
                   "s3:ListBucketMultipartUploads"
               ],
               "Resource": "arn:aws:s3:::<bucket-name>/*"
           }
       ]
   }
   

Configure MSR on Amazon S3

Kubernetes deployments

MSR Operator

1. Add the following values to the custom resource manifest. If you are using IAM role authentication, do not include the lines for accesskey and secretkey. Running Kubernetes on AWS requires that you include v4auth: true.

   spec:
     registry:
       storage:
         backend: "s3"
         s3:
           region: <region>
           bucket: <bucket-name>
           accesskey: <access-key>
           secretkey: <secret-key>
           v4auth: true
         persistentVolume:
           size: <size>
   

2. Apply the changes to the custom resource:

   kubectl apply -f cr-sample-manifest.yaml
   

3. Verify completion of the reconciliation process for the custom resource:

   kubectl get msrs.msr.mirantis.com
   kubectl get rethinkdbs.rethinkdb.com
   

Helm chart

1. Set registry.storage.backend to s3.

2. Specify registry.storage.s3.region and registry.storage.s3.bucket.

3. If you are not using IAM role authentication, you must also set registry.storage.s3.accesskey and registry.storage.s3.secretkey.

4. To activate the new storage configuration settings, issue the helm upgrade command.

Example configuration command at install time:

helm install msr msrofficial/msr \
--version 1.0.0 \
--set registry.storage.backend=s3 \
--set registry.storage.s3.accesskey=<> \
--set registry.storage.s3.secretkey=<> \
--set registry.storage.s3.region=us-east-2 \
--set registry.storage.s3.bucket=testing-msr-kube

Example configuration command at time of upgrade:

helm upgrade msr msrofficial/msr \
--version 1.0.0 \
--set registry.storage.backend=s3 \
--set registry.storage.s3.accesskey=<> \
--set registry.storage.s3.secretkey=<> \
--set registry.storage.s3.region=us-east-2 \
--set registry.storage.s3.bucket=testing-msr-kube

Swarm deployments

1. Update your values.yaml file to include the following values.

   Note

   If you are using IAM role authentication, do not include the lines that set the accesskey and secretkey values.

   registry:
     storage:
       backend: 's3'
       s3:
         region: <region>
         bucket: <bucket-name>
         accesskey: <access-key>
         secretkey: <secret-key>
   

2. Install or upgrade your deployment, as needed.

The following parameters are available for configuration in the registry.storage.s3 section of the custom resource manifest, Helm chart, or Swarm cluster values.yaml file:

Amazon S3

Field	Description	Level
accesskey	AWS Access Key.	Standard
secretkey	AWS Secret key.	Standard
region	The AWS region in which your bucket exists.	Standard
regionendpoint	The endpoint for S3 compatible storage services.	Standard
bucket	The name of the bucket in which image data is stored.	Standard
encrypt	Indicates whether images are stored in encrypted format.	Advanced
keyid	The KMS key ID to use for encryption of images.	Advanced
secure	Indicates whether to use HTTPS for data transfers to the bucket.	Advanced
v4auth	Indicates whether to use AWS Signature Version 4 to authenticate requests.	Advanced
chunksize	The default part size for multipart uploads.	Advanced
rootdirectory	A prefix that is applied to all object keys to allow you to segment data in your bucket if necessary.	Advanced
storageclass	The S3 storage class applied to each registry file. Valid options are “STANDARD” and “REDUCED_REDUNDANCY”.	Advanced

MSR supports the following S3 regions:

us-east-1	us-east-2
us-west-1	us-west-2
eu-west-1	eu-west-2
eu-central-1	ap-south-1
ap-southeast-1	ap-southeast-2
ap-northeast-1	ap-northeast-2
sa-east-1	cn-north-1
us-gov-west-1	ca-central-1

Restore MSR with your previous settings

Restore MSR with S3 settings

To restore MSR using your previously configured S3 settings, use restore.

Restore MSR with non-S3 cloud storage provider settings

For S3-compatible cloud storage providers other than Amazon S3, configure the following parameters in the registry.storage section of the custom resource manifest, Helm chart, or Swarm cluster values.yaml file:

Microsoft Azure

Field	Description	Level
accountname	The name of the Azure Storage Account.	Standard
accountkey	The Primary or Secondary Key for the Storage Account.	Standard
container	The name of the Azure root storage container in which image data is stored.	Standard
realm	The domain name suffix for the Storage API endpoint.	Advanced

OpenStack Swift

Field	Description	Level
authurl	OpenStack user name.	Standard
username	OpenStack user name.	Standard
password	OpenStack password.	Standard
container	The name of the Swift container in which to store the registry images.	Standard
region	The contents of a service account private key file in JSON format that is used for Service Account Authentication.	Advanced
tenant	OpenStack tenant name.	Advanced
tenantid	OpenStack tenant ID.	Advanced
domain	OpenStack domain name for Identity v3 API.	Advanced
domainid	OpenStack domain id for Identity v3 API.	Advanced
trustid	OpenStack trust ID for Identity v3 API.	Advanced
insecureskipverify	Skips TLS server certificate verification.	Advanced
chunksize	Data segments for the Swift Dynamic Large Objects.	Advanced
prefix	A prefix that is applied to all Swift object keys that allows you to segment data in your container, if necessary.	Advanced
secretkey	The secret key used to generate temporary URLs.	Advanced
accesskey	The access key to generate temporary URLs.	Advanced
authversion	Specifies the OpenStack Auth version.	Advanced
endpointtype	The endpoint type used when connecting to Swift.	Advanced

Google Cloud Storage

Field	Description	Level
bucket	The name of the Google Cloud Storage bucket in which image data is stored.	Standard
credentials	The contents of a service account private key file in JSON format that is used for Service Account Authentication.	Advanced
rootdirectory	The root directory tree in which all registry files are stored. The prefix is applied to all Google Cloud Storage keys, to allow you to segment data in your bucket as necessary.	Advanced
chunksize	The chunk size used for uploading large blobs.	Advanced

Alibaba Cloud Object Storage Service

Field	Description	Level
accesskeyid	Access key ID.	Standard
accesskeysecret	Access key secret.	Standard
region	The ID of the OSS region in which you would like to store objects.	Standard
bucket	The name of the OSS bucket in which to store objects.	Standard
endpoint	The endpoint domain name for accessing OSS.	Advanced
internal	Indicates whether to use the internal endpoint instead of the public endpoint, for OSS access.	Advanced
encrypt	Indicates whether to encrypt your data on the server side.	Advanced
secure	Indicates whether to transfer data to the bucket over HTTPS.	Advanced
chunksize	The default part size for multipart uploads.	Advanced
rootdirectory	A prefix that is applied to all object keys that allows you to segment data in your bucket, if necessary.	Advanced