Configuring Replication

Introduction to Replication

  • Purpose of Replication: Replication is a critical feature that allows the synchronization of container images across multiple registry instances. It is often employed for:

    • Disaster Recovery: Creating replicas in geographically distant locations provides redundancy and ensures accessibility during outages.

    • Load Balancing: Distributing image pull requests across several registries improves performance and reduces latency.

    • Collaborative Environments: In complex deployment scenarios, replication enables teams across locations to access synchronized image repositories.

  • Key Concepts:

    • Replication Endpoint: An endpoint defines the registry location MSR will replicate images to or from. This includes both internal and external registries.

    • Replication Rule: Rules specify which images to replicate, with filters based on namespace, tags, or patterns. This rule framework ensures only relevant data is synchronized, saving time and storage space.

    • Triggers: Triggers determine the timing and conditions under which replication occurs. Common triggers include manual, immediate replication, or scheduled replications.

Configuring Replication Endpoints

We start by creating a Replication Endpoint in the MSR4 UI

  1. Log into the MSR4 Web Interface: Use your admin credentials to access the MSR4 web interface.

  2. Navigate to Registries:

    • From the main menu, select Administration > Registries.

    • Here, you will manage all endpoints that your MSR4 instance connects to for replication purposes

  3. Creating a New Endpoint:

    • Click + New Endpoint to start setting up an endpoint.

    • Select Provider Type

      • Choose from options like MSR, Docker Registry, Harbor, or AWS ECR, each with unique requirements.

    • Endpoint Name: Enter a name that clearly describes the endpoint’s function (e.g., “US-West Registry” or “Production Backup”). You can add additional information in the Description field.

    • Endpoint URL: Input the full URL of the target registry (e.g., https://example-registry.com).

    • Access ID: Is the username for the remote registry

    • Access Secret: Is the password for the account to access the remote registry.

    • Verify Connection:

      • Click Test Connection to ensure MSR4 can reach the endpoint successfully. A success message confirms network connectivity and credential accuracy.

  4. Save Endpoint Configuration:

    • After successful testing, click Save to finalize the endpoint configuration.

Considerations: Always verify that the registry URL and credentials are current and correct. Expired tokens or incorrect URLs can interrupt replication jobs and require troubleshooting.

Creating Replication Rules

Replication rules define the replication’s scope, ensuring that only necessary images are synchronized. This approach conserves bandwidth and maintains efficient storage use.

Setting Up a New Replication Rule in MSR4

  1. Access the Replication Rules Panel:

    • In the MSR4 web interface, go to Administration > Replications.

    • The Replications page displays all existing rules and allows you to add new rules or modify existing ones.

  2. Define a New Rule:

    • Click + New Replication Rule to open the rule configuration screen.

    • Name: Assign a unique name (e.g., “Sync to Europe Backup”) that indicates the rule’s purpose.

    • Replication Mode: Select Push to send data to the remote location, or pull to copy data from the remote location.

    • Source Resource Filter: This is where you can filter a subset of images by name, tag, label, or resource type.

      • Namespace: Sync only images within specific namespaces.

      • Tag Patterns: Define tag patterns to limit replication to specific versions or releases (e.g., *latest).

      • Label: Replicate images tagged with specific labels.

      • If you set name to ** you will download all images. .

    • Destination Registry: Select from the list of previously configured endpoints.

    • Name Space & Flattening: When you mirror MSR4 Harbor has the ability to flatten the name space.

    • Configure the Trigger Mode:: Specify how and when the replication should occur:

      • Manual: Requires an admin to start replication manually

      • Immediate: Begins replication as soon as an image is pushed to the source registry.

      • Scheduled: Allows you to define a CRON-based schedule (e.g., daily at midnight).

    • Save and Activate the Rule:

      • Once configured, click Create to save and activate the rule.

Managing and Monitoring Replications

Efficient replication management and monitoring are essential to ensure seamless synchronization and detect issues early.

Monitoring Replication Jobs

  1. Accessing Replication Jobs:

    • Go to Administration > Replications in the MSR4 interface to view all replication rules.

    • Select the replication rule of interest, then selection Actions > Edit., You can now modify the existing replication rule.

  2. Running a Replication Job Manually:

    • In Administration > Replications. To manually start a replication, select the relevant rule and click Replicate. This action initiates replication immediately, even if the rule is set to a schedule.

  3. Viewing Job Details:

    • Go to Administration > Replications in the MSR4 interface to monitor and manage ongoing and completed replication jobs.

    • Select the replication rule, and below you should see the historical data of executions. Including any current and past replications.

    • Click on a job entry ID to view logs, error messages, and specific replication statistics. This information aids in troubleshooting and verifying data integrity.

  4. Re-running Failed Jobs:

    • For any job that has encountered issues, select Replicate. Ensure that the endpoint connection and credentials are valid before re-running jobs.