Use secured sources for mirrors, repositories, and files

Use secured sources for mirrors, repositories, and files

Note

This feature is available starting from the MCP 2019.2.5 maintenance update. Before enabling the feature, follow the steps described in Apply maintenance updates.

This section provides an instruction on how to configure your cluster model if you plan to download Debian packages, Git mirrors, VM images, or any files required for cluster deployment from a secured HTTP/HTTPS server that can be accessible through login credentials. Such functionality may be required for offline installations when internal mirrors are secured.

If the source HTTP/HTTPS server is secured, the source or url parameters should still include the user ID and password, for example, http://user:password@example.mirantis.com/xenial. Previously, MCP did not enable you to use encrypted pillar inside another variable. Starting from MCP 2019.2.5, you can use a secured HTTP/HTTPS server even if the secrets encryption feature in Reclass is enabled as described in Enable all secrets encryption.

Warning

We recommend that you apply the procedure before the cluster deployment to avoid the cluster breakdown and to automatically apply the changes.

To define secured APT repositories on the cluster nodes:

Note

The exemplary default structure of the APT repositories definition in the cluster model:

linux:
  system:
    repo:
      repo-example:
        source: 'deb http://example.com/ubuntu xenial main'
  • Define a secured APT repository, for example:

    linux:
      system:
        repo:
          repo-example:
            secure: true
            url: example.com/ubuntu
            arch: deb
            protocol: http
            user: foo
            password: bar
            distribution: xenial
            component: main
    
  • Define the APT repositories in case of several APT repositories under the same HTTP/HTTPS secured server with the same credentials. The exemplary structure:

    linux:
      system:
        common_repo_secured:
          arch: deb
          protocol: http
          user: foo
          password: bar
          distribution: xenial
          component: main
        repo:
          test1:
            secure: true
            url: example1.com/ubuntu
          test2:
            secure: true
            url: example2.com/ubuntu
    

Warning

We do not recommend that you apply the changes after the MCP cluster deployment. Though, on your own responsibility, you can apply the changes as follows:

  1. Log in to the Salt Master node.

  2. Run:

    salt '*' saltutil.refresh_pillar
    salt '*' state.apply linux.system.repo
    

To define a secured file source on cluster nodes:

Note

The exemplary default structure of the file sources definition in the cluster model:

linux:
  system:
    file:
      /tmp/sample.txt:
        source: http://techslides.com/demos/samples/sample.txt
        source_hash: 5452459724e85b4e12277d5f8aab8fc9
      sample2.txt:
        name: /tmp/sample2.txt
        source: http://techslides.com/demos/samples/sample.txt

Define a secured file source, for example:

linux:
  system:
    file:
      sample3.tar.gz:
        name: /tmp/sample3.tar.gz
        secured_source:
          protocol: http #optional
          user: username
          password: password
          url: wordpress.org/latest.tar.gz
        secured_hash: #optional
          url: wordpress.org/latest.tar.gz.md5

Warning

We do not recommend that you apply the changes after the MCP cluster deployment. Though, on your own responsibility, you can apply the changes as follows:

  1. Log in to the Salt Master node.

  2. Run:

    salt '*' saltutil.refresh_pillar
    salt '*' state.apply linux.system.repo
    

To define a secured image source on cluster nodes:

Note

The exemplary default structure of the image sources definition in cluster model:

salt:
  control:
    cluster:
      cluster-name:
        node:
          node1:
            provider: node01.domain.com
            size: medium
            image: http://ubuntu.com/download/ubuntu.qcow2
  • Define a secured image sources. The exemplary structure:

    salt:
      control:
        cluster:
          cluster-name:
            node:
              node1:
                provider: node01.domain.com
                size: medium
                image_source:
                  secured: true
                  protocol: http
                  user: foo
                  password: bar
                  url_prefix: ubuntu.com/download
                  url_path: ubuntu.qcow2
    
  • Define the image sources in case of several images from the same HTTP/HTTPS secured server with the same credentials. The exemplary structure:

    salt:
      control:
        common_image_source:
          protocol: http
          user: foo
          password: bar
          url_prefix: ubuntu.com/download
        cluster:
          cluster-name:
            node:
              node1:
                provider: node01.domain.com
                size: medium
                image_source:
                  secured: true
                  url_path: ubuntu-xenial.qcow2
              node2:
                provider: node02.domain.com
                size: medium
                image_source:
                  secured: true
                  url_path: ubuntu-bionic.qcow2
    

Warning

Do not apply the changes after the MCP cluster deployment to avoid the cluster breakdown.

To define a secured Git repositories source for CI/CD nodes:

  1. Update the configuration of the Gerrit project source:

    Note

    The exemplary default structure of the Gerrit project sources in cluster model:

    gerrit:
      client:
        enabled: True
        project:
          test_salt_project:
            enabled: true
            upstream: https://github.com/example/library
    

    Define a secured Gerrit project source, fore example:

    gerrit:
      client:
        enabled: True
        project:
          test_salt_project:
            enabled: true
            upstream_secured: true
            protocol: https
            username: foo
            password: bar
            address: github.com/example/library
    
  2. If the target Gerrit repositories are any of mcp-ci/pipeline-library or mk/mk-pipelines, or they are required for the pipelines execution in Jenkins, add the Jenkins login credentials:

    1. Navigate to the root folder of your cluster model. On the Salt Master node, this is the /srv/salt/reclass directory.

    2. Add the following parameters into ./classes/cluster/<cluster_name>/infra/config/jenkins.yml for Jenkins on the Salt Master node and ./classes/cluster/<cluster_name>/cicd/control/leader.yml for Jenkins on the CI/CD nodes:

      parameters:
        _param:
          source_git_username: <ENCRYPTED_USERNAME>
          source_git_password: <ENCRYPTED_PASSWORD>
      
    3. Include the system.jenkins.client.credential.source_git class into same files for both Jenkins instances:

      classes:
        ...
        - system.jenkins.client.credential.source_git
        ...
      

Warning

We do not recommend that you apply the changes after the MCP cluster deployment. Though, on your own responsibility, you can apply the changes as follows:

  1. Log in to the Salt Master node.

  2. Refresh the pillars:

    salt -C 'I@gerrit:client' saltutil.refresh_pillar
    salt -C 'I@jenkins:client' saltutil.refresh_pillar
    
  3. Apply the gerrit and jenkins states:

    salt -C 'I@gerrit:client' state.apply gerrit.client
    salt -C 'I@jenkins:client' state.apply jenkins.client