This section contains the MCP 2019.2.18 known issues and workarounds. For other MCP known issues also applicable to MCP 2019.2.18, see Known issues and corresponding sections in the previous maintenance updates.
Fixed in 2019.2.19
Due to a community issue causing incompatibility of several plugins with the
current Jenkins version, a pipeline-library error occurs after applying MCP
maintenance updates.
Workaround:
Log in to the Salt Master node.
Update the salt-formula-jenkins package to the version from the MCP
maintenance update 2019.2.19 (or newer, if any):
wget http://mirror.mirantis.com/update/2019.2.19/salt-formulas/xenial/pool/main/s/salt-formula-jenkins/salt-formula-jenkins_2017.8%2B202204261034.22b4455~xenial1_all.deb
dpkg -i salt-formula-jenkins*.deb
Refresh pillars and synchronize Salt modules:
salt '*' saltutil.refresh_pillar
salt '*' saltutil.sync_all
Apply the jenkins.client.plugin state:
salt -C "I@jenkins:client and not I@salt:master" state.sls jenkins.client.plugin
Execute the following script to manually trigger Jenkins restart:
JENKINS_USERNAME=$(salt --out newline_values_only -C 'I@jenkins:client and not I@salt:master' config.get jenkins:client:master:username)
JENKINS_PASSWORD=$(salt --out newline_values_only -C 'I@jenkins:client and not I@salt:master' config.get jenkins:client:master:password)
JENKINS_HOST=$(salt --out newline_values_only -C 'I@jenkins:client and not I@salt:master' config.get jenkins:client:master:host)
JENKINS_PORT=$(salt --out newline_values_only -C 'I@jenkins:client and not I@salt:master' config.get jenkins:client:master:port)
JENKINS_PROTO=$(salt --out newline_values_only -C 'I@jenkins:client and not I@salt:master' config.get jenkins:client:master:proto)
cd $(mktemp -d)
wget "${JENKINS_PROTO}://${JENKINS_HOST}:${JENKINS_PORT}/jnlpJars/jenkins-cli.jar"
java -jar jenkins-cli.jar -s "${JENKINS_PROTO}://${JENKINS_HOST}:${JENKINS_PORT}" -auth "${JENKINS_USERNAME}:${JENKINS_PASSWORD}" safe-restart
Verify that Jenkins has successfully rebooted. The following command should return a list of plugins instead of a 50* error:
java -jar jenkins-cli.jar -s "${JENKINS_PROTO}://${JENKINS_HOST}:${JENKINS_PORT}" -auth "${JENKINS_USERNAME}:${JENKINS_PASSWORD}" list-plugins
Apply the changes:
salt -C "I@jenkins:client and not I@salt:master" state.sls jenkins.client
Fixed in 2019.2.19
During the Galera upgrade to v5.7, the mysql-common package should be
upgraded to version 5.7.35. If mysql-common 5.7.36 was already installed,
the Deploy - upgrade Galera cluster Jenkins pipeline job may
fail.
As a workaround, set the OS_DIST_UPGRADE flag in the parameters of the Deploy - upgrade Galera cluster Jenkins pipeline job.
Fixed in 2019.2.19
Performing an offline update of DriveTrain to MCP maintenance update 2019.2.18
using local mirrors leads to broken plugin dependencies. In this case, Jenkins
jobs fail to load Git-based repositories that include pipeline-library, the
entry point to all Jenkins pipeline jobs.
Workaround:
Download all required plugins from a host with Internet access:
mkdir -p temp_plugins
cd temp_plugins
rm -rf *.hpi
cat > urls.txt << EOF
http://archives.jenkins-ci.org/plugins/workflow-step-api/622.vb_8e7c15b_c95a_/workflow-step-api.hpi
http://archives.jenkins-ci.org/plugins/workflow-cps/2659.v52d3de6044d0/workflow-cps.hpi
http://archives.jenkins-ci.org/plugins/github-branch-source/2.9.9/github-branch-source.hpi
http://archives.jenkins-ci.org/plugins/apache-httpcomponents-client-4-api/4.5.13-1.0/apache-httpcomponents-client-4-api.hpi
http://archives.jenkins-ci.org/plugins/command-launcher/1.2/command-launcher.hpi
http://archives.jenkins-ci.org/plugins/build-monitor-plugin/1.12+build.201809061734/build-monitor-plugin.hpi
http://archives.jenkins-ci.org/plugins/config-file-provider/3.8.0/config-file-provider.hpi
http://archives.jenkins-ci.org/plugins/bootstrap5-api/5.1.1-1/bootstrap5-api.hpi
http://archives.jenkins-ci.org/plugins/discard-old-build/1.05/discard-old-build.hpi
http://archives.jenkins-ci.org/plugins/blueocean-personalization/1.24.8/blueocean-personalization.hpi
http://archives.jenkins-ci.org/plugins/external-monitor-job/1.4/external-monitor-job.hpi
http://archives.jenkins-ci.org/plugins/token-macro/2.13/token-macro.hpi
http://archives.jenkins-ci.org/plugins/workflow-cps-global-lib/2.19/workflow-cps-global-lib.hpi
http://archives.jenkins-ci.org/plugins/favorite/2.3.2/favorite.hpi
http://archives.jenkins-ci.org/plugins/trilead-api/1.0.13/trilead-api.hpi
http://archives.jenkins-ci.org/plugins/popper-api/1.16.1-2/popper-api.hpi
http://archives.jenkins-ci.org/plugins/jsch/0.1.55.2/jsch.hpi
http://archives.jenkins-ci.org/plugins/credentials-binding/1.27.1/credentials-binding.hpi
http://archives.jenkins-ci.org/plugins/pam-auth/1.5.1/pam-auth.hpi
http://archives.jenkins-ci.org/plugins/okhttp-api/4.9.3-105.vb96869f8ac3a/okhttp-api.hpi
http://archives.jenkins-ci.org/plugins/pipeline-stage-tags-metadata/1.9.3/pipeline-stage-tags-metadata.hpi
http://archives.jenkins-ci.org/plugins/ldap/1.20/ldap.hpi
http://archives.jenkins-ci.org/plugins/performance/3.12/performance.hpi
http://archives.jenkins-ci.org/plugins/copyartifact/1.46.2/copyartifact.hpi
http://archives.jenkins-ci.org/plugins/pipeline-utility-steps/2.1.0/pipeline-utility-steps.hpi
http://archives.jenkins-ci.org/plugins/blueocean-pipeline-api-impl/1.24.8/blueocean-pipeline-api-impl.hpi
http://archives.jenkins-ci.org/plugins/workflow-remote-loader/1.5/workflow-remote-loader.hpi
http://archives.jenkins-ci.org/plugins/blueocean-display-url/2.4.1/blueocean-display-url.hpi
http://archives.jenkins-ci.org/plugins/mercurial/2.14/mercurial.hpi
http://archives.jenkins-ci.org/plugins/echarts-api/5.2.1-2/echarts-api.hpi
http://archives.jenkins-ci.org/plugins/simple-theme-plugin/0.5.1/simple-theme-plugin.hpi
http://archives.jenkins-ci.org/plugins/github-api/1.301-378.v9807bd746da5/github-api.hpi
http://archives.jenkins-ci.org/plugins/matrix-auth/2.6.6/matrix-auth.hpi
http://archives.jenkins-ci.org/plugins/pubsub-light/1.13/pubsub-light.hpi
http://archives.jenkins-ci.org/plugins/workflow-aggregator/2.5/workflow-aggregator.hpi
http://archives.jenkins-ci.org/plugins/bootstrap4-api/4.6.0-3/bootstrap4-api.hpi
http://archives.jenkins-ci.org/plugins/test-stability/2.3/test-stability.hpi
http://archives.jenkins-ci.org/plugins/multiple-scms/0.6/multiple-scms.hpi
http://archives.jenkins-ci.org/plugins/workflow-durable-task-step/2.40/workflow-durable-task-step.hpi
http://archives.jenkins-ci.org/plugins/jdk-tool/1.1/jdk-tool.hpi
http://archives.jenkins-ci.org/plugins/extensible-choice-parameter/1.6.0/extensible-choice-parameter.hpi
http://archives.jenkins-ci.org/plugins/pipeline-graph-analysis/1.11/pipeline-graph-analysis.hpi
http://archives.jenkins-ci.org/plugins/blueocean-rest-impl/1.24.8/blueocean-rest-impl.hpi
http://archives.jenkins-ci.org/plugins/blueocean-jwt/1.24.8/blueocean-jwt.hpi
http://archives.jenkins-ci.org/plugins/windows-slaves/1.0/windows-slaves.hpi
http://archives.jenkins-ci.org/plugins/envinject/2.3.0/envinject.hpi
http://archives.jenkins-ci.org/plugins/email-ext/2.84/email-ext.hpi
http://archives.jenkins-ci.org/plugins/caffeine-api/2.9.2-29.v717aac953ff3/caffeine-api.hpi
http://archives.jenkins-ci.org/plugins/blueocean-autofavorite/1.2.5/blueocean-autofavorite.hpi
http://archives.jenkins-ci.org/plugins/handy-uri-templates-2-api/2.1.8-1.0/handy-uri-templates-2-api.hpi
http://archives.jenkins-ci.org/plugins/prometheus/2.0.0/prometheus.hpi
http://archives.jenkins-ci.org/plugins/cloudbees-folder/6.688.vfc7a_a_69059e0/cloudbees-folder.hpi
http://archives.jenkins-ci.org/plugins/structs/308.v852b473a2b8c/structs.hpi
http://archives.jenkins-ci.org/plugins/workflow-support/3.8/workflow-support.hpi
http://archives.jenkins-ci.org/plugins/envinject-api/1.8/envinject-api.hpi
http://archives.jenkins-ci.org/plugins/blueocean-rest/1.24.8/blueocean-rest.hpi
http://archives.jenkins-ci.org/plugins/extended-choice-parameter/0.76/extended-choice-parameter.hpi
http://archives.jenkins-ci.org/plugins/javadoc/1.6/javadoc.hpi
http://archives.jenkins-ci.org/plugins/workflow-scm-step/2.13/workflow-scm-step.hpi
http://archives.jenkins-ci.org/plugins/blueocean-events/1.24.8/blueocean-events.hpi
http://archives.jenkins-ci.org/plugins/sse-gateway/1.24/sse-gateway.hpi
http://archives.jenkins-ci.org/plugins/blueocean/1.24.8/blueocean.hpi
http://archives.jenkins-ci.org/plugins/monitoring/1.87.0/monitoring.hpi
http://archives.jenkins-ci.org/plugins/ant/1.8/ant.hpi
http://archives.jenkins-ci.org/plugins/jquery3-api/3.6.0-2/jquery3-api.hpi
http://archives.jenkins-ci.org/plugins/artifactory/3.15.4/artifactory.hpi
http://archives.jenkins-ci.org/plugins/ace-editor/1.1/ace-editor.hpi
http://archives.jenkins-ci.org/plugins/ssh-agent/1.17/ssh-agent.hpi
http://archives.jenkins-ci.org/plugins/handlebars/1.1.1/handlebars.hpi
http://archives.jenkins-ci.org/plugins/blueocean-dashboard/1.24.8/blueocean-dashboard.hpi
http://archives.jenkins-ci.org/plugins/popper2-api/2.10.2-1/popper2-api.hpi
http://archives.jenkins-ci.org/plugins/display-url-api/2.3.5/display-url-api.hpi
http://archives.jenkins-ci.org/plugins/branch-api/2.6.2/branch-api.hpi
http://archives.jenkins-ci.org/plugins/pipeline-model-extensions/1.9.3/pipeline-model-extensions.hpi
http://archives.jenkins-ci.org/plugins/jenkins-design-language/1.24.8/jenkins-design-language.hpi
http://archives.jenkins-ci.org/plugins/font-awesome-api/5.15.4-1/font-awesome-api.hpi
http://archives.jenkins-ci.org/plugins/junit/1.53/junit.hpi
http://archives.jenkins-ci.org/plugins/mailer/1.34.2/mailer.hpi
http://archives.jenkins-ci.org/plugins/antisamy-markup-formatter/1.5/antisamy-markup-formatter.hpi
http://archives.jenkins-ci.org/plugins/workflow-multibranch/2.24/workflow-multibranch.hpi
http://archives.jenkins-ci.org/plugins/credentials/2.6.1/credentials.hpi
http://archives.jenkins-ci.org/plugins/git-client/3.9.0/git-client.hpi
http://archives.jenkins-ci.org/plugins/github/1.34.2/github.hpi
http://archives.jenkins-ci.org/plugins/run-condition/1.5/run-condition.hpi
http://archives.jenkins-ci.org/plugins/lockable-resources/2.14/lockable-resources.hpi
http://archives.jenkins-ci.org/plugins/pipeline-build-step/2.16/pipeline-build-step.hpi
http://archives.jenkins-ci.org/plugins/blueocean-jira/1.24.8/blueocean-jira.hpi
http://archives.jenkins-ci.org/plugins/blueocean-i18n/1.24.8/blueocean-i18n.hpi
http://archives.jenkins-ci.org/plugins/jira/3.3/jira.hpi
http://archives.jenkins-ci.org/plugins/blueocean-core-js/1.24.8/blueocean-core-js.hpi
http://archives.jenkins-ci.org/plugins/rebuild/1.33/rebuild.hpi
http://archives.jenkins-ci.org/plugins/heavy-job/1.1/heavy-job.hpi
http://archives.jenkins-ci.org/plugins/blueocean-web/1.24.8/blueocean-web.hpi
http://archives.jenkins-ci.org/plugins/metrics/4.0.2.2/metrics.hpi
http://archives.jenkins-ci.org/plugins/jobConfigHistory/2.18.2/jobConfigHistory.hpi
http://archives.jenkins-ci.org/plugins/pipeline-milestone-step/1.3.1/pipeline-milestone-step.hpi
http://archives.jenkins-ci.org/plugins/pipeline-rest-api/2.10/pipeline-rest-api.hpi
http://archives.jenkins-ci.org/plugins/authentication-tokens/1.3/authentication-tokens.hpi
http://archives.jenkins-ci.org/plugins/snakeyaml-api/1.29.1/snakeyaml-api.hpi
http://archives.jenkins-ci.org/plugins/ivy/2.1/ivy.hpi
http://archives.jenkins-ci.org/plugins/pipeline-stage-step/291.vf0a8a7aeeb50/pipeline-stage-step.hpi
http://archives.jenkins-ci.org/plugins/htmlpublisher/1.25/htmlpublisher.hpi
http://archives.jenkins-ci.org/plugins/docker-workflow/1.17/docker-workflow.hpi
http://archives.jenkins-ci.org/plugins/momentjs/1.1.1/momentjs.hpi
http://archives.jenkins-ci.org/plugins/maven-plugin/3.8.1/maven-plugin.hpi
http://archives.jenkins-ci.org/plugins/build-timeout/1.19/build-timeout.hpi
http://archives.jenkins-ci.org/plugins/categorized-view/1.10/categorized-view.hpi
http://archives.jenkins-ci.org/plugins/pipeline-input-step/427.va6441fa17010/pipeline-input-step.hpi
http://archives.jenkins-ci.org/plugins/blueocean-config/1.24.8/blueocean-config.hpi
http://archives.jenkins-ci.org/plugins/bouncycastle-api/2.17/bouncycastle-api.hpi
http://archives.jenkins-ci.org/plugins/slack/2.49/slack.hpi
http://archives.jenkins-ci.org/plugins/durable-task/1.37/durable-task.hpi
http://archives.jenkins-ci.org/plugins/plugin-util-api/2.5.0/plugin-util-api.hpi
http://archives.jenkins-ci.org/plugins/workflow-basic-steps/2.24/workflow-basic-steps.hpi
http://archives.jenkins-ci.org/plugins/blueocean-bitbucket-pipeline/1.24.8/blueocean-bitbucket-pipeline.hpi
http://archives.jenkins-ci.org/plugins/pipeline-model-definition/1.9.3/pipeline-model-definition.hpi
http://archives.jenkins-ci.org/plugins/gradle/1.29/gradle.hpi
http://archives.jenkins-ci.org/plugins/plain-credentials/1.8/plain-credentials.hpi
http://archives.jenkins-ci.org/plugins/checks-api/1.7.2/checks-api.hpi
http://archives.jenkins-ci.org/plugins/variant/1.4/variant.hpi
http://archives.jenkins-ci.org/plugins/blueocean-pipeline-editor/1.24.8/blueocean-pipeline-editor.hpi
http://archives.jenkins-ci.org/plugins/script-security/1138.v8e727069a_025/script-security.hpi
http://archives.jenkins-ci.org/plugins/audit-trail/3.8/audit-trail.hpi
http://archives.jenkins-ci.org/plugins/blueocean-commons/1.24.8/blueocean-commons.hpi
http://archives.jenkins-ci.org/plugins/build-blocker-plugin/1.7.3/build-blocker-plugin.hpi
http://archives.jenkins-ci.org/plugins/ssh-credentials/1.18.1/ssh-credentials.hpi
http://archives.jenkins-ci.org/plugins/timestamper/1.16/timestamper.hpi
http://archives.jenkins-ci.org/plugins/git-server/1.9/git-server.hpi
http://archives.jenkins-ci.org/plugins/pipeline-stage-view/2.10/pipeline-stage-view.hpi
http://archives.jenkins-ci.org/plugins/throttle-concurrents/2.0.1/throttle-concurrents.hpi
http://archives.jenkins-ci.org/plugins/description-setter/1.10/description-setter.hpi
http://archives.jenkins-ci.org/plugins/gerrit-trigger/2.35.2/gerrit-trigger.hpi
http://archives.jenkins-ci.org/plugins/matrix-project/1.18.1/matrix-project.hpi
http://archives.jenkins-ci.org/plugins/blueocean-git-pipeline/1.24.8/blueocean-git-pipeline.hpi
http://archives.jenkins-ci.org/plugins/pipeline-model-api/1.9.3/pipeline-model-api.hpi
http://archives.jenkins-ci.org/plugins/git/4.8.3/git.hpi
http://archives.jenkins-ci.org/plugins/jquery/1.12.4-0/jquery.hpi
http://archives.jenkins-ci.org/plugins/build-user-vars-plugin/1.5/build-user-vars-plugin.hpi
http://archives.jenkins-ci.org/plugins/workflow-job/1145.v7f2433caa07f/workflow-job.hpi
http://archives.jenkins-ci.org/plugins/permissive-script-security/0.3/permissive-script-security.hpi
http://archives.jenkins-ci.org/plugins/jquery-detached/1.2.1/jquery-detached.hpi
http://archives.jenkins-ci.org/plugins/scm-api/2.6.5/scm-api.hpi
http://archives.jenkins-ci.org/plugins/blueocean-pipeline-scm-api/1.24.8/blueocean-pipeline-scm-api.hpi
http://archives.jenkins-ci.org/plugins/blueocean-github-pipeline/1.24.8/blueocean-github-pipeline.hpi
http://archives.jenkins-ci.org/plugins/workflow-api/1138.v619fd5201b_2f/workflow-api.hpi
http://archives.jenkins-ci.org/plugins/docker-commons/1.13/docker-commons.hpi
http://archives.jenkins-ci.org/plugins/ssh-slaves/1.29.4/ssh-slaves.hpi
http://archives.jenkins-ci.org/plugins/pipeline-model-declarative-agent/1.1.1/pipeline-model-declarative-agent.hpi
http://archives.jenkins-ci.org/plugins/jackson2-api/2.13.1-246.va8a9f3eaf46a/jackson2-api.hpi
http://archives.jenkins-ci.org/plugins/plot/2.1.0/plot.hpi
http://archives.jenkins-ci.org/plugins/cloudbees-bitbucket-branch-source/726.vb0c1ea6c9336/cloudbees-bitbucket-branch-source.hpi
http://archives.jenkins-ci.org/plugins/jjwt-api/0.11.2-9.c8b45b8bb173/jjwt-api.hpi
EOF
wget -i urls.txt
Using any available tools and data channels, move all .hpi files to
/root/temp_plugins/ of the DriveTrain LCM engine node (cid01).
Log in to the cid01 node as root.
Obtain the DriveTrain Jenkins endpoint and credentials:
salt-call pillar.get jenkins:client:master
Copy these elements to use in the commands that follow: Jenkins IP, port, login, password.
Download the jenkins-cli client from the DriveTrain Jenkins to the
/root folder:
wget https://%JENKINS_IP%:%JENKINS_PORT%/jnlpJars/jenkins-cli.jar
Verify that the jenkins-cli client is operating properly:
java -jar jenkins-cli.jar -s https://%JENKINS_IP%:%JENKINS_PORT%/ -auth %JENKINS_LOGIN%:%JENKINS_PASSWORD% list-plugins
The command output should include a list of all installed plugins and versions.
Note
In the steps that follow, assume that the .hpi files are
located in the /root/temp_plugins directory.
Copy the .hpi plugin files to the Docker volume.
Note
The CLI tool requires an accessible URI to install the plugin,
file:// in the example below.
rm -rf /srv/volumes/jenkins/fresh_plugins
mkdir /srv/volumes/jenkins/fresh_plugins
chmod 755 /srv/volumes/jenkins/fresh_plugins
cd /root/temp_plugins
cp *.hpi /srv/volumes/jenkins/fresh_plugins/
chmod 644 /srv/volumes/jenkins/fresh_plugins/*.hpi
Install the plugins:
ls *.hpi | xargs -tI{} java -jar ../jenkins-cli.jar -s https://%JENKINS_IP%:%JENKINS_PORT%/ -auth %JENKINS_LOGIN%:%JENKINS_PASSWORD% install-plugin file:///var/jenkins_home/fresh_plugins/{}
Remove the temporary folder from the Docker volume:
rm -rf /srv/volumes/jenkins/fresh_plugins
Reboot Jenkins:
cd /root/
java -jar jenkins-cli.jar -s https://%JENKINS_IP%:%JENKINS_PORT%/ -auth %JENKINS_LOGIN%:%JENKINS_PASSWORD% safe-restart
Rebooting requires some time. During this time, Jenkins responds with
503 errors.
Verify the successful rebooting of Jenkins:
java -jar jenkins-cli.jar -s https://%JENKINS_IP%:%JENKINS_PORT%/ -auth %JENKINS_LOGIN%:%JENKINS_PASSWORD% list-plugins
If Jenkins is up and running, the output will include plugins in the place of Java tracebacks.