Configure Salt Master threads and batching¶

You can configure the number of worker threads for Salt Master based on the number of CPUs available on your Salt Master node.

Also, you can set up batching for the pipeline jobs to run Salt states, targeted for a large number of nodes, on a batch of nodes and define the batch size. By default, batching is force-enabled for the Deploy - OpenStack and Deploy - upgrade MCP DriveTrain Jenkins pipeline jobs as they do not support node targeting. Batch sizing is by default supported by the Deploy - OpenStack, Deploy - update system package(s), and Deploy - upgrade MCP DriveTrain Jenkins pipeline jobs.

To configure Salt Master threads:

1. Open your Git project repository with the Reclass model on the cluster level.

2. In infra/config/init.yml, specify the following pillar for the cfg01 node:

   salt:
     master:
       worker_threads_per_cpu: <value>
   

   Depending on the amount of CPUs, the total amount of worker threads is based on worker_threads_per_cpu multiplied by the number of CPUs. By default, the number of worker threads is set to 40 using the following pillar:

   salt:
     master:
       worker_threads: 40
   

   If both worker_threads_per_cpu and worker_threads are defined, worker_threads_per_cpu is prioritized.

3. Log in to the Salt Master node.

4. Apply the following state:

   salt-call state.sls salt.master
   

5. Verify that the required settings have been applied:

   cat /etc/salt/master.d/master.conf | grep worker_threads
   

To configure Salt Master batching:

1. Open the required Jenkins pipeline job.

2. Configure batch sizing:

   • For the Deploy - OpenStack, Deploy - update system package(s), and Deploy - upgrade MCP DriveTrain Jenkins pipeline jobs, set the BATCH_SIZE parameter to an integer or percentage. For example, 20 or 20%.

     Batch sizing applies using the following workflow:

     1. Verifies that the BATCH_SIZE pipeline job parameter exists.

     2. Verifies the SALT_MASTER_OPT_WORKER_THREADS environment variable.

     3. Verifies the worker_threads_per_cpu pillar parameter and the available number of CPUs.

     4. Verifies the worker_threads pillar parameter.

     5. If none of the steps above match:

        • Prior to the MCP 2019.2.8 maintenance update, disables batching.

        • Starting from the MCP 2019.2.8 maintenance update, sets batching to 2/3 of the available Salt Master worker threads.

   • For other pipeline jobs, to use batching, set the SALT_MASTER_OPT_WORKER_THREADS environment variable in the global Jenkins settings or directly in the pipeline jobs to an integer or percentage. For example, 20 or 20%.

     Batch sizing applies using the following workflow:

     1. Verifies that the BATCH_SIZE pipeline job parameter exists.

     2. Verifies the SALT_MASTER_OPT_WORKER_THREADS environment variable.

     3. If none of the steps above match:

        • Prior to the MCP 2019.2.8 maintenance update, disables batching.

        • Starting from the MCP 2019.2.8 maintenance update, sets batching to 2/3 of the available Salt Master worker threads.