Exclude images from kubelet garbage collection

Available as of MKE 3.8.17

When the image filesystem on a node fills up, the kubelet runs image garbage collection and removes unused images to free disk space. To prevent the kubelet from removing images that the cluster requires, you can pin those images. The kubelet never removes pinned images during image garbage collection.

By default, MKE pins all MKE and MSR images, as every MKE image carries the com.docker.ucp.version label and every MSR image carries the com.docker.dtr.version label. You can pin additional images and adjust the disk usage thresholds that trigger image garbage collection.

Note

Kubelet image garbage collection is separate from the MKE image pruning feature. For information on how to schedule the cleanup of unused images by MKE, refer to Schedule image pruning.

To exclude images from kubelet garbage collection:

  1. Obtain the current MKE configuration file for your cluster.

  2. In the [cluster_config] section of the configuration file, set the following parameters as required:

    • kubelet_image_gc_high_threshold_percent

    • kubelet_image_gc_low_threshold_percent

    • kubelet_pinned_image_labels

    • kubelet_pinned_images

    For the parameter descriptions and default values, refer to cluster_config table (required).

    For example:

    [cluster_config]
      kubelet_image_gc_high_threshold_percent = 90
      kubelet_image_gc_low_threshold_percent = 70
      kubelet_pinned_image_labels = ["com.docker.ucp.version", "com.docker.dtr.version"]
      kubelet_pinned_images = ["nginx"]
    
  3. Upload the new MKE configuration file.

    MKE recreates the kubelet on each node to apply the new configuration. Recreating the kubelet does not affect running workloads.