Enable admission controller enforcement

Learn how to enforce security policies in your cluster before deployments are created.

The StackRox Kubernetes Security Platform works with Kubernetes Admission Controllers and OpenShift Admission plug-ins to allow you to enforce security policies before Kubernetes (or OpenShift) creates workloads (for example, deployments, daemon sets or jobs). The StackRox admission controller prevents users from creating workloads that violate policies you configure in the StackRox Kubernetes Security Platform. Beginning from the StackRox Kubernetes Security Platform version 3.0.41, you can also configure the admission controller to prevent updates to workloads that violate policies.

The StackRox Kubernetes Security Platform uses the ValidatingAdmissionWebhook controller to verify that the resource being provisioned complies with the specified security policies. To handle this, the StackRox Kubernetes Security Platform creates a ValidatingWebhookConfiguration which contains multiple webhook rules. When the Kubernetes (or OpenShift) API server receives a request that matches one of the webhook rules, the API server sends an AdmissionReview request to the StackRox Kubernetes Security Platform. The StackRox Kubernetes Security Platform then accepts or rejects the request based on the configured security policies.

To use admission controller enforcement on OpenShift, you need the StackRox Kubernetes Security Platform version 3.0.49 or newer.

Contact StackRox support before enabling admission controller enforcement. We’ll work with you to make your rollout a success.

If you intend to use admission controller enforcement, consider the following:

  1. API latency: Using admission controller enforcement increases Kubernetes (or OpenShift) API latency as it involves additional API validation requests. Many standard Kubernetes libraries, such as fabric8, have short Kubernetes (or OpenShift) API timeouts by default. Also, consider API timeouts in any custom automation you may be using.

  2. Image scanning: You can choose whether the admission controller scans images while reviewing requests by setting the Contact Image Scanners option in the cluster configuration panel.

    • If you enable this setting, the StackRox Kubernetes Security Platform contacts image scanners if scan results aren’t already available, which adds considerable latency.
    • If you disable this setting, the enforcement decision only considers image scan criteria if cached scan results are available.
      • For the StackRox Kubernetes Security Platform version 3.0.41 and newer, the cached scan results are only available for images referenced by their image digest (@sha256:...).
  3. You can use admission controller enforcement for:

    • options in the pod securityContext,
    • deployment configurations, and
    • image components and vulnerabilities.

    However, you can’t use it for:

    • any runtime behavior, such as processes, or
    • any policies based on port exposure.
  4. The admission controller may fail if there are connectivity issues between the Kubernetes (or OpenShift) API server and StackRox Sensor. To resolve this issue, delete the ValidatingWebhookConfiguration object as described in the Disable admission controller enforcement section.

  5. If you have deploy-time enforcement enabled for a policy and you enable the admission controller, the StackRox Kubernetes Security Platform attempts to block deployments that violate the policy. If a noncompliant deployment slips past the admission controller (for example, in case of a timeout), the StackRox Kubernetes Security Platform still applies other deploy-time enforcement mechanisms, such as scaling to zero replicas.

Enable admission controller enforcement

You can enable admission controller enforcement from the Clusters view when you Install a Sensor or edit an existing cluster configuration.

For the StackRox Kubernetes Security Platform version 3.0.55 and newer

Beginning form the StackRox Kubernetes Security Platform version 3.0.55.0, the admission controller webhook is deployed into the cluster by default.

  1. On the StackRox portal, navigate to Platform Configuration > Clusters.

  2. Select an existing cluster from the list or select + New Cluster.

  3. In the cluster configuration panel, fill in the details for your cluster.

  4. We recommend that you only turn on the Configure Admission Controller Webhook to listen on creates toggle if you are planning to use the admission controller to enforce on object create events.

  5. We recommend that you only turn on the Configure Admission Controller Webhook to listen on updates toggle, if you are planning to use the admission controller to enforce on update events.

  6. We recommend that you only turn on the Enable Admission Controller Webhook to listen on exec and port-forward events toggle if you are planning to use the admission controller to enforce on pod execution and pod port forwards events.

  7. Configure the following options:

    • Enforce on Object Creates: This toggle controls the behavior of the admission control service. You must have the Configure Admission Controller Webhook to listen on creates toggle turned on for this to work.
    • Enforce on Object Updates: This toggle controls the behavior of the admission control service. You must have the Configure Admission Controller Webhook to listen on updates toggle turned on for this to work. 
  8. Select Next.

  9. In the Download files section, select Download YAML Files and Keys.

    When enabling admission controller for an existing cluster, if you make any changes in the:

    • Static Configuration section, you must download the YAML files and redeploy the Sensor.
    • Dynamic Configuration section, you can skip downloading the files and deployment, as the StackRox Kubernetes Security Platform automatically syncs the Sensor and applies the changes.
  10. Select Finish.

  11. After you provision a new cluster with the generated YAML, run the following command to verify if admission controller enforcement is set up:

    Copy
    $ kubectl get ValidatingWebhookConfiguration
    NAME       CREATED AT
    stackrox   2019-09-24T06:07:34Z
    Copy
    $ oc get ValidatingWebhookConfiguration
    NAME       CREATED AT
    stackrox   2019-09-24T06:07:34Z

For the StackRox Kubernetes Security Platform version 3.0.54 and older

  1. On the StackRox portal, navigate to Platform Configuration > Clusters.

  2. Select an existing cluster from the list or select + New Cluster.

  3. In the cluster configuration panel, fill in the details for your cluster.

  4. Turn on the Create Admission Controller Webhook toggle in the Static Configuration section. This setting controls whether Kubernetes (or OpenShift) is configured to contact the StackRox Kubernetes Security Platform with AdmissionReview requests.

  5. We recommend that you only turn on the Configure Admission Controller Webhook to listen on updates toggle, if you are planning to use the admission controller to enforce on updates.

    The Configure Admission Controller Webhook to listen on updates option is only available in the StackRox Kubernetes Security Platform version 3.0.41 and newer.

    When you keep it turned off, the StackRox Kubernetes Security Platform create the ValidatingWebhookConfiguration in a way that causes the Kubernetes (or OpenShift) API server not to send object update events. Since the volume of object updates is usually higher than the object creates, leaving this turned off limits the load on the admission control service.

  6. Turn on the Enable Admission Controller toggle in the Dynamic Configuration section. This setting controls whether the StackRox Kubernetes Security Platform evaluates policies; if it’s disabled, all AdmissionReview requests are automatically accepted.

  7. Configure the following options:

    • Enforce on Updates: This toggle controls the behavior of the admission control service. You must have the Configure Admission Controller Webhook to listen on updates toggle turned on for this to work.

      The Enforce on Updates option is only available in the StackRox Kubernetes Security Platform version 3.0.41 and newer.

    • Timeout: The maximum time in seconds, the StackRox Kubernetes Security Platform should wait while evaluating admission review requests. Use it to set request timeouts when you enable image scanning. If the image scan runs longer than the specified time, the StackRox Kubernetes Security Platform accepts the request. Other enforcement options, such as scaling the deployment to zero replicas, are still applied later if the image violates applicable policies.

    • Contact Image Scanners: Turn on this toggle to enable image scanning. See information about image scanning before you enable this toggle.

    • Disable Use of Bypass Annotation: Turn on this toggle to disable bypassing the admission controller.

  8. Select Next.

  9. In the Download files section, select Download YAML Files and Keys.

    When enabling admission controller for an existing cluster, if you make any changes in the:

    • Static Configuration section, you must download the YAML files and redeploy the Sensor.
    • Dynamic Configuration section, you can skip downloading the files and deployment, as the StackRox Kubernetes Security Platform automatically syncs the Sensor and applies the changes.
  10. Select Finish.

  11. After you provision a new cluster with the generated YAML, run the following command to verify if admission controller enforcement is set up:

    Copy
    $ kubectl get ValidatingWebhookConfiguration
    NAME       CREATED AT
    stackrox   2019-09-24T06:07:34Z
    Copy
    $ oc get ValidatingWebhookConfiguration
    NAME       CREATED AT
    stackrox   2019-09-24T06:07:34Z

Bypass admission controller enforcement

To bypass the admission controller, add the admission.stackrox.io/break-glass annotation to your configuration YAML. Bypassing the admission controller triggers a policy violation which includes deployment details. We recommend providing an issue-tracker link or some other reference as the value of this annotation so that others can understand why you bypassed the admission controller.

Disable admission controller enforcement

For the StackRox Kubernetes Security Platform version 3.0.55 and newer

To disable the admission controller enforcement on creates and updates:

  1. Navigate to Platform Configuration > Clusters.
  2. Select an existing cluster from the list.
  3. Turn off the Enforce on Object Creates and Enforce on Object Updates toggle in the Dynamic Configuration section.
  4. Select Next.
  5. Select Finish.

For the StackRox Kubernetes Security Platform version 3.0.54 and older

To disable the admission controller:

  1. Navigate to Platform Configuration > Clusters.
  2. Select an existing cluster from the list.
  3. Turn off the Enable Admission Controller toggle in the Dynamic Configuration section.
  4. Select Next.
  5. Select Finish.

To disable the admission controller enforcement on pod executions and pod port forwards, you can either disable the system policies associated with the admission controller or you can disable the webhook.

Disable associated policies

You can turn off the enforcement on relevant policies, which in turn instructs the admission controller to skip enforcements. To do this:

  1. Navigate to Platform Configuration > System Policies.
  2. Disable enforcement on the default policies:
    • In the policies view, scroll down and move your mouse over to the Kubernetes Actions: Exec into Pod policy. Then select the power icon to disable the policy.
    • In the policies view, scroll down and move your mouse over to the Kubernetes Actions: Port Forward to Pod policy. Then select the power icon to disable the policy.
  3. Disable enforcement on any other custom policies that you’ve created by using criteria from the default Kubernetes Actions: Port Forward to Pod and Kubernetes Actions: Exec into Pod policies.

Disable the webhook

If you disable the admission controller by turning off the webhook, you must redeploy the sensor bundle.

  1. Navigate to Platform Configuration > Clusters.

  2. Select an existing cluster from the list.

  3. Turn off the Enable Admission Controller Webhook to listen on exec and port-forward events toggle in the Static Configuration section.

  4. Select Next to continue with Sensor setup.

  5. Click Download YAML File and Keys.

  6. From a system that has access to the monitored cluster, unzip and run the sensor script:

    Copy
    unzip -d sensor sensor-[cluster-name].zip
    
    ./sensor/sensor.sh

    If you get a warning that you don’t have the required permissions to deploy the sensor, follow the on-screen instructions, or contact your cluster administrator for assistance.

    After the sensor is deployed, it contacts StackRox Central and provides cluster information.

  7. Return to the StackRox portal and check if the deployment is successful. If it’s successful, a green checkmark appears under section #2.

    • If you don’t see a green checkmark, use the following command to check for problems:

      Copy
      kubectl get pod -n stackrox -w
      Copy
      oc get pod -n stackrox -w
  8. Select Finish.

When you disable the admission controller, the StackRox Kubernetes Security Platform doesn’t delete the ValidatingWebhookConfiguration. However, instead of checking requests for violations, it accepts all AdmissionReview requests.

To remove the ValidatingWebhookConfiguration object, run the following command in the secured cluster:

Copy
kubectl delete ValidatingWebhookConfiguration/stackrox
Copy
oc delete ValidatingWebhookConfiguration/stackrox

Additional information

This section is only applicable if you are using the StackRox Kubernetes Security Platform version 3.0.41 or newer because of the changes made to the StackRox admission controller.

ValidatingWebhookConfiguration YAML changes

  1. The webhook isn’t the part of Sensor anymore, and the reference service changed from Sensor to a dedicated admission control service. When you upgrade the StackRox Kubernetes Security Platform to version 3.0.41 or newer, the upgrade instructions handle this change.
  2. With the new configuration, you can enforce security policies on object updates as well, in addition to enforcing on object creation.
  3. Beginning for the StackRox Kubernetes Security Platform version 3.0.55, you can enforce security policies on pod execution and pod port forward events.

If Central or Sensor is unavailable

The admission controller requires an initial configuration from Sensor to work. Kubernetes (or OpenShift) saves this configuration, and it remains accessible even if all admission control service replicas are rescheduled onto other nodes. If this initial configuration exists, the admission controller enforces all configured deploy-time policies.

If Sensor or Central becomes unavailable later:

  • you won’t be able to run image scans, or query information about cached image scans. However, admission controller enforcement still functions based on the available information gathered before the timeout expires, even if the gathered information is incomplete.

  • you won’t be able to disable the admission controller from the StackRox portal or modify enforcement for an existing policy as the changes won’t get propagated to the admission control service.

    If you need to disable admission control enforcement, you can delete the validating webhook configuration by running the following command:

    Copy
    kubectl delete ValidatingWebhookConfiguration/stackrox
    Copy
    oc delete ValidatingWebhookConfiguration/stackrox

Make the admission controller more reliable

  1. We recommend that you schedule the admission control service on the control plane and not on worker nodes. The deployment YAML file includes a soft preference for running on the control plane, however it’s not enforced.

  2. By default, the admission control service runs 3 replicas. To increase reliability, you could increase the replicas by running the following command:

    Copy
    kubectl -n stackrox scale deploy/admission-control --replicas=<number-of-replicas>
    Copy
    oc -n stackrox scale deploy/admission-control --replicas=<number-of-replicas>

Use with the roxctl CLI

You can use the following options when you generate a sensor deployment YAML file:

  • --admission-controller-listen-on-updates: If you use this option, the StackRox Kubernetes Security Platform generates a sensor bundle with a validatingwebhookconfiguration pre-configured to receive update events from the Kubernetes (or OpenShift) API server.
  • --admission-controller-enforce-on-updates: If you use this option, the StackRox Kubernetes Security Platform configures Central such that the admission controller also enforces security policies object updates.

Both these options are optional, and are false by default.

Questions?

We're happy to help! Reach out to us to discuss questions, issues, or feature requests.

© 2021 StackRox Inc. All rights reserved.