Enable admission controller enforcement

Learn how to enforce security policies in your cluster before deployments are created.

The StackRox Kubernetes Security Platform works with Kubernetes Admission Controllers and OpenShift Admission plug-ins to allow you to enforce security policies before Kubernetes (or OpenShift) creates workloads (for example, deployments, daemon sets or jobs). The StackRox admission controller prevents users from creating workloads that violate policies you configure in the StackRox Kubernetes Security Platform. Beginning from the StackRox Kubernetes Security Platform version 3.0.41, you can also configure the admission controller to prevent updates to workloads that violate policies.

The StackRox Kubernetes Security Platform uses the ValidatingAdmissionWebhook controller to verify that the resource being provisioned complies with the specified security policies. To handle this, the StackRox Kubernetes Security Platform creates a ValidatingWebhookConfiguration which contains multiple webhook rules. When the Kubernetes (or OpenShift) API server receives a request that matches one of the webhook rules, the API server sends an AdmissionReview request to the StackRox Kubernetes Security Platform. The StackRox Kubernetes Security Platform then accepts or rejects the request based on the configured security policies.

To use admission controller enforcement on OpenShift, you need the StackRox Kubernetes Security Platform version 3.0.49 or newer.

Contact StackRox support before enabling admission controller enforcement. We’ll work with you to make your rollout a success.

If you intend to use admission controller enforcement, consider the following:

  1. API latency: Using admission controller enforcement increases Kubernetes (or OpenShift) API latency as it involves additional API validation requests. Many standard Kubernetes libraries, such as fabric8, have short Kubernetes (or OpenShift) API timeouts by default. Also, consider API timeouts in any custom automation you may be using.

  2. Image scanning: You can choose whether the admission controller scans images while reviewing requests by setting the Contact Image Scanners option in the cluster configuration panel.

    • If you enable this setting, the StackRox Kubernetes Security Platform contacts image scanners if scan results aren’t already available, which adds considerable latency.
    • If you disable this setting, the enforcement decision only considers image scan criteria if cached scan results are available.
      • For the StackRox Kubernetes Security Platform version 3.0.41 and newer, the cached scan results are only available for images referenced by their image digest (@sha256:...).
  3. You can use admission controller enforcement for:

    • options in the pod securityContext,
    • deployment configurations, and
    • image components and vulnerabilities.

    However, you can’t use it for:

    • any runtime behavior, such as processes, or
    • any policies based on port exposure.
  4. The admission controller may fail if there are connectivity issues between the Kubernetes (or OpenShift) API server and StackRox Sensor. To resolve this issue, delete the ValidatingWebhookConfiguration object as described in the Disable admission controller enforcement section.

  5. If you have deploy-time enforcement enabled for a policy and you enable the admission controller, the StackRox Kubernetes Security Platform attempts to block deployments that violate the policy. If a noncompliant deployment slips past the admission controller (for example, in case of a timeout), the StackRox Kubernetes Security Platform still applies other deploy-time enforcement mechanisms, such as scaling to zero replicas.

Enable admission controller enforcement

You can enable admission controller enforcement from the Clusters view when you Install a Sensor or edit an existing cluster configuration.

  1. On the StackRox portal, navigate to Platform Configuration > Clusters.

  2. Select an existing cluster from the list or select + New Cluster.

  3. In the cluster configuration panel, fill in the details for your cluster.

  4. Turn on the Create Admission Controller Webhook toggle in the Static Configuration section. This setting controls whether Kubernetes (or OpenShift) is configured to contact the StackRox Kubernetes Security Platform with AdmissionReview requests.

  5. We recommend that you only turn on the Configure Admission Controller Webhook to listen on updates toggle, if you are planning to use the admission controller to enforce on updates.

    The Configure Admission Controller Webhook to listen on updates option is only available in the StackRox Kubernetes Security Platform version 3.0.41 and newer.

    When you keep it turned off, the StackRox Kubernetes Security Platform create the ValidatingWebhookConfiguration in a way that causes the Kubernetes (or OpenShift) API server not to send object update events. Since the volume of object updates is usually higher than the object creates, leaving this turned off limits the load on the admission control service.

  6. Turn on the Enable Admission Controller toggle in the Dynamic Configuration section. This setting controls whether the StackRox Kubernetes Security Platform evaluates policies; if it’s disabled, all AdmissionReview requests are automatically accepted.

  7. Configure the following options:

    • Enforce on Updates: This toggle controls the behavior of the admission control service. You must have the Configure Admission Controller Webhook to listen on updates toggle turned on for this to work.

      The Enforce on Updates option is only available in the StackRox Kubernetes Security Platform version 3.0.41 and newer.

    • Timeout: The maximum time in seconds, the StackRox Kubernetes Security Platform should wait while evaluating admission review requests. Use it to set request timeouts when you enable image scanning. If the image scan runs longer than the specified time, the StackRox Kubernetes Security Platform accepts the request. Other enforcement options, such as scaling the deployment to zero replicas, are still applied later if the image violates applicable policies.

    • Contact Image Scanners: Turn on this toggle to enable image scanning. See information about image scanning before you enable this toggle.

    • Disable Use of Bypass Annotation: Turn on this toggle to disable bypassing the admission controller.

  8. Select Next.

  9. In the Download files section, select Download YAML Files and Keys.

    When enabling admission controller for an existing cluster, if you make any changes in the:

    • Static Configuration section, you must download the YAML files and redeploy the Sensor.
    • Dynamic Configuration section, you can skip downloading the files and deployment, as the StackRox Kubernetes Security Platform automatically syncs the Sensor and applies the changes.
  10. Select Finish.

  11. After you provision a new cluster with the generated YAML, run the following command to verify if admission controller enforcement is set up:

    Copy
    $ kubectl get ValidatingWebhookConfiguration
    NAME       CREATED AT
    stackrox   2019-09-24T06:07:34Z
    Copy
    $ oc get ValidatingWebhookConfiguration
    NAME       CREATED AT
    stackrox   2019-09-24T06:07:34Z

Bypass admission controller enforcement

To bypass the admission controller, add the admission.stackrox.io/break-glass annotation to your configuration YAML. Bypassing the admission controller triggers a policy violation which includes deployment details. We recommend providing an issue-tracker link or some other reference as the value of this annotation so that others can understand why you bypassed the admission controller.

Disable admission controller enforcement

To disable the admission controller:

  1. Navigate to Platform Configuration > Clusters.
  2. Select an existing cluster from the list.
  3. Turn off the Enable Admission Controller toggle in the Dynamic Configuration section.
  4. Select Next.
  5. Select Finish.

When you disable the admission controller, the StackRox Kubernetes Security Platform doesn’t delete the ValidatingWebhookConfiguration. However, instead of checking requests for violations, it accepts all AdmissionReview requests.

To remove the ValidatingWebhookConfiguration object, run the following command in the secured cluster:

Copy
kubectl delete ValidatingWebhookConfiguration/stackrox
Copy
oc delete ValidatingWebhookConfiguration/stackrox

Additional information

This section is only applicable if you are using the StackRox Kubernetes Security Platform version 3.0.41 or newer because of the changes made to the StackRox admission controller.

ValidatingWebhookConfiguration YAML changes

  1. The webhook isn’t the part of Sensor anymore, and the reference service changed from Sensor to a dedicated admission control service. When you upgrade the StackRox Kubernetes Security Platform to version 3.0.41 or newer, the upgrade instructions handle this change.
  2. With the new configuration, you can enforce security policies on object updates as well, in addition to enforcing on object creation.

If Central or Sensor is unavailable

The admission controller requires an initial configuration from Sensor to work. Kubernetes (or OpenShift) saves this configuration, and it remains accessible even if all admission control service replicas are rescheduled onto other nodes. If this initial configuration exists, the admission controller enforces all configured deploy-time policies.

If Sensor or Central becomes unavailable later:

  • you won’t be able to run image scans, or query information about cached image scans. However, admission controller enforcement still functions based on the available information gathered before the timeout expires, even if the gathered information is incomplete.

  • you won’t be able to disable the admission controller from the StackRox portal or modify enforcement for an existing policy as the changes won’t get propagated to the admission control service.

    If you need to disable admission control enforcement, you can delete the validating webhook configuration by running the following command:

    Copy
    kubectl delete ValidatingWebhookConfiguration/stackrox
    Copy
    oc delete ValidatingWebhookConfiguration/stackrox

Make the admission controller more reliable

  1. We recommend that you schedule the admission control service on the control plane and not on worker nodes. The deployment YAML file includes a soft preference for running on the control plane, however it’s not enforced.

  2. By default, the admission control service runs 3 replicas. To increase reliability, you could increase the replicas by running the following command:

    Copy
    kubectl -n stackrox scale deploy/admission-control --replicas=<number-of-replicas>
    Copy
    oc -n stackrox scale deploy/admission-control --replicas=<number-of-replicas>

Use with the roxctl CLI

You can use the following options when you generate a sensor deployment YAML file:

  • --admission-controller-listen-on-updates: If you use this option, the StackRox Kubernetes Security Platform generates a sensor bundle with a validatingwebhookconfiguration pre-configured to receive update events from the Kubernetes (or OpenShift) API server.
  • --admission-controller-enforce-on-updates: If you use this option, the StackRox Kubernetes Security Platform configures Central such that the admission controller also enforces security policies object updates.

Both these options are optional, and are false by default.

Questions?

We're happy to help! Reach out to us to discuss questions, issues, or feature requests.

© 2021 StackRox Inc. All rights reserved.