Validation Reference

• 1: Domains
• 1.1: API Domain
• 1.2: File Domain
• 1.3: Kubernetes Domain
• 2: Providers
• 2.1: Kyverno Provider
• 2.2: OPA Provider
• 3: Version Specification

lula-version: ">=v0.2.0"
metadata:
  name: Validate pods with label foo=bar
  uuid: 123e4567-e89b-12d3-a456-426655440000
domain:
  type: kubernetes
  kubernetes-spec:
    resources:
      - name: podsvt
        resource-rule:
          version: v1
          resource: pods
          namespaces: [validation-test]
provider:
  type: opa
  opa-spec:
    rego: |
      package validate

      import future.keywords.every

      validate {
        every pod in input.podsvt {
          podLabel := pod.metadata.labels.foo
          podLabel == "bar"
        }
      }

"yaml.schemas": {
    "${PATH_TO_LULA}/lula/src/pkg/common/schemas/validation.json": "*validation*.yaml"
},

1 - Domains

Domains, generically, are the areas or categories from which data is collected to be evaluated by a Lula Validation. Currently supported Domains are:

• Kubernetes
• API
• File

The domain block of a Lula Validation is given as follows, where the sample is indicating a Kubernetes domain is in use:

# ... Rest of Lula Validation
domain:
    type: kubernetes   # kubernetes or api accepted
    kubernetes-spec:
        # ... Rest of kubernetes-spec
# ... Rest of Lula Validation

Each domain has a particular specification, given by the respective <domain>-spec field of the domain property of the Lula Validation. The sub-pages describe each of these specifications in greater detail.

1.1 - API Domain

The API Domain allows for collection of data (via HTTP Get Requests) generically from API endpoints.

Specification

The API domain Specification accepts a list of Requests and an Options block. Options can be configured at the top-level and will apply to all requests except those which have embedded Options. Request-level options will override top-level Options.

domain: 
  type: api
  api-spec:
    # Options specified at this level will apply to all requests except those with an embedded options block.
    options:
      # Timeout configures the request timeout. The default timeout is 30 seconds (30s). The timeout string is a number followed by a unit suffix (ms, s, m, h, d), such as 30s or 1m.
      timeout: 30s
      # Proxy specifies a proxy server for all requests.
      proxy: "https://my.proxy"
      # Headers is a map of key value pairs to send with all requests.
      headers: 
        key: "value"
        my-customer-header: "my-custom-value"
    # Requests is a list of URLs to query. The request name is the map key used when referencing the resources returned by the API.
    requests:
      # A descriptive name for the request.
      - name: "healthcheck" 
      # The URL of the request. The API domain supports any rfc3986-formatted URI. Lula also supports URL parameters as a separate argument. 
        url: "https://example.com/health/ready"
        # Parameters to append to the URL. Lula also supports full URIs in the URL.
        parameters: 
          key: "value"
        # Request-level options have the same specification as the api-spec-level options. These options apply only to this request.
        options:
          # Configure the request timeout. The default timeout is 30 seconds (30s). The timeout string is a number followed by a unit suffix (ms, s, m, h, d), such as 30s or 1m.
          timeout: 30s
          # Proxy specifies a proxy server for this request.
          proxy: "https://my.proxy"
          # Headers is a map of key value pairs to send with this request.
          headers: 
            key: "value"
            my-customer-header: "my-custom-value"
      - name: "readycheck"
      # etc ...

API Domain Resources

The API response body is serialized into a json object with the Request’s Name as the top-level key. The API status code is included in the output domain resources.

Example output:

"healthcheck": {
  "status": 200,
  "healthy": "ok"
}

1.2 - File Domain

The File domain allows for validation of arbitrary file contents from a list of supported file types. The file domain can evaluate local files and network files. Files are copied to a temporary directory for evaluation and deleted afterwards.

Specification

The File domain specification accepts a descriptive name for the file as well as its path. The filenames and descriptive names must be unique.

domain:
  type: file
  file-spec:
    filepaths:
    - name: config
      path: grafana.ini
      parser: ini         # optionally specify which parser to use for the file type

Supported File Types

The file domain uses OPA’s conftest to parse files into a json-compatible format for validations. Both OPA and Kyverno (using kyverno-json) can validate files parsed by the file domain.

The file domain includes the following file parsers:

• cue
• cyclonedx
• dockerfile
• dotenv
• edn
• hcl1
• hcl2
• hocon
• ignore
• ini
• json
• jsonc
• jsonnet
• properties
• spdx
• string
• textproto
• toml
• vcl
• xml
• yaml

The file domain can also parse arbitrary file types as strings. The entire file contents will be represented as a single string.

The file parser can usually be inferred from the file extension. However, if the file extension does not match the filetype you are parsing (for example, if you have a json file that does not have a .json extension), or if you wish to parse an arbitrary file type as a string, use the parser field in the FileSpec to specify which parser to use. The list above contains all the available parses.

Validations

When writing validations against files, the filepath name must be included as the top-level key in the validation. The placement varies between providers.

Given the following ini file:

[server]
# Protocol (http, https, socket)
protocol = http

The below Kyverno policy validates the protocol is https by including Grafana as the top-level key under check:

metadata:
  name: check-grafana-protocol
  uuid: ad38ef57-99f6-4ac6-862e-e0bc9f55eebe
domain:
  type: file
  file-spec:
    filepaths:
    - name: 'grafana'
      path: 'custom.ini'
provider:
  type: kyverno
  kyverno-spec:
    policy:
      apiVersion: json.kyverno.io/v1alpha1
      kind: ValidatingPolicy
      metadata:
        name: grafana-config
      spec:
        rules:
        - name: protocol-is-https
          assert:
            all:
            - check:
                grafana:
                  server:
                    protocol: https

While in an OPA policy, the filepath Name is the input key to access the config:

metadata:
  name: validate-grafana-config
  uuid: ad38ef57-99f6-4ac6-862e-e0bc9f55eebe
domain:
  type: file
  file-spec:
    filepaths:
    - name: 'grafana'
      path: 'custom.ini'
provider:
  type: opa
  opa-spec:
    rego: |
      package validate
      import rego.v1

      # Default values
      default validate := false
      default msg := "Not evaluated"
      
      validate if {
       check_grafana_config.result
      }
      msg = check_grafana_config.msg

      config := input["grafana"]
      protocol := config.server.protocol

      check_grafana_config = {"result": true, "msg": msg} if {
        protocol == "https"
        msg := "Server protocol is set to https"
      } else = {"result": false, "msg": msg} if {
        protocol == "http"
        msg := "Grafana protocol is insecure"
      }

    output:
      validation: validate.validate
      observations:
        - validate.msg

Parsing files as arbitrary strings

Files that are parsed as strings are represented as a key-value pair where the key is the user-supplied file name and the value is a string representation of the file contexts, including special characters, for e.g. newlines (\n).

As an example, let’s parse a similar file as before as an arbitrary string.

When reading the following multiline file contents as a string:

server = https
port = 3000

The resources for validation will be formatted as a single string with newline characters:

{"config": "server = https\nport = 3000"}

And the following validation will confirm if the server is configured for https:

  domain:
    type: file
    file-spec:
      filepaths:
      - name: 'config'
        path: 'server.txt'
        parser: string
  provider:
    type: opa
    opa-spec:
      rego: |
        package validate
        import rego.v1

        # Default values
        default validate := false
        default msg := "Not evaluated"
        
        validate if {
          check_server_protocol.result
        }
        msg = check_server_protocol.msg
        
        config := input["config"]
        
        check_server_protocol = {"result": true, "msg": msg} if {
          regex.match(
            `server = https\n`,
            config
          )
          msg := "Server protocol is set to https"
        } else = {"result": false, "msg": msg} if {
          regex.match(
            `server = http\n`,
            config
          )
          msg := "Server Protocol must be https - http is disallowed"
        }

      output:
        validation: validate.validate
        observations:
          - validate.msg

Note on Compose

While the file domain is capable of referencing relative file paths in the file-spec, Lula does not de-reference those paths during composition. If you are composing multiple files together, you must either use absolute filepaths (including network filepaths), or ensure that all referenced filepaths are relative to the output directory of the compose command.

Evidence Collection

The use of lula dev get-resources and lula validate --save-resources will produce evidence in the form of json files. These files provide point-in-time evidence for auditing and review purposes.

Evidence collection occurs for each file specified and - in association with any error will produce an empty representation of the target file(s) data to be collected.

1.3 - Kubernetes Domain

The Kubernetes domain provides Lula access to data collection of Kubernetes resources.

[!Important] This domain supports both read and write operations on the Kubernetes cluster in the current context, so use with care

Specification

The Kubernetes specification can be used to return Kubernetes resources as JSON data to the providers. The specification allows for both manifest and resource field data to be retrieved.

The following sample kubernetes-spec yields a list of all pods in the validation-test namespace.

domain:
  type: kubernetes
  kubernetes-spec:
    resources:                          # Optional - Group of resources to read from Kubernetes
    - name: podsvt                      # Required - Identifier to the list or set read by the policy
      resource-rule:                    # Required - Resource selection criteria, at least one resource rule is required
        name:                           # Optional - Used to retrieve a specific resource in a single namespace
        group:                          # Optional - empty or "" for core group
        version: v1                     # Required - Version of resource
        resource: pods                  # Required - Resource type (API-recognized type, not Kind)
        namespaces: [validation-test]   # Optional - Namespaces to validate the above resources in. Empty or "" for all namespace or non-namespaced resources
        field:                          # Optional - Field to grab in a resource if it is in an unusable type, e.g., string json data. Must specify named resource to use.
          jsonpath:                     # Required - Jsonpath specifier of where to find the field from the top level object
          type:                         # Optional - Accepts "json" or "yaml". Default is "json".
          base64:                       # Optional - Boolean whether field is base64 encoded

Lula supports eventual-consistency through use of an optional wait field in the kubernetes-spec. This parameter supports waiting for a specified resource to be Ready in the cluster. This may be particularly useful if evaluating the status of a selected resource or evaluating the children of a specified resource.

domain:
  type: kubernetes
  kubernetes-spec:
    wait:                               # Optional - Group of resources to read from Kubernetes
      group:                            # Optional - Empty or "" for core group
      version: v1                       # Required - Version of resource
      resource: pods                    # Required - Resource type (API-recognized type, not Kind)Required - Resource type (API-recognized type, not Kind)
      name: test-pod-wait               # Required - Name of the resource to wait for
      namespace: validation-test        # Optional - For namespaced resources
      timeout: 30s                      # Optional - Defaults to 30s
    resources:
    - name: podsvt
      resource-rule:
        group:
        version: v1
        resource: pods
        namespaces: [validation-test]

[!Tip] Both resources and wait use the Group, Version, Resource constructs to identify the resource to be evaluated. To identify those using kubectl, executing kubectl explain <resource/kind/short name> will provide the Group and Version, the resource field is the API-recognized type and can be confirmed by consulting the list provided by kubectl api-resources.

Resource Creation

The Kubernetes domain also supports creating, reading, and destroying test resources in the cluster. This feature should be used with caution since it’s writing to the cluster and ideally should be implemented on separate namespaces to make any erroneous outcomes easier to mitigate.

domain:
  type: kubernetes
  kubernetes-spec:
    create-resources:                   # Optional - Group of resources to be created/read/destroyed in Kubernetes
      - name: testPod                   # Required - Identifier to be read by the policy
        namespace: validation-test      # Optional - Namespace to be created if applicable (no need to specify if ns exists OR resource is non-namespaced)
        manifest: |                     # Optional - Manifest string for resource(s) to create; Only optional if file is not specified
          <some manifest(s)>
        file: '<some url>'              # Optional - File name where resource(s) to create are stored; Only optional if manifest is not specified. Currently does not support relative paths.

In addition to simply creating and reading individual resources, you can create a resource, wait for it to be ready, then read the possible children resources that should be created. For example the following kubernetes-spec will create a deployment, wait for it to be ready, and then read the pods that should be children of that deployment:

domain: 
  type: kubernetes
  kubernetes-spec:
    create-resources:
    - name: testDeploy
      manifest: |
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: test-deployment
          namespace: validation-test
        spec:
          replicas: 1
          selector:
            matchLabels:
              app: test-app
          template:
            metadata:
              labels:
                app: test-app
            spec:
              containers:
              - name: test-container
                image: nginx
    wait:
      group: apps
      version: v1
      resource: deployments
      name: test-deployment
      namespace: validation-test
    resources:
    - name: validationTestPods
      resource-rule:
        version: v1
        resource: pods
        namespaces: [validation-test]

[!NOTE] The create-resources is evaluated prior to the wait, and wait is evaluated prior to the resources.

Lists vs Named Resource

When Lula retrieves all targeted resources (bounded by namespace when applicable), the payload is a list of resources. When a resource Name is specified - the payload will be a single object.

Example

Let’s get all pods in the validation-test namespace and evaluate them with the OPA provider:

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: podsvt
      resource-rule:
        group:
        version: v1
        resource: pods
        namespaces: [validation-test]
provider: 
  type: opa
  opa-spec:
    rego: |
      package validate

      import future.keywords.every

      validate {
        every pod in input.podsvt {
          podLabel := pod.metadata.labels.foo
          podLabel == "bar"
        }
      }

[!IMPORTANT] Note how the rego evaluates a list of items that can be iterated over. The podsvt field is the name of the field in the kubernetes-spec.resources that contains the list of items.

Now let’s retrieve a single pod from the validation-test namespace:

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: podvt
      resource-rule:
        name: test-pod-label
        group:
        version: v1
        resource: pods
        namespaces: [validation-test]
provider: 
  type: opa
  opa-spec:  
    rego: |
      package validate

      validate {
        podLabel := input.podvt.metadata.labels.foo
        podLabel == "bar"
      }

[!IMPORTANT] Note how the rego now evaluates a single object called podvt. This is the name of the resource that is being validated.

We can also retrieve a single cluster-scoped resource as follows, where the rego evaluates a single object called namespaceVt.

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: namespaceVt
      resource-rule:
        name: validation-test
        version: v1
        resource: namespaces
provider: 
  type: opa
  opa-spec:  
    rego: |
      package validate

      validate {
        input.namespaceVt.metadata.name == "validation-test"
      }

Extracting Resource Field Data

Many of the tool-specific configuration data is stored as json or yaml text inside configmaps and secrets. Some valuable data may also be stored in json or yaml strings in other resource locations, such as annotations. The field parameter of the resource-rule allows this data to be extracted and used by the Rego.

Here’s an example of extracting config.yaml from a test configmap:

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: configdata
      resource-rule:
        name: test-configmap
        group:
        version: v1
        resource: configmaps
        namespaces: [validation-test]
        field:
          jsonpath: .data.my-config.yaml
          type: yaml
provider: 
  type: opa
  opa-spec:
    rego: |
      package validate

      validate {
        configdata.configuration.foo == "bar"
      }

Where the raw ConfigMap data would look as follows:

configuration:
  foo: bar
  anotherValue:
    subValue: ba

This field also supports grabbing secret data using the optional base64 field as follows

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: secretdata
      resource-rule:
        name: test-secret
        group:
        version: v1
        resource: secrets
        namespaces: [validation-test]
        field: 
          jsonpath: .data.secret-value.json
          type: json
          base64: true

Evidence Collection

The use of lula dev get-resources and lula validate --save-resources will produce evidence in the form of json files. These files provide point-in-time evidence for auditing and review purposes.

The Kubernetes domain requires connectivity to a cluster in order to perform data collection. The inability to connect to a cluster during the evaluation of a validation with --save-resources will result in an empty payload in the associated observation evidence file.

Evidence collection occurs for each resource specified and - in association with any error will produce an empty representation of the target resource(s) to be collected.

2 - Providers

Providers are the policy engines which evaluate the input data from the specified domain. Currently supported Providers are:

• OPA (Open Policy Agent)
• Kyverno

The provider block of a Lula Validation is given as follows, where the sample is indicating the OPA provider is in use:

# ... Rest of Lula Validation
provider:
    type: opa   # opa or kyverno accepted
    opa-spec:
        # ... Rest of opa-spec
# ... Rest of Lula Validation

Each domain specification retreives a specific dataset, and each will return that data to the selected Provider in a domain-specific format. However, this data will always take the form of a JSON object when input to a Provider. For that reason, it is important that Domain and Providerspecifications are not built wholly independently in a given Validation.

2.1 - Kyverno Provider

The Kyverno provider provides Lula with the capability to evaluate the domain in against a Kyverno policy.

Payload Expectation

The validation performed should use the form of provider with the type of kyverno and using the kyverno-spec, along with a valid domain.

Example:

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: podsvt
      resource-rule:
        group:
        version: v1
        resource: pods
        namespaces: [validation-test]
provider: 
  type: kyverno
  kyverno-spec:
    policy:
      apiVersion: json.kyverno.io/v1alpha1          # Required
      kind: ValidatingPolicy                        # Required
      metadata:
        name: pod-policy                            # Required
      spec:
        rules:
          - name: no-latest                         # Required
            # Match payloads corresponding to pods
            match:                                  # Optional
              any:                                  # Assertion Tree
              - apiVersion: v1
                kind: Pod
            assert:                                 # Required
              all:                                  # Assertion Tree
              - message: Pod `{{ metadata.name }}` uses an image with tag `latest`
                check:
                  ~.podsvt:
                    spec:
                      # Iterate over pod containers
                      # Note the `~.` modifier, it means we want to iterate over array elements in descendants
                      ~.containers:
                        image:
                          # Check that an image tag is present
                          (contains(@, ':')): true
                          # Check that the image tag is not `:latest`
                          (ends_with(@, ':latest')): false

You can have mutiple policies defined. Optionally, output.validation can be specified in the kyverno-spec to control which (Policy, Rule) pair control validation allowance/denial, which is in the structure of a comma separated list of rules: policy-name1.rule-name-1,policy-name-1.rule-name-2. If you have a desired observation to include, output.observations can be added to payload to observe violations by a certain (Policy, Rule) pair such as:

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: podsvt
      resource-rule: 
        group: 
        version: v1 
        resource: pods
        namespaces: [validation-test] 
provider: 
  type: kyverno
  kyverno-spec:
    policy:
      apiVersion: json.kyverno.io/v1alpha1
      kind: ValidatingPolicy
      metadata:
        name: labels
      spec:
        rules:
        - name: foo-label-exists
          assert:
            all:
            - check:
                ~.podsvt:
                  metadata:
                    labels:
                      foo: bar
    output:
      validation: labels.foo-label-exists
      observations:
      - labels.foo-label-exists

The validatation and observations fields must specify a (Policy, Rule) pair. These observations will be printed out in the remarks section of relevant-evidence in the assessment results.

2.2 - OPA Provider

The OPA provider provides Lula with the capability to evaluate the domain against a rego policy.

Payload Expectation

The validation performed should use the form of provider with the type of opa and using the opa-spec, along with a valid domain.

Example:

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: podsvt
      resource-rule:
        group:
        version: v1
        resource: pods
        namespaces: [validation-test]
provider:
  type: opa
  opa-spec:
    rego: |                                   # Required - Rego policy used for data validation
      package validate                        # Required - Package name

      import future.keywords.every            # Optional - Any imported keywords

      validate {                              # Required - Rule Name for evaluation - "validate" is the only supported rule
        every pod in input.podsvt {
          podLabel == "bar"
        }
      }

Optionally, an output can be specified in the opa-spec. Currently, the default validation allowance/denial is given by validate.validate, which is really of the structure <package-name>.<json-path-to-boolean-variable>. If you have a desired alternative validation boolean variable, as well as additional observations to include, an output can be added such as:

domain: 
  type: kubernetes
  kubernetes-spec:
    resource-rules: 
    - group: 
      version: v1 
      resource: pods
      namespaces: [validation-test] 
provider:
  type: opa
  opa-spec:
    rego: |
      package mypackage 

      result {
        input.kind == "Pod"
        podLabel := input.metadata.labels.foo
        podLabel == "bar"
      }
      test := "my test string"
    output:
      validation: mypackage.result
      observations:
      - validate.test

The validatation field must specify a json path that resolves to a boolean value. The observations array currently only support variables that resolve as strings. These observations will be printed out in the remarks section of relevant-evidence in the assessment results.

Policy Creation

The required structure for writing a validation in rego for Lula to validate is as follows:

rego: |
  package validate 

  validate {

  }

This structure can be utilized to evaluate an expression directly:

rego: |
  package validate 

  validate {
    input.kind == "Pod"
    podLabel := input.metadata.labels.foo
    podLabel == "bar"
  }

The expression can also use multiple rule bodies as such:

rego: |
  package validate

  foolabel {
    input.kind == "Pod"
    podLabel := input.metadata.labels.foo
    podLabel == "bar"
  }

  validate {
    foolabel
  }

[!IMPORTANT] package validate and validate are required package and rule for Lula use currently when an output.validation value has not been set.

3 - Version Specification

In cases where a specific version of Lula is desired, either for typing constraints or desired functionality, a lula-version property is recognized in the description (component-definition.back-matter.resources[_]):

- uuid: 88AB3470-B96B-4D7C-BC36-02BF9563C46C
  remarks: >-
    No outputs in payload
  description: |
    lula-version: ">=0.0.2"
    domain: 
      type: kubernetes
      kubernetes-spec:
        resources:
        - name: podsvt
          resource-rule:
            group:
            version: v1
            resource: pods
            namespaces: [validation-test]
    provider:
      type: opa
      opa-spec:
        rego: |                                   
          package validate

          import future.keywords.every

          validate { 
            every pod in input.podsvt {
              podLabel == "bar"
            }
          }

If included, the lula-version must be a string and should indicate the version constraints desired, if any. Our implementation uses Hashicorp’s go-version library, and constraints should follow their conventions.

If an invalid string is passed or the current Lula version does not meet version constraints, the implementation will automatically be marked “not-satisfied” and a remark will be created in the Assessment Report detailing the rationale.