Skip to content

Module

A Timoni module contains a set of CUE definitions and constraints organised into a CUE module with an opinionated file structure.

File Structure

A module consists of a collection of CUE files inside a directory with the following structure:

myapp
├── README.md
├── cue.mod
│   ├── gen # Kubernetes APIs and CRDs schemas
│   ├── pkg # Timoni APIs schemas
│   └── module.cue # Module metadata
├── templates
│   ├── config.cue # Config schema and default values
│   ├── deployment.cue # Kubernetes Deployment template
│   └── service.cue # Kubernetes Service template
├── timoni.cue # Timoni entry point
├── timoni.ignore # Timoni ignore rules
└── values.cue # Timoni values placeholder

To create a new module in the current directory:

timoni mod init myapp .

Entry point

The timoni.cue file contains the definition of how Timoni should validate, build and deploy a module instance.

This file is generated by timoni mod init with the following content:

// source: myapp/timoni.cue

package main

import (
    templates "timoni.sh/myapp/templates"
)

// Define the schema for the user-supplied values.
// At runtime, Timoni injects the supplied values
// and validates them according to the Config schema.
values: templates.#Config

// Define how Timoni should build, validate and
// apply the Kubernetes resources.
timoni: {
    apiVersion: "v1alpha1"

    // Define the instance that generates the Kubernetes resources.
    // At runtime, Timoni builds the instance and validates
    // the resulting resources according to their Kubernetes schema.
    instance: templates.#Instance & {
        // The user-supplied values are merged with the
        // default values at runtime by Timoni.
        config: values
        // These values are injected at runtime by Timoni.
        config: {
            metadata: {
                name:      string @tag(name)
                namespace: string @tag(namespace)
            }
            moduleVersion: string @tag(mv, var=moduleVersion)
            kubeVersion:   string @tag(kv, var=kubeVersion)
        }
    }

    // Enforce minimum Kubernetes version.
    kubeMinorVersion: int & >=20
    kubeMinorVersion: strconv.Atoi(strings.Split(instance.config.kubeVersion, ".")[1])

    // Pass the generated Kubernetes resources
    // to Timoni's multi-step apply.
    apply: all: [ for obj in instance.objects {obj}]
}

Kubernetes min version

At apply-time, Timoni injects the Kubernetes version from the live cluster. To enforce a minimum supported version for your module, set a constraint for the minor version e.g. kubeMinorVersion: int & >=20.

To test the constraint, you can use the TIMONI_KUBE_VERSION env var with timoni mod vet and timoni build.

$ TIMONI_KUBE_VERSION=1.19.0 timoni mod vet ./myapp
validation failed: timoni.kubeMinorVersion: invalid value 19 (out of bound >=20)

Ignore

The timoni.ignore file contains rules in the .gitignore pattern format. The paths matching the defined rules are excluded when publishing the module to a container registry.

The recommended ignore patterns are:

# VCS
.git/
.gitignore
.gitmodules
.gitattributes

# Go
vendor/
go.mod
go.sum

# CUE
*_tool.cue

Values

The values.cue file holds the default values. Note that this file must have no imports and all values must be concrete.

// source: myapp/values.cue

values: {
    message: "Hello World"
    image: {
        repository: "cgr.dev/chainguard/nginx"
        digest:     "sha256:d2b0e52d7c2e5dd9fe5266b163e14d41ed97fd380deb55a36ff17efd145549cd"
        tag:        "1.25.1"
    }
}

The values schema is set in the timoni.cue file:

// source: myapp/timoni.cue

values: templates.#Config

Note that the README.md file should contain the config values schema documentation.

Templates

The templates directory is where module authors define Kubernetes resources and their configuration schema.

Config

The schema and defaults for the user-supplied values are defined in templates/config.cue.

Example of a minimal config for an app deployment:

// source: myapp/templates/config.cue

#Config: {
    moduleVersion!: string
    kubeVersion!:   string

    metadata: timoniv1.#Metadata & {#Version: moduleVersion}
    selector: timoniv1.#Selector & {#Name:    metadata.name}

    image:           timoniv1.#Image
    imagePullPolicy: *"IfNotPresent" | string

    replicas: *1 | int & >0
    service: port: *80 | int & >0 & <=65535
    resources?: corev1.#ResourceRequirements
}

The user-supplied values can:

  • add annotations and labels to metadata
  • change the service port to a different value
  • set resource requirements requests and/or limits
  • set the image repository, tag and digest
// source: myapp-values/values.cue

values: {
    metadata: {
        labels: "app.kubernetes.io/part-of": "frontend"
        annontations: "my.org/owner":        "web-team"
    }
    service: port: 8080
    resources: limits: memory: "1Gi"
}

When creating an instance, Timoni unifies the user-supplied values with the defaults and sets the metadata for all Kubernetes resources to:

metadata:
  name: "<instance-name>"
  namespace: "<instance-namespace>"
  labels:
    app.kubernetes.io/name: "<instance-name>"
    app.kubernetes.io/version: "<module-name>"
    app.kubernetes.io/part-of: "frontend"
  annotations:
    my.org/owner: "web-team"

Note that app.kubernetes.io/name and app.kubernetes.io/version labels are automatically generated by timoniv1.#Metadata.

Instance

Example of defining an instance containing a Kubernetes Service and Deployment:

// source: myapp/templates/config.cue

#Instance: {
    config: #Config

    objects: {
        svc:    #Service & {_config:        config}
        deploy: #Deployment & {_config:     config}
    }
}

Kubernetes resources

Example of a Kubernetes Service template:

// source: myapp/templates/service.cue

package templates

import (
    corev1 "k8s.io/api/core/v1"
)

#Service: corev1.#Service & {
    _config:    #Config
    apiVersion: "v1"
    kind:       "Service"
    metadata:   _config.metadata
    spec:       corev1.#ServiceSpec & {
        type:     corev1.#ServiceTypeClusterIP
        selector: _config.selector.labels
        ports: [
            {
                port:       _config.service.port
                protocol:   "TCP"
                name:       "http"
                targetPort: name
            },
        ]
    }
}

Note that the service pod selector is automatically set to app.kubernetes.io/name: <instance-name> by timoniv1.#Selector.

End-users can add their own custom labels to the selector e.g.:

// source: myapp-values/values.cue

values: {
    selector: labels: {
        "app.kubernetes.io/component": "auth"
        "app.kubernetes.io/part-of":   "frontend"
    }
}

Timoni will add the custom labels to the Deployment selector, PodSpec labels and Service selector.

Controlling the apply behaviour

Timoni allows module authors to change the default apply behaviour of Kubernetes resources using the following annotations:

Annotation Values
action.timoni.sh/force - enabled
- disabled
action.timoni.sh/one-off - enabled
- disabled
action.timoni.sh/prune - enabled
- disabled

To recreate immutable resources such as Kubernetes Jobs, these resources can be annotated with action.timoni.sh/force: "enabled".

To apply resources only if they don't exist on the cluster, these resources can be annotated with action.timoni.sh/one-off: "enabled".

To prevent Timoni's garbage collector from deleting certain resources such as Kubernetes Persistent Volumes, these resources can be annotated with action.timoni.sh/prune: "disabled".

Running tests with Kubernetes Jobs

Module authors can write end-to-end tests that are run by Timoni, after the app workloads are deployed on a cluster. Tests are defined as Kubernetes Jobs that can be placed inside the templates directory.

Example of a test that verifies that an app is accessible from inside the cluster:

// source: myapp/templates/job.cue

#TestJob: batchv1.#Job & {
    _config:    #Config
    apiVersion: "batch/v1"
    kind:       "Job"
    metadata:   _config.metadata
    metadata: annotations: timoniv1.action.force
    spec: batchv1.#JobSpec & {
        template: corev1.#PodTemplateSpec & {
            metadata: labels: _config.metadata.labels
            let _checksum = uuid.SHA1(uuid.ns.DNS, yaml.Marshal(_config))
            metadata: annotations: "timoni.sh/checksum": "\(_checksum)"
            spec: {
                containers: [{
                    name:            "curl"
                    image:           _config.test.image.reference
                    imagePullPolicy: _config.imagePullPolicy
                    command: [
                        "curl",
                        "-v",
                        "-m",
                        "5",
                        "\(_config.metadata.name):\(_config.service.port)",
                    ]
                }]
                restartPolicy: "Never"
            }
        }
    }
}

After the app workloads are installed and become ready, Timoni will apply the Kubernetes Jobs and wait for the created pods to run to completion. On upgrades, Timoni will delete the previous test pods and will recreate the Jobs for the current module values and version.

Test runs are idempotent, if the values or the module version doesn't change, Timoni will not create new test pods, tests are run only when a drift is detected in desired state.

A complete example of defining end-to-end tests can be found in modules created with timoni mod init.

Kubernetes schemas

To ensure that the Kubernetes resources defined in a module are in conformance with their OpenAPI schema, Timoni offers commands for vendoring CUE definitions generated from the Kubernetes builtin APIs and CRDs.

Kubernetes builtin APIs

The cue.mod/gen/k8s.io directory contains the Kubernetes GA API types and their schema. These files are automatically generated by CUE from the Kubernetes API Go packages.

cue.mod/gen/
└── k8s.io
    ├── api
    │   ├── admission
    │   ├── admissionregistration
    │   ├── apps
    │   ├── authentication
    │   ├── authorization
    │   ├── autoscaling
    │   ├── batch
    │   ├── certificates
    │   ├── coordination
    │   ├── core
    │   ├── discovery
    │   ├── events
    │   ├── networking
    │   ├── node
    │   ├── policy
    │   ├── rbac
    │   ├── scheduling
    │   └── storage
    ├── apiextensions-apiserver
    └── apimachinery

To update the schemas to a specific Kubernetes version, run the following command from within the module root directory:

timoni mod vendor k8s -v 1.28

Kubernetes CRDs

To use 3rd-party Kubernetes APIs e.g. Prometheus Operator, you can point Timoni to a YAML file which contains the Kubernetes CRDs:

timoni mod vendor crds -f https://github.com/prometheus-operator/prometheus-operator/releases/download/v0.68.0/stripped-down-crds.yaml

Timoni generates the CUE schemas corresponding to the Kubernetes CRDs inside the cue.mod/gen directory:

cue.mod/gen/
└── monitoring.coreos.com
    ├── alertmanager
    ├── alertmanagerconfig
    ├── podmonitor
    ├── probe
    ├── prometheus
    ├── prometheusagent
    ├── prometheusrule
    ├── scrapeconfig
    ├── servicemonitor
    └── thanosruler

Example of a ServiceMonitor custom resource:

// source: myapp/templates/servicemonitor.cue

package templates

import (
    promv1 "monitoring.coreos.com/servicemonitor/v1"
)

#ServiceMonitor: promv1.#ServiceMonitor & {
    _config:    #Config
    metadata:   _config.metadata
    spec: {
        endpoints: [{
            path:     "/metrics"
            port:     "http-metrics"
            interval: "\(_config.monitoring.interval)s"
        }]
        namespaceSelector: matchNames: [_config.metadata.namespace]
        selector: matchLabels: _config.selector.labels
    }
}

Note that for Kubernetes custom resources, you don't need to specify the apiVersion and kind, these fields are set by Timoni in the generated schema.