Module
A Timoni module contains a set of CUE definitions and constraints organised into a CUE module with an opinionated file structure.
File Structure
A module consists of a collection of CUE files inside a directory with the following structure:
myapp
├── README.md
├── cue.mod
│ ├── gen # Kubernetes APIs and CRDs schemas
│ ├── pkg # Timoni APIs schemas
│ └── module.cue # Module metadata
├── templates
│ ├── config.cue # Config schema and default values
│ ├── deployment.cue # Kubernetes Deployment template
│ └── service.cue # Kubernetes Service template
├── timoni.cue # Timoni entry point
├── timoni.ignore # Timoni ignore rules
└── values.cue # Timoni values placeholder
To create a new module in the current directory:
timoni mod init myapp .
Entry point
The timoni.cue
file contains the definition of how Timoni should
validate, build and deploy a module instance.
This file is generated by timoni mod init
with the following content:
// source: myapp/timoni.cue
package main
import (
templates "timoni.sh/myapp/templates"
)
// Define the schema for the user-supplied values.
// At runtime, Timoni injects the supplied values
// and validates them according to the Config schema.
values: templates.#Config
// Define how Timoni should build, validate and
// apply the Kubernetes resources.
timoni: {
apiVersion: "v1alpha1"
// Define the instance that generates the Kubernetes resources.
// At runtime, Timoni builds the instance and validates
// the resulting resources according to their Kubernetes schema.
instance: templates.#Instance & {
// The user-supplied values are merged with the
// default values at runtime by Timoni.
config: values
// These values are injected at runtime by Timoni.
config: {
metadata: {
name: string @tag(name)
namespace: string @tag(namespace)
}
moduleVersion: string @tag(mv, var=moduleVersion)
kubeVersion: string @tag(kv, var=kubeVersion)
}
}
// Enforce minimum Kubernetes version.
kubeMinorVersion: int & >=20
kubeMinorVersion: strconv.Atoi(strings.Split(instance.config.kubeVersion, ".")[1])
// Pass the generated Kubernetes resources
// to Timoni's multi-step apply.
apply: all: [ for obj in instance.objects {obj}]
}
Kubernetes min version
At apply-time, Timoni injects the Kubernetes version from the live cluster.
To enforce a minimum supported version for your module, set a constraint for the minor
version e.g. kubeMinorVersion: int & >=20
.
To test the constraint, you can use the TIMONI_KUBE_VERSION
env var
with timoni mod vet
and timoni build
.
$ TIMONI_KUBE_VERSION=1.19.0 timoni mod vet ./myapp
validation failed: timoni.kubeMinorVersion: invalid value 19 (out of bound >=20)
Ignore
The timoni.ignore
file contains rules in the
.gitignore pattern format.
The paths matching the defined rules are excluded when publishing
the module to a container registry.
The recommended ignore patterns are:
# VCS
.git/
.gitignore
.gitmodules
.gitattributes
# Go
vendor/
go.mod
go.sum
# CUE
*_tool.cue
Values
The values.cue
file holds the default values.
Note that this file must have no imports and all values must be concrete.
// source: myapp/values.cue
values: {
message: "Hello World"
image: {
repository: "cgr.dev/chainguard/nginx"
digest: "sha256:d2b0e52d7c2e5dd9fe5266b163e14d41ed97fd380deb55a36ff17efd145549cd"
tag: "1.25.1"
}
}
The values
schema is set in the timoni.cue
file:
// source: myapp/timoni.cue
values: templates.#Config
Note that the README.md
file should contain the config values schema documentation.
Templates
The templates directory is where module authors define Kubernetes resources and their configuration schema.
Config
The schema and defaults for the user-supplied values are defined in templates/config.cue
.
Example of a minimal config for an app deployment:
// source: myapp/templates/config.cue
#Config: {
moduleVersion!: string
kubeVersion!: string
metadata: timoniv1.#Metadata & {#Version: moduleVersion}
selector: timoniv1.#Selector & {#Name: metadata.name}
image: timoniv1.#Image
imagePullPolicy: *"IfNotPresent" | string
replicas: *1 | int & >0
service: port: *80 | int & >0 & <=65535
resources?: corev1.#ResourceRequirements
}
The user-supplied values can:
- add annotations and labels to metadata
- change the service port to a different value
- set resource requirements requests and/or limits
- set the image repository, tag and digest
// source: myapp-values/values.cue
values: {
metadata: {
labels: "app.kubernetes.io/part-of": "frontend"
annontations: "my.org/owner": "web-team"
}
service: port: 8080
resources: limits: memory: "1Gi"
}
When creating an instance, Timoni unifies the user-supplied values with the defaults and sets the metadata for all Kubernetes resources to:
metadata:
name: "<instance-name>"
namespace: "<instance-namespace>"
labels:
app.kubernetes.io/name: "<instance-name>"
app.kubernetes.io/version: "<module-name>"
app.kubernetes.io/part-of: "frontend"
annotations:
my.org/owner: "web-team"
Note that app.kubernetes.io/name
and app.kubernetes.io/version
labels
are automatically generated by timoniv1.#Metadata
.
Instance
Example of defining an instance containing a Kubernetes Service and Deployment:
// source: myapp/templates/config.cue
#Instance: {
config: #Config
objects: {
svc: #Service & {_config: config}
deploy: #Deployment & {_config: config}
}
}
Kubernetes resources
Example of a Kubernetes Service template:
// source: myapp/templates/service.cue
package templates
import (
corev1 "k8s.io/api/core/v1"
)
#Service: corev1.#Service & {
_config: #Config
apiVersion: "v1"
kind: "Service"
metadata: _config.metadata
spec: corev1.#ServiceSpec & {
type: corev1.#ServiceTypeClusterIP
selector: _config.selector.labels
ports: [
{
port: _config.service.port
protocol: "TCP"
name: "http"
targetPort: name
},
]
}
}
Note that the service pod selector is automatically set to
app.kubernetes.io/name: <instance-name>
by timoniv1.#Selector
.
End-users can add their own custom labels to the selector e.g.:
// source: myapp-values/values.cue
values: {
selector: labels: {
"app.kubernetes.io/component": "auth"
"app.kubernetes.io/part-of": "frontend"
}
}
Timoni will add the custom labels to the Deployment selector, PodSpec labels and Service selector.
Controlling the apply behaviour
Timoni allows module authors to change the default apply behaviour of Kubernetes resources using the following annotations:
Annotation | Values |
---|---|
action.timoni.sh/force |
- enabled - disabled |
action.timoni.sh/one-off |
- enabled - disabled |
action.timoni.sh/prune |
- enabled - disabled |
To recreate immutable resources such as Kubernetes Jobs,
these resources can be annotated with action.timoni.sh/force: "enabled"
.
To apply resources only if they don't exist on the cluster,
these resources can be annotated with action.timoni.sh/one-off: "enabled"
.
To prevent Timoni's garbage collector from deleting certain
resources such as Kubernetes Persistent Volumes,
these resources can be annotated with action.timoni.sh/prune: "disabled"
.
Running tests with Kubernetes Jobs
Module authors can write end-to-end tests that are run by Timoni,
after the app workloads are deployed on a cluster.
Tests are defined as Kubernetes Jobs that can be placed inside the templates
directory.
Example of a test that verifies that an app is accessible from inside the cluster:
// source: myapp/templates/job.cue
#TestJob: batchv1.#Job & {
_config: #Config
apiVersion: "batch/v1"
kind: "Job"
metadata: _config.metadata
metadata: annotations: timoniv1.action.force
spec: batchv1.#JobSpec & {
template: corev1.#PodTemplateSpec & {
metadata: labels: _config.metadata.labels
let _checksum = uuid.SHA1(uuid.ns.DNS, yaml.Marshal(_config))
metadata: annotations: "timoni.sh/checksum": "\(_checksum)"
spec: {
containers: [{
name: "curl"
image: _config.test.image.reference
imagePullPolicy: _config.imagePullPolicy
command: [
"curl",
"-v",
"-m",
"5",
"\(_config.metadata.name):\(_config.service.port)",
]
}]
restartPolicy: "Never"
}
}
}
}
After the app workloads are installed and become ready, Timoni will apply the Kubernetes Jobs and wait for the created pods to run to completion. On upgrades, Timoni will delete the previous test pods and will recreate the Jobs for the current module values and version.
Test runs are idempotent, if the values or the module version doesn't change, Timoni will not create new test pods, tests are run only when a drift is detected in desired state.
A complete example of defining end-to-end tests can be found
in modules created with timoni mod init
.
Kubernetes schemas
To ensure that the Kubernetes resources defined in a module are in conformance with their OpenAPI schema, Timoni offers commands for vendoring CUE definitions generated from the Kubernetes builtin APIs and CRDs.
Kubernetes builtin APIs
The cue.mod/gen/k8s.io
directory contains the Kubernetes GA API types and their schema.
These files are automatically generated by CUE from the Kubernetes API Go packages.
cue.mod/gen/
└── k8s.io
├── api
│ ├── admission
│ ├── admissionregistration
│ ├── apps
│ ├── authentication
│ ├── authorization
│ ├── autoscaling
│ ├── batch
│ ├── certificates
│ ├── coordination
│ ├── core
│ ├── discovery
│ ├── events
│ ├── networking
│ ├── node
│ ├── policy
│ ├── rbac
│ ├── scheduling
│ └── storage
├── apiextensions-apiserver
└── apimachinery
To update the schemas to a specific Kubernetes version, run the following command from within the module root directory:
timoni mod vendor k8s -v 1.28
Kubernetes CRDs
To use 3rd-party Kubernetes APIs e.g. Prometheus Operator, you can point Timoni to a YAML file which contains the Kubernetes CRDs:
timoni mod vendor crds -f https://github.com/prometheus-operator/prometheus-operator/releases/download/v0.68.0/stripped-down-crds.yaml
Timoni generates the CUE schemas corresponding to the Kubernetes CRDs
inside the cue.mod/gen
directory:
cue.mod/gen/
└── monitoring.coreos.com
├── alertmanager
├── alertmanagerconfig
├── podmonitor
├── probe
├── prometheus
├── prometheusagent
├── prometheusrule
├── scrapeconfig
├── servicemonitor
└── thanosruler
Example of a ServiceMonitor
custom resource:
// source: myapp/templates/servicemonitor.cue
package templates
import (
promv1 "monitoring.coreos.com/servicemonitor/v1"
)
#ServiceMonitor: promv1.#ServiceMonitor & {
_config: #Config
metadata: _config.metadata
spec: {
endpoints: [{
path: "/metrics"
port: "http-metrics"
interval: "\(_config.monitoring.interval)s"
}]
namespaceSelector: matchNames: [_config.metadata.namespace]
selector: matchLabels: _config.selector.labels
}
}
Note that for Kubernetes custom resources, you don't need to specify the apiVersion
and kind
,
these fields are set by Timoni in the generated schema.