+ - 0:00:00
Notes for current slide
Notes for next slide

Using Jsonnet mixins for configurations

anarcher@gmail.com

2019-08-20

1 / 22

jsonnet

A data templating language for app and tool developers.

A simple extension of JSON.

2 / 22

3 / 22

Jsonnet Demo

4 / 22

K8S Deployment example: helm

apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: {{ template "nginx.fullname" . }}
labels:
app: {{ template "nginx.name" . }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
app: {{ template "nginx.name" . }}
template:
metadata:
labels:
app: {{ template "nginx.name" . }}
spec:
containers:
- name: {{ template "nginx.name" . }}
image: {{ Values.Image }}
ports:
- containerPort: 80
5 / 22

K8S Deployment example: jsonnet

nginx-base.jsonnet

local deploy = import "kube-deployment.libsonnet";
deploy + {
name: "my-nginx",
container: {
image: "nginx:1.12",
},
}
$kubectl apply -h
Apply a configuration to a resource by filename or stdin. The resource name must be specified. This resource will be
created if it doesn't exist yet. To use 'apply', always create the resource initially with either 'apply' or 'create
--save-config'.
JSON and YAML formats are accepted.

GOTO:#11

6 / 22

K8S Deployment example: jsonnet

kube-deployment.libsonnet

{
name:: error "name is required",
container:: error "container is required",
apiVersion: "apps/v1beta1",
kind: "Deployment",
metadata: {
name: $.name,
labels: { app: $.name },
},
spec: {
selector: { matchLabels: $.metadata.labels },
template: {
metadata: { labels: $.metadata.labels },
spec: {
containers: [
$.container { name: $.name },
],
},
},
},
}
7 / 22

References

  • self refers to the current object.
  • $ refers to the outer-most object.
  • ['foo'] looks up a field.
  • .f can be used if the field name is an identifier.

Using Jsonnet with Kubernetes

# View all available commands
ks --help
# Create an app directory (and reference your cluster with $KUBECONFIG)
KUBECONFIG="${KUBECONFIG:-$HOME/.kube/config}"
ks init ks-example # Add '--api-spec=version:v1.8.0' to this command if you're running Kubernetes 1.8!
cd ks-example
# Autogenerate a basic manifest
ks generate deployed-service guestbook-ui \
--image gcr.io/heptio-images/ks-guestbook-demo:0.1 \
--type ClusterIP
# Deploy the manifest to your cluster, and view results
ks apply default
kubectl get svc guestbook-ui
# Teardown (Don't worry, this only removes resources defined in your ksonnet app,
# and should not affect any other resources in your 'default' namespace)
ks delete default
9 / 22

Using Jsonnet with Prometheus

  • kube-prometheus
    • Use Prometheus to monitor Kubernetes and applications running on Kubernetes
  • Prometheus Monitoring Mixin for Kubernetes
    • A set of Grafana dashboards and Prometheus alerts for Kubernetes.
  • 2018-02 Prometheus Monitoring Mixins Design Document
    • A monitoring mixin is a package of configuration containing Prometheus alerts, Prometheus recording rules and Grafana dashboards.
    • Mixins will be maintained in version controlled repos (eg git) as a set of files. Versioning of mixins will be provided by the version control system; mixins themselves should not contain multiple versions.
    • Jsonnet offers the ability to parameterise configuration, allow for basic customisation. Furthermore, in Jsonnet one can reference another part of the data structure, reducing repetition.
  • Mixin Conceptually, Mixins provide dynamic inheritance, at runtime instead of compile time, which lets you combine them freely to modify objects or create new ones.
10 / 22

Prometheus Monitoring Mixins

mixin.libsonnet

{
_config+:: {...},
grafanaDashboards+:: {
“dashboard-name.json”: {...},
},
prometheusAlerts+:: [...],
prometheusRules+:: [...],
}

GOTO:#6

  • We’d like to suggest some standardisation of how configuration is supplied to mixins.
  • A top level _config dictionary has various parameters for substitution
  • jsonnet-bundler : A jsonnet package manager.
    jb install https://github.com/anguslees/kustomize-libsonnet
    jb install https://github.com/coreos/prometheus-operator/jsonnet/prometheus-operator
11 / 22

Prometheus Monitoring Mixins

mixin.libsonnet

local kubernetes = import "kubernetes-mixin/mixin.libsonnet";
kubernetes {
_config+:: {
kubeStateMetricsSelector: 'job="kube-state-metrics"',
cadvisorSelector: 'job="kubernetes-cadvisor"',
nodeExporterSelector: 'job="kubernetes-node-exporter"',
kubeletSelector: 'job="kubernetes-kubelet"',
grafanaK8s+:: {
dashboardNamePrefix: 'Mixin / ',
dashboardTags: ['kubernetes', 'infrastucture'],
},
},
}
$ jb init
$ jb install github.com/kubernetes-monitoring/kubernetes-mixin
12 / 22

Prometheus Monitoring Mixins

$ jsonnet -J vendor -m dashboards -e '(import "config.libsonnet").grafanaDashboards'
dashboards/kube-apiserver.json
dashboards/kube-controller-manager.json
dashboards/kube-proxy.json
dashboards/kube-scheduler.json
dashboards/kubelet.json
$ jsonnet -J vendor -S -e 'std.manifestYamlDoc((import "mixin.libsonnet").prometheusAlerts)' > alerts.yml
alerts.yml
$ jsonnet -J vendor -S -e 'std.manifestYamlDoc((import "mixin.libsonnet").prometheusRules)' >files/rules.yml
rules.yml

Next: Adding Dashboards to Grafana & Prometheus

13 / 22

kube-prometheus

$ mkdir my-kube-prometheus; cd my-kube-prometheus
$ jb init # Creates the initial/empty jsonnetfile.json
# Install the kube-prometheus dependency
$ jb install github.com/coreos/kube-prometheus/jsonnet/kube-prometheus@release-0.1 # Creates vendor/ & jsonnetfile.lock.json, and fills in jsonnetfile.json

minxin.jsonnet

local kp = (import 'kube-prometheus/kube-prometheus.libsonnet') +
{
_config+:: {
namespace: 'monitoring',
},
};
{ ['00prometheus-operator-' + name + '.json']: kp.prometheusOperator[name] for name in std.objectFields(kp.prometheusOperator) }
{ ['0prometheus-' + name + '.json']: kp.prometheus[name] for name in std.objectFields(kp.prometheus) }
$kubecfg update -J vendor/ mixin.jsonnet
15 / 22

kube-prometheus

$kubecfg update -J vendor/ mixin.jsonnet
$kubecfg show -o yaml -J vendor/ mixin.jsonnet
$jsonnet -J vendor -m manifests/ minxin.jsonnet
├── jsonnetfile.json
├── jsonnetfile.lock.json
├── manifests
├── mixin.jsonnet
└── vendor
├── etcd-mixin
├── grafana
├── grafana-builder
├── grafonnet
├── ksonnet
├── kube-prometheus
├── kubernetes-mixin
├── prometheus
└── prometheus-operator
16 / 22

ksonnet

https://ksonnet.io/

local k = import 'ksonnet/ksonnet.beta.4/k.libsonnet';
{
serviceAccount:
local serviceAccount = k.core.v1.serviceAccount;
serviceAccount.new('prometheus-' + $._config.prometheus.name) +
serviceAccount.mixin.metadata.withNamespace($._config.namespace),
}
  • Move away from ksonnet-lib since it is deprecated

    "I have created a PR to add ksonnet.beta.4 to ksonnet-lib that is based on Kubernetes 1.14 and we (as in maintainers of kube-prometheus, Promethues Operator and kubernetes-mixin) have no plans to abandon ksonnet any time soon."

17 / 22

ksonnet-lib

https://github.com/ksonnet/ksonnet-lib

local k = import "ksonnet.beta.2/k.libsonnet";
// Specify the import objects that we need
local container = k.extensions.v1beta1.deployment.mixin.spec.template.spec.containersType;
local containerPort = container.portsType;
local deployment = k.extensions.v1beta1.deployment;
local targetPort = 80;
local podLabels = {app: "nginx"};
local nginxContainer =
container.new("nginx", "nginx:1.7.9") +
container.ports(containerPort.containerPort(targetPort));
local nginxDeployment =
deployment.new("nginx-deployment", 2, nginxContainer, podLabels);
k.core.v1.list.new(nginxDeployment)
18 / 22

Jsonnet dashboard

  • Grafonnet : a simple DSL using jsonnet to generate Grafana dashboards.
local grafana = import 'grafonnet/grafana.libsonnet';
local kp = (import 'kube-prometheus/kube-prometheus.libsonnet') + {
grafanaDashboards+:: {
'my-dashboard.json':
dashboard.new('My Dashboard v2')
.addTemplate(template.datasource(
'datasource',
'prometheus',
'Prometheus',
))
.addRow(
row.new().addPanel(
graphPanel.new('My Panel', span=6, datasource='$datasource')
.addTarget(prometheus.target('vector(1)'))
),
),
}
}
19 / 22

Jsonnet dashboard

$kubecfg -J vendor/ update mixin.jsonnet
INFO Creating configmaps monitoring.grafana-dashboard-apiserver
INFO Creating configmaps monitoring.grafana-dashboard-controller-manager
INFO Creating configmaps monitoring.grafana-dashboard-k8s-resources-cluster
INFO Creating configmaps monitoring.grafana-dashboard-k8s-resources-namespace
INFO Creating configmaps monitoring.grafana-dashboard-k8s-resources-pod
INFO Creating configmaps monitoring.grafana-dashboard-k8s-resources-workload
INFO Creating configmaps monitoring.grafana-dashboard-k8s-resources-workloads-namespace
INFO Creating configmaps monitoring.grafana-dashboard-kubelet
INFO Creating configmaps monitoring.grafana-dashboard-my-dashboard
INFO Creating configmaps monitoring.grafana-dashboard-persistentvolumesusage
INFO Creating configmaps monitoring.grafana-dashboard-pods
INFO Creating configmaps monitoring.grafana-dashboard-prometheus
INFO Creating configmaps monitoring.grafana-dashboard-prometheus-remote-write
INFO Creating configmaps monitoring.grafana-dashboard-proxy
INFO Creating configmaps monitoring.grafana-dashboard-scheduler
INFO Creating configmaps monitoring.grafana-dashboard-statefulset
INFO Creating configmaps monitoring.grafana-dashboards
20 / 22

Jsonnet dashboard

$k get cm
NAME DATA AGE
grafana-dashboard-apiserver 1 26m
grafana-dashboard-controller-manager 1 26m
grafana-dashboard-k8s-resources-cluster 1 26m
grafana-dashboard-k8s-resources-namespace 1 26m
grafana-dashboard-k8s-resources-pod 1 26m
grafana-dashboard-k8s-resources-workload 1 26m
grafana-dashboard-k8s-resources-workloads-namespace 1 26m
grafana-dashboard-kubelet 1 26m
grafana-dashboard-my-dashboard 1 26m
grafana-dashboard-persistentvolumesusage 1 26m
grafana-dashboard-pods 1 26m
grafana-dashboard-prometheus 1 26m
grafana-dashboard-prometheus-remote-write 1 26m
grafana-dashboard-proxy 1 26m
grafana-dashboard-scheduler 1 26m
grafana-dashboard-statefulset 1 26m
grafana-dashboards 1 26m
21 / 22

Jsonnet dashboard

22 / 22

jsonnet

A data templating language for app and tool developers.

A simple extension of JSON.

2 / 22
Paused

Help

Keyboard shortcuts

, , Pg Up, k Go to previous slide
, , Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow