Prow as Tekton pipeline trigger

## anarcher@gmail.com
## 2019-07-06

---
class: pic

---

## What is Prow?

[Prow](https://github.com/kubernetes/test-infra/tree/master/prow) is a **Kubernetes**-based CI/CD system

**Jobs** can be triggered by various types of events and report their status to many different services.

In addition to job execution, Prow provides **GitHub automationn** in the form of policy enforcement, chat-ops via `/foo` style commands, and automatic PR merging.

Microservices architecture provides the core components that you
need and allows for **extendability and replacement**.

The Kubernetes project does a lot of testing, on the **order of 10000 jobs per day** covering everything from build and unit tests, to **end-to-end testing** on real clusters deployed from source all the way up to **~5000 node** scalability and performance tests.

- [prow.k8s.io](https://prow.k8s.io)
- [prow.istio.io](https://prow.istio.io/)
- [openshift ci](https://deck-ci.svc.ci.openshift.org/)

---

![](./prow-arch.png)

---

## Prow/horologium

Responsible for scheduling Periodic jobs

```yaml
kind: ConfigMap
metadata:
 name: config
data:
 config.yaml: |
 periodics:
 - interval: 10m
 agent: kubernetes
 name: echo-test
 spec:
 containers:
 - image: alpine
 command: ["/bin/date"]
```

---

<pre><code class="hljs remark-code" style="font-size: 70%;">apiVersion: prow.k8s.io/v1
kind: ProwJob
metadata:
  creationTimestamp: "2019-07-10T19:43:11Z"
  labels:
    created-by-prow: "true"
    prow.k8s.io/id: f379e510-a34a-11e9-a876-1afb1ac8a26e
    prow.k8s.io/job: echo-test
    prow.k8s.io/type: periodic
  name: f379e510-a34a-11e9-a876-1afb1ac8a26e
  namespace: prow
spec:
  agent: kubernetes
  cluster: default
  job: echo-test
  namespace: prow-test-pods
  pod_spec:
    containers:
    - command:
      - /bin/date
      image: alpine
      name: ""
      resources: {}
  type: periodic
status:
  build_id: "1149041418853945344"
  completionTime: "2019-07-10T19:43:47Z"
  description: Job succeeded.
  pod_name: f379e510-a34a-11e9-a876-1afb1ac8a26e
  prev_report_states:
    github-reporter: success
  startTime: "2019-07-10T19:43:11Z"
  state: success

</code></pre>

---

# horologium

![](./horologium-1.png) ![](./prow-arch.png)

---

## Prow/plank

Controller that manages job execution and lifecycle of
kubernetes jobs.

```yamla
spec:
  agent: kubernetes
  cluster: default
  context: test-pod
  job: test-pod
  namespace: prow-msshin-test-pods
  pod_spec:
    containers:
    - image: alpine
      command: ["/bin/printenv"]
status:
  build_id: "1148512931902656513"
  completionTime: "2019-07-09T08:43:45Z"
  description: Job succeeded.
  pod_name: 956c3fc4-a225-11e9-aadf-a660c8b9b0b2
  startTime: "2019-07-09T08:43:11Z"
  state: success
```

---

## Prow/deck

![](./deck-1.png)

---

## Prow/sinker

- Clean up
- Deletes completed prow jobs & pods

---

## Prow/hook

- It is a stateless server that listens for GitHub webhooks and dispatches them to the appropriate plugins. 
- Hook's plugins are used to trigger jobs, implement 'slash' commands, post to Slack, and more.
- [https://prow.k8s.io/plugins](https://prow.k8s.io/plugins)

![](./demo1.png)

---

## config.yaml

- [https://github.com/kubernetes/test-infra/blob/master/prow/jobs.md](https://github.com/kubernetes/test-infra/blob/master/prow/jobs.md)

```yml
postsubmits:
  org/repo:
  - name: bar-job         # As for periodics.
    decorate: true        # As for periodics.
    spec: {}              # As for periodics.
    max_concurrency: 10   # Run no more than this number concurrently.
    branches:             # Regexps, only run against these branches.
    - ^master$
    skip_branches:        # Regexps, do not run against these branches.
    - ^release-.*$
```

---

## plugin.yaml

```yml
plugins:
  {ORG}/{REPO}:
  - size
  - trigger
  - help
  - hold
```

[prow.k8s.io](https://prow.k8s.io/plugins?repo=kubernetes%2Fapiserver)

---

## Tekton pipeline controller in Prow

```yml
postsubmits:
  - name: hello-pipeline
    agent: tekton-pipeline
    namespace: test-pods
    labels:
      anarcher.dev/job: prow
    pipeline_run_spec:
      pipelineRef:
        name: hello
      resources:
      - name: source-repo
        resourceRef:
          name: PROW_IMPLICIT_GIT_REF
      - name: docker-image
        resourceRef:
          name: test-image
    branches:
    - ^master$
```

---

## Prow/pipeline

- Pipeline controller makes  `tekton/PipelineRun ` and `tekton/PipelineResource` (git resource references)

- reference values in `pipeline_run_spec` git resource references like `PROW_IMPLICIT_GIT_REF` will be substituted for the actual resource names by the controller before the pipelinerun is created.

```yml
resources:
- name: source-repo
  resourceRef:
    name: PROW_IMPLICIT_GIT_REF
```

---

![](./pipeline1.png)

---

## Tekton pipeline controller

- https://github.com/kubernetes/test-infra/pull/11888
- https://github.com/kubernetes/test-infra/pull/13114

---