2021-06-18 13:34:01 +00:00
# convention 4 kubernetes: c4k-common
2021-06-18 14:58:06 +00:00
[](https://clojars.org/org.domaindrivenarchitecture/c4k-common-clj) [](https://clojars.org/org.domaindrivenarchitecture/c4k-common-cljs) [](https://gitlab.com/domaindrivenarchitecture/c4k-common/-/commits/master)
2021-06-18 13:34:01 +00:00
[<img src="https://domaindrivenarchitecture.org/img/delta-chat.svg" width=20 alt="DeltaChat"> chat over e-mail ](mailto:buero@meissa-gmbh.de?subject=community-chat ) | [<img src="https://meissa-gmbh.de/img/community/Mastodon_Logotype.svg" width=20 alt="team@social.meissa-gmbh.de"> team@social.meissa-gmbh.de ](https://social.meissa-gmbh.de/@team ) | [Website & Blog ](https://domaindrivenarchitecture.org )
2024-02-17 13:39:20 +00:00
## Rationale
2021-06-18 13:34:01 +00:00
2024-02-17 13:39:20 +00:00
There are many comparable solutions for creating c4k deployments like `helm` or `kustomize` .
`kustomize` is great to manage your k8s manifests by splitting huge files into handy parts.
`helm` is great because of its large community.
2022-12-23 15:02:30 +00:00
2024-02-17 13:39:20 +00:00
Why do we need another one? Why do you continue the reading here?
2022-12-23 15:02:30 +00:00
2024-02-26 14:37:25 +00:00
We combine the simplicity of `kustomize` with the ability to do real programming like software developers would do.
2022-12-23 15:02:30 +00:00
2024-02-17 13:39:20 +00:00
Following the principle
"Use programming language for programming"
2024-02-26 14:37:25 +00:00
we clearly enjoy writing Kubernetes manifests with Clojure. In comparison to helms templating, things such as business logic, conventions, input validation, versions, dependencies and reuse are much easier and much more reliable to implement with c4k.
2024-02-17 13:39:20 +00:00
By the way, c4k means "convention for kubernetes".
### Features
c4k-common supports the following use cases:
2024-02-20 17:15:17 +00:00
- [convention 4 kubernetes: c4k-common ](#convention-4-kubernetes-c4k-common )
- [Rationale ](#rationale )
- [Features ](#features )
2024-02-27 08:58:31 +00:00
- [Target CLI and Web Frontend ](#target-cli-and-web-frontend )
- [Separate Configuration From Credentials ](#separate-configuration-from-credentials )
2024-02-20 17:15:17 +00:00
- [Input as EDN or Yaml ](#input-as-edn-or-yaml )
2024-02-27 08:58:31 +00:00
- [Inline k8s Resources for Versioning \& Dependencies ](#inline-k8s-resources-for-versioning--dependencies )
2024-03-06 09:39:40 +00:00
- [Work on Structured Data Instead of Flat Templating ](#work-on-structured-data-instead-of-flat-templating )
2024-02-27 08:58:31 +00:00
- [Validate Your Inputs ](#validate-your-inputs )
2024-02-20 17:15:17 +00:00
- [Namespaces ](#namespaces )
- [Ingress ](#ingress )
- [Postgres Database ](#postgres-database )
2024-02-27 08:58:31 +00:00
- [Monitoring With Grafana Cloud ](#monitoring-with-grafana-cloud )
2024-02-20 17:15:17 +00:00
- [Refactoring \& Module Overview ](#refactoring--module-overview )
2024-02-27 08:58:31 +00:00
- [Development \& Mirrors ](#development--mirrors )
2024-02-20 17:15:17 +00:00
- [License ](#license )
2024-02-27 08:58:31 +00:00
#### Target CLI and Web Frontend
2022-12-23 15:02:30 +00:00
2024-02-27 11:08:21 +00:00
To create your own c4k module set up your cli analogous to the following:
2022-12-23 15:02:30 +00:00
```clojure
2024-02-17 13:39:20 +00:00
(defn -main [& cmd-args]
(uberjar/main-common
"c4k-forgejo" ;; name of your app
core/config? ;; schema for config validation
core/auth? ;; schema for credential validation
core/config-defaults ;; want to set default values?
core/k8s-objects ;; the function generate the k8s manifest
cmd-args ;; command line arguments given
))
```
The full example can be found here: https://repo.prod.meissa.de/meissa/c4k-forgejo/src/branch/main/src/main/clj/dda/c4k_forgejo/uberjar.clj
2024-02-27 11:08:21 +00:00
With c4k instead of using cli to generate manifests, you can also generate your manifests via web-application (using page local js without server interaction)
2024-02-17 13:39:20 +00:00
```html
2024-02-26 14:37:25 +00:00
< html >
2024-02-17 13:39:20 +00:00
< head >
2024-02-26 14:50:44 +00:00
< link href = "bootstrap.min.css" rel = "stylesheet" type = "text/css" / >
2024-02-17 13:39:20 +00:00
< / head >
< body >
< div id = "c4k-content" > < / div >
< script src = "js/main.js" > < / script >
< / body >
< / html >
```
[](https://domaindrivenarchitecture.org/pages/dda-provision/c4k-forgejo/)
See: https://repo.prod.meissa.de/meissa/c4k-forgejo/src/branch/main/public/index.html
and: https://repo.prod.meissa.de/meissa/c4k-forgejo/src/branch/main/src/main/cljs/dda/c4k_forgejo/browser.cljs
2022-12-23 15:02:30 +00:00
2024-02-27 08:58:31 +00:00
#### Separate Configuration From Credentials
2024-02-17 13:39:20 +00:00
2024-02-27 11:08:21 +00:00
We think it is a good idea to have credentials separated from configuration. All our functions, cli and frontend are following this principle. Thus, for executing separated config and authentication infos have to be provided.
2022-12-23 15:02:30 +00:00
2024-02-27 08:58:31 +00:00
The following command line yields the resulting k8s manifests in `k8s-manifest.yaml` :
```bash
java -jar c4k-common.jar config.edn auth.edn > k8s-manifest.yaml
```
Using the tool `jarwrapper` the command line can even be shortened to:
2022-12-23 15:02:30 +00:00
```bash
2024-02-17 13:39:20 +00:00
c4k-common config.edn auth.edn > k8s-manifest.yaml
2022-12-23 15:02:30 +00:00
```
2024-02-17 13:39:20 +00:00
#### Input as EDN or Yaml
2021-06-18 13:34:01 +00:00
2024-02-26 14:37:25 +00:00
c4k-common supports yaml and edn as format for all of its resources (input and output).
Hence, the following command line will also work:
2024-02-17 13:39:20 +00:00
```bash
c4k-common config.yaml auth.yaml > k8s-manifest.yaml
```
2021-06-18 13:34:01 +00:00
2024-02-27 08:58:31 +00:00
#### Inline k8s Resources for Versioning & Dependencies
2021-06-18 13:34:01 +00:00
2024-02-17 13:39:20 +00:00
We inline all resources used in our libraries & applications. You can generate k8s manifests everywhere without additional external dependencies.
2021-06-18 13:34:01 +00:00
2024-02-17 13:39:20 +00:00
In case of
* java: Resources are included in the jar-file out of the box (see https://repo.prod.meissa.de/meissa/c4k-forgejo/src/branch/main/project.clj#L13).
* js: With a slim macro call we inline resources to the resulting js file (see https://repo.prod.meissa.de/meissa/c4k-forgejo/src/branch/main/src/main/cljc/dda/c4k_forgejo/forgejo.cljc#L72-L74)
2024-02-26 14:37:25 +00:00
* native: On native builds we also inline resources (see https://repo.prod.meissa.de/meissa/c4k-forgejo/src/branch/main/build.py#L126)
2021-07-09 14:39:36 +00:00
2024-02-27 08:58:31 +00:00
#### Work on Structured Data Instead of Flat Templating
2021-07-09 14:39:36 +00:00
2024-02-26 14:37:25 +00:00
To keep things simple, we also do templating. But we convert given k8s resources to structured data.
2024-02-17 13:39:20 +00:00
This allows us to have more control and do unit tests:
k8s-resource:
```yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: ratelimit
spec:
rateLimit:
average: AVG
burst: BRS
```
Replace values:
2021-09-08 11:03:43 +00:00
2024-02-17 13:39:20 +00:00
```clojure
(defn-spec generate-rate-limit-middleware pred/map-or-seq?
[config rate-limit-config?]
(let [{:keys [max-rate max-concurrent-requests]} config]
(->
(yaml/load-as-edn "forgejo/middleware-ratelimit.yaml")
(cm/replace-key-value :average max-rate)
(cm/replace-key-value :burst max-concurrent-requests))))
```
Have a unit-test:
```clojure
(deftest should-generate-middleware-ratelimit
(is (= {:apiVersion "traefik.containo.us/v1alpha1",
:kind "Middleware",
:metadata {:name "ratelimit"},
:spec {:rateLimit {:average 10, :burst 5}}}
(cut/generate-rate-limit-middleware {:max-rate 10, :max-concurrent-requests 5}))))
```
2024-02-27 08:58:31 +00:00
#### Validate Your Inputs
2024-02-17 13:39:20 +00:00
2024-02-26 14:37:25 +00:00
Have you recognized the `defn-spec` macro above? We use allover validation, e.g.
2024-02-17 13:39:20 +00:00
```clojure
(def rate-limit-config? (s/keys :req-un [::max-rate
::max-concurrent-requests]))
(defn-spec generate-rate-limit-middleware pred/map-or-seq?
[config rate-limit-config?]
...)
```
2024-02-20 17:15:17 +00:00
#### Namespaces
We support namespaces for ingress & postgres (monitoring lives in it's own namespace `monitoring` ).
```clojure
2024-02-27 10:09:51 +00:00
(dda.c4k-common.namespace/generate {:namespace "myapp"})
```
yields:
```clojure
[{:apiVersion "v1"
:kind "Namespace"
:metadata {:name "myapp"}}]
```
2024-02-27 10:59:00 +00:00
which renders to:
2024-02-27 10:09:51 +00:00
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: myapp
2024-02-20 17:15:17 +00:00
```
2024-02-17 13:39:20 +00:00
#### Ingress
2024-02-26 14:37:25 +00:00
In most cases we use `generate-ingress-and-cert` which generates an ingress in combination with letsencrypt cert for a named service.
2024-02-17 13:39:20 +00:00
```clojure
2024-02-27 10:09:51 +00:00
(dda.c4k-common.ingress/generate-ingress-and-cert
{:fqdns ["test.jit.si"]
:service-name "web"
:service-port 80})
```
yields:
```clojure
[{:apiVersion "cert-manager.io/v1",
:kind "Certificate",
...
:spec
{:secretName "web",
:commonName "test.jit.si",
:duration "2160h",
:renewBefore "720h",
:dnsNames ["test.jit.si"],
:issuerRef {:name "staging", :kind "ClusterIssuer"}}}
{:apiVersion "networking.k8s.io/v1",
:kind "Ingress",
...
:spec
{:tls [{:hosts ["test.jit.si"], :secretName "web"}],
:rules
[{:host "test.jit.si",
:http {:paths [{:path "/",
:pathType "Prefix",
:backend
{:service {:name "web",
:port {:number 80}}}}]}}]}}]
```
2024-02-27 10:59:00 +00:00
which renders to:
2024-02-27 10:09:51 +00:00
```yaml
apiVersion: cert-manager.io/v1
kind: Certificate
...
spec:
secretName: web
commonName: test.jit.si
duration: 2160h
renewBefore: 720h
dnsNames:
- test.jit.si
issuerRef:
name: staging
kind: ClusterIssuer
---
apiVersion: networking.k8s.io/v1
kind: Ingress
...
spec:
tls:
- hosts:
- test.jit.si
secretName: web
rules:
- host: test.jit.si
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: web
port:
number: 80
2024-02-17 13:39:20 +00:00
```
#### Postgres Database
If your application needs a database, we often use postgres:
```clojure
2024-02-27 10:32:17 +00:00
(cut/generate-deployment {:postgres-image "postgres:16"})
```
yields:
```clojure
{:apiVersion "apps/v1",
:kind "Deployment",
...
:spec
{:selector {:matchLabels {:app "postgresql"}},
:strategy {:type "Recreate"},
:template
{:metadata {:labels {:app "postgresql"}},
:spec
{:containers
[{:image "postgres:16",
:name "postgresql",
:env
[{:name "POSTGRES_USER", :valueFrom {:secretKeyRef {:name "postgres-secret", :key "postgres-user"}}}
{:name "POSTGRES_PASSWORD", :valueFrom {:secretKeyRef {:name "postgres-secret", :key "postgres-password"}}}
{:name "POSTGRES_DB", :valueFrom {:configMapKeyRef {:name "postgres-config", :key "postgres-db"}}}],
:ports [{:containerPort 5432, :name "postgresql"}],
:volumeMounts
[...],
:volumes
[...]}}}}
```
2024-02-27 10:59:00 +00:00
which renders to:
2024-02-27 10:32:17 +00:00
```yaml
apiVersion: apps/v1
kind: Deployment
...
spec:
selector:
matchLabels:
app: postgresql
strategy:
type: Recreate
template:
metadata:
labels:
app: postgresql
spec:
containers:
- image: postgres:16
name: postgresql
env:
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgres-secret
key: postgres-user
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-secret
key: postgres-password
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: postgres-config
key: postgres-db
ports:
- containerPort: 5432
name: postgresql
volumeMounts:
...
volumes:
...
2024-02-17 13:39:20 +00:00
```
We optimized our db installation to run between 2Gb anf 16Gb Ram usage.
2024-02-27 08:58:31 +00:00
#### Monitoring With Grafana Cloud
2024-02-17 13:39:20 +00:00
With minimal config of
```clojure
(def conf
{:k3s-cluster-name "your-cluster-name"
:k3s-cluster-stage :prod
:grafana-cloud-url "your-url"})
(def auth
{:grafana-cloud-user "user"
:grafana-cloud-password "password"})
(monitoring/generate conf auth)
```
2024-02-26 14:37:25 +00:00
You can connect your application to grafana cloud.
2024-02-17 13:39:20 +00:00
## Refactoring & Module Overview
2022-11-18 11:18:26 +00:00
2024-02-21 10:02:39 +00:00
| Module | Version | [common load-as-edn][edn1] | [groups for webview][bgrp1] | [use common ingress][ing1] | [use common monitoring][mon1] | [validate examples][val1] | [ci with pyb][cipyb] | [inline-macro to load resources][macro] | [native build][native] | namespaces |
2024-02-23 15:48:30 +00:00
| ------------- |---------|:--------------------------:|:---------------------------:|:--------------------------:|:-----------------------------:|:-------------------------:|:--------------------:|:---------------------------------------:|:----------------------:|:----------:|
2024-02-21 10:02:39 +00:00
| c4k-keycloak | 0.2 | x | x | x | x | x | | | | |
2024-02-23 15:48:30 +00:00
| c4k-taiga | 0.1 | x | x | x | x | x | x | x | x | |
2024-05-31 15:06:56 +00:00
| c4k-nextcloud | 10.2 | x | x | x | x | x | x | x | x | x |
2024-06-07 16:07:53 +00:00
| c4k-jitsi | 2.1 | x | x | x | x | x | x | x | x | |
2024-02-21 10:02:39 +00:00
| c4k-forgejo | 3.0 | x | x | x | x | x | x | x | x | |
| c4k-shynet | 1.0 | | | | | | | | | |
2024-03-06 09:39:40 +00:00
| c4k-website | 2.0 | x | x | x | x | x | x | x | x | |
2023-07-22 08:47:01 +00:00
2022-11-09 10:46:22 +00:00
[edn1]: https://gitlab.com/domaindrivenarchitecture/c4k-website/-/merge_requests/1
2022-12-28 19:15:20 +00:00
[ing1]: https://repo.prod.meissa.de/meissa/c4k-jitsi/commit/214aa41c28662fbf7a49998e17404e7ac9216430
[bgrp1]: https://repo.prod.meissa.de/meissa/c4k-jitsi/commit/7ea442adaef727d5b48b242fd0baaaf51902d06e
[mon1]: https://repo.prod.meissa.de/meissa/c4k-jitsi/commit/19e580188ea56ea26ff3a0bfb08ca428b881ad9a
[val1]: https://repo.prod.meissa.de/meissa/c4k-jitsi/commit/5f08a108072569473463fb8f19150a12e564e54f
2023-07-22 08:47:01 +00:00
[repo1]: https://repo.prod.meissa.de/meissa/c4k-forgejo/commit/e9ee6136f3347d5fccefa6b5b4a02d30c4dc42e1
[cipyb]: https://gitlab.com/domaindrivenarchitecture/c4k-jitsi/-/merge_requests/1
2023-11-10 12:30:18 +00:00
[macro]: https://repo.prod.meissa.de/meissa/c4k-jitsi/commit/61d05ceedb6dcbc6bb96b96fe6f03598e2878195
2024-02-17 11:18:10 +00:00
[native]: https://repo.prod.meissa.de/meissa/c4k-forgejo/pulls/4/files
2021-10-08 09:41:46 +00:00
2024-02-27 08:58:31 +00:00
## Development & Mirrors
2023-07-28 11:55:55 +00:00
Development happens at: https://repo.prod.meissa.de/meissa/c4k-common
Mirrors are:
* https://gitlab.com/domaindrivenarchitecture/c4k-common (issues and PR)
* https://github.com/DomainDrivenArchitecture/c4k-common
For more details about our repository model see: https://repo.prod.meissa.de/meissa/federate-your-repos
2021-06-18 13:34:01 +00:00
## License
2024-02-17 13:39:20 +00:00
Copyright © 2022, 2023, 2024 meissa GmbH
2021-06-18 13:34:01 +00:00
Licensed under the [Apache License, Version 2.0 ](LICENSE ) (the "License")
2023-07-28 11:55:55 +00:00
Pls. find licenses of our subcomponents [here ](doc/SUBCOMPONENT_LICENSE )
