meissa/c4k-forgejo
6
0
Fork 0
Code Issues Pull requests Projects Releases 27 Packages Wiki Activity

Merge branch 'main' into forgejo-upgrade

Browse source
This commit is contained in:
Michael Jerger 2024-08-09 17:08:06 +02:00
commit a7c298a824
12 changed files with 115 additions and 91 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -46,8 +46,8 @@ Development happens at: https://repo.prod.meissa.de/meissa/c4k-forgejo
Mirrors are:
* https://codeberg.org/meissa/c4k-forgejo (Issues and PR)
* https://gitlab.com/domaindrivenarchitecture/c4k-forgejo (CI)
* https://codeberg.org/meissa/c4k-forgejo (issues and PR)
* https://github.com/DomainDrivenArchitecture/c4k-forgejo
For more details about our repository model see: https://repo.prod.meissa.de/meissa/federate-your-repos
@ -55,6 +55,6 @@ For more details about our repository model see: https://repo.prod.meissa.de/mei
## License
Copyright © 2024 meissa GmbH
Copyright © 2023, 2024 meissa GmbH
Licensed under the [Apache License, Version 2.0](LICENSE) (the "License")
Pls. find licenses of our subcomponents [here](doc/SUBCOMPONENT_LICENSE)

28
doc/Runbook_UpgradeFrom1.19To7.0.5.md
View file

@ -6,6 +6,7 @@
* 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0
* 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0
* 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0
* 8.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#8-0-0
## Preparations
@ -17,8 +18,8 @@
### Create 2nd Repo Prod Server
1. Terraform Preparations for 2nd Server: TODO
1. Install c4k-forgejo Version TODO
with config `"forgejo-image-version-overwrite": "1.19.3-0"`
1. Install c4k-forgejo Version `3.5.0`!
with config `"forgejo-image-version-overwrite": "1.19.3-0"` (in server-setup)
1. Stop Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Disable Backup Cron: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'`
1. Scale up Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1`
@ -36,7 +37,7 @@
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `1.20.1-0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors
1. Check for errors: `k logs -n forgejo forgejo-...`
## Upgrade to 1.21.1-0
@ -44,7 +45,7 @@
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `1.21.1-0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors
1. Check for errors: `k logs -n forgejo forgejo-...`
1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`.
## Upgrade to 7.0.0
@ -55,17 +56,24 @@
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `7.0.0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors
1. Check for errors: `k logs -n forgejo forgejo-...`
## Upgrade to 7.0.5 (no breaking changes)
TODO: Upgrade to 8.0.0 instead after Release!
## Upgrade to 8.0.0 (no relevant breaking changes)
1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `7.0.5` with `k edit -n forgejo deployment forgejo`
1. Set version to `8.0.0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors
1. Check for errors: `k logs -n forgejo forgejo-...`
## Enable Federation
1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Adjust configmap: `k edit -n forgejo cm forgejo-env`
1. Change `FORGEJO__federation__ENABLED: "false"` TO `FORGEJO__federation__ENABLED: "true"`
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors: `k logs -n forgejo forgejo-...`
## Post Work

2
infrastructure/backup/build.py
View file

@ -6,7 +6,7 @@ from ddadevops import *
name = "c4k-forgejo"
MODULE = "backup"
PROJECT_ROOT_PATH = "../.."
version = "3.4.5-dev"
version = "3.5.1-dev"
@init

2
infrastructure/federated/build.py
View file

@ -6,7 +6,7 @@ from ddadevops import *
name = 'c4k-forgejo'
MODULE = 'federated'
PROJECT_ROOT_PATH = '../..'
version = "3.4.5-dev"
version = "3.5.1-dev"
@init
def initialize(project):

2
package.json
View file

@ -2,7 +2,7 @@
"name": "c4k-forgejo",
"description": "Generate c4k yaml for a forgejo deployment.",
"author": "meissa GmbH",
"version": "3.4.5-SNAPSHOT",
"version": "3.5.1-SNAPSHOT",
"homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme",
"repository": "https://www.npmjs.com/package/c4k-forgejo",
"license": "APACHE2",

7
project.clj
View file

@ -1,16 +1,17 @@
(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.5-SNAPSHOT"
(defproject org.domaindrivenarchitecture/c4k-forgejo "3.5.1-SNAPSHOT"
:description "forgejo c4k-installation package"
:url "https://domaindrivenarchitecture.org"
:license {:name "Apache License, Version 2.0"
:url "https://www.apache.org/licenses/LICENSE-2.0.html"}
:dependencies [[org.clojure/clojure "1.11.3" :scope "provided"]
[org.clojure/tools.reader "1.4.2"]
[org.domaindrivenarchitecture/c4k-common-clj "6.4.1"]
[org.domaindrivenarchitecture/c4k-common-clj "7.0.0"]
[hickory "0.7.1" :exclusions [viebel/codox-klipse-theme]]]
:target-path "target/%s/"
:source-paths ["src/main/cljc"
"src/main/clj"]
:resource-paths ["src/main/resources"]
:resource-paths ["src/main/resources"
"project.clj"]
:repositories [["snapshots" :clojars]
["releases" :clojars]]
:deploy-repositories [["snapshots" {:sign-releases false :url "https://clojars.org/repo"}]

2
shadow-cljs.edn
View file

@ -4,7 +4,7 @@
"src/test/cljc"
"src/test/cljs"
"src/test/resources"]
:dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "6.4.1"]
:dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "7.0.0"]
[hickory "0.7.1"]]
:builds {:frontend {:target :browser
:modules {:main {:init-fn dda.c4k-forgejo.browser/init}}

5
src/main/clj/dda/c4k_forgejo/uberjar.clj
View file

@ -7,10 +7,11 @@
(set! *warn-on-reflection* true)
(defn -main [& cmd-args]
(uberjar/main-common
(uberjar/main-cm
"c4k-forgejo"
core/config?
core/auth?
core/config-defaults
core/k8s-objects
core/config-objects
core/auth-objects
cmd-args))

94
src/main/cljc/dda/c4k_forgejo/core.cljc
View file

@ -1,23 +1,31 @@
(ns dda.c4k-forgejo.core
(:require
[clojure.spec.alpha :as s]
[dda.c4k-common.yaml :as yaml]
[dda.c4k-common.common :as cm]
[dda.c4k-common.monitoring :as mon]
[dda.c4k-forgejo.forgejo :as forgejo]
[dda.c4k-forgejo.backup :as backup]
[dda.c4k-common.postgres :as postgres]
[dda.c4k-common.namespace :as ns]))
(:require
[clojure.spec.alpha :as s]
[dda.c4k-common.yaml :as yaml]
[dda.c4k-common.common :as cm]
[dda.c4k-common.monitoring :as mon]
[dda.c4k-forgejo.forgejo :as forgejo]
[dda.c4k-forgejo.backup :as backup]
[dda.c4k-common.postgres :as postgres]
[dda.c4k-common.namespace :as ns]))
(def config-defaults {:issuer "staging", :deploy-federated "false", :federation-enabled "false"})
(def config-defaults {:namespace "forgejo"
:issuer "staging"
:deploy-federated "false"
:federation-enabled "false"
:db-name "forgejo"
:pv-storage-size-gb 5
:pvc-storage-class-name ""
:postgres-image "postgres:14"
:postgres-size :2gb})
(def rate-limit-defaults {:max-rate 10, :max-concurrent-requests 5})
(def config? (s/keys :req-un [::forgejo/fqdn
::forgejo/mailer-from
::forgejo/mailer-host
(def config? (s/keys :req-un [::forgejo/fqdn
::forgejo/mailer-from
::forgejo/mailer-host
::forgejo/mailer-port
::forgejo/service-noreply-address]
:opt-un [::forgejo/issuer
:opt-un [::forgejo/issuer
::forgejo/deploy-federated
::forgejo/federation-enabled
::forgejo/default-app-name
@ -34,37 +42,39 @@
(def vol? (s/keys :req-un [::forgejo/volume-total-storage-size]))
(def postgres-config {:db-name "forgejo"
:pv-storage-size-gb 5
:pvc-storage-class-name ""
:postgres-image "postgres:14"
:postgres-size :2gb})
(defn k8s-objects [config auth] ; ToDo: ADR for generate functions - vector or no vector?
(let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path)
resolved-config (merge {:namespace "forgejo"} postgres-config config)]
(defn config-objects [config] ; ToDo: ADR for generate functions - vector or no vector?
(let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path)]
(map yaml/to-string
(filter #(not (nil? %))
(cm/concat-vec
(ns/generate resolved-config)
[(postgres/generate-config resolved-config)
(postgres/generate-secret {:namespace "forgejo"} auth)
(when (contains? resolved-config :postgres-data-volume-path)
(postgres/generate-persistent-volume (select-keys resolved-config [:postgres-data-volume-path :pv-storage-size-gb])))
(postgres/generate-pvc (merge resolved-config {:pvc-storage-class-name storage-class}))
(postgres/generate-deployment resolved-config)
(postgres/generate-service resolved-config)
(forgejo/generate-deployment resolved-config)
(ns/generate config)
[(postgres/generate-configmap config)
(when (contains? config :postgres-data-volume-path)
(postgres/generate-persistent-volume (select-keys config [:postgres-data-volume-path :pv-storage-size-gb])))
(postgres/generate-pvc (merge config {:pvc-storage-class-name storage-class}))
(postgres/generate-deployment config)
(postgres/generate-service config)
(forgejo/generate-deployment config)
(forgejo/generate-service)
(forgejo/generate-service-ssh)
(forgejo/generate-data-volume resolved-config)
(forgejo/generate-appini-env resolved-config)
(forgejo/generate-secrets auth)] ; this does not have a vector as output
(forgejo/generate-ratelimit-ingress-and-cert resolved-config) ; this function has a vector as output
(when (contains? resolved-config :restic-repository)
[(backup/generate-config resolved-config)
(backup/generate-secret auth)
(forgejo/generate-data-volume config)
(forgejo/generate-appini-env config)]
(forgejo/generate-ratelimit-ingress-and-cert config) ; this function has a vector as output
(when (contains? config :restic-repository)
[(backup/generate-config config)
(backup/generate-cron)
(backup/generate-backup-restore-deployment resolved-config)])
(when (:contains? resolved-config :mon-cfg)
(mon/generate (:mon-cfg resolved-config) (:mon-auth auth))))))))
(backup/generate-backup-restore-deployment config)])
(when (contains? config :mon-cfg)
(mon/generate-config)))))))
(defn auth-objects [config auth]
(map yaml/to-string
(filter #(not (nil? %))
(cm/concat-vec
(ns/generate config)
[(postgres/generate-secret config auth)
(forgejo/generate-secrets auth)]
(when (contains? config :restic-repository)
[(backup/generate-secret auth)])
(when (contains? config :mon-cfg)
(mon/generate-auth (:mon-cfg config) (:mon-auth auth)))))))

20
src/main/cljc/dda/c4k_forgejo/forgejo.cljc
View file

@ -73,7 +73,7 @@
(def federated-image-name "domaindrivenarchitecture/c4k-forgejo-federated")
(def federated-image-version "latest")
(def non-federated-image-name "codeberg.org/forgejo/forgejo")
(def non-federated-image-version "7.0")
(def non-federated-image-version "8.0")
(defn-spec generate-image-str string?
[config config?]
@ -102,15 +102,15 @@
federation-enabled-bool (boolean-from-string federation-enabled)]
(->
(yaml/load-as-edn "forgejo/appini-env-configmap.yaml")
(cm/replace-all-matching "APPNAME" default-app-name)
(cm/replace-all-matching "FQDN" fqdn)
(cm/replace-all-matching "URL" (str "https://" fqdn))
(cm/replace-all-matching "FROM" mailer-from)
(cm/replace-all-matching "MAILERHOST" mailer-host)
(cm/replace-all-matching "MAILERPORT" mailer-port)
(cm/replace-all-matching "WHITELISTDOMAINS" service-domain-whitelist)
(cm/replace-all-matching "NOREPLY" service-noreply-address)
(cm/replace-all-matching "IS_FEDERATED"
(cm/replace-all-matching-values-by-new-value "APPNAME" default-app-name)
(cm/replace-all-matching-values-by-new-value "FQDN" fqdn)
(cm/replace-all-matching-values-by-new-value "URL" (str "https://" fqdn))
(cm/replace-all-matching-values-by-new-value "FROM" mailer-from)
(cm/replace-all-matching-values-by-new-value "MAILERHOST" mailer-host)
(cm/replace-all-matching-values-by-new-value "MAILERPORT" mailer-port)
(cm/replace-all-matching-values-by-new-value "WHITELISTDOMAINS" service-domain-whitelist)
(cm/replace-all-matching-values-by-new-value "NOREPLY" service-noreply-address)
(cm/replace-all-matching-values-by-new-value "IS_FEDERATED"
(if federation-enabled-bool
"true"
"false")))))

36
src/main/cljs/dda/c4k_forgejo/browser.cljs
View file

@ -4,7 +4,7 @@
[clojure.tools.reader.edn :as edn]
[dda.c4k-forgejo.core :as core]
[dda.c4k-forgejo.forgejo :as forgejo]
[dda.c4k-common.browser :as br]
[dda.c4k-common.browser :as br]
[dda.c4k-common.common :as cm]))
(defn generate-group
@ -73,14 +73,13 @@
:mailer-host (br/get-content-from-element "mailer-host")
:mailer-port (br/get-content-from-element "mailer-port")
:service-noreply-address (br/get-content-from-element "service-noreply-address")
:volume-total-storage-size (br/get-content-from-element "volume-total-storage-size" :deserializer js/parseInt)}
:volume-total-storage-size (br/get-content-from-element "volume-total-storage-size" :deserializer js/parseInt)}
(when (not (st/blank? issuer))
{:issuer issuer})
(when (not (st/blank? app-name))
{:default-app-name app-name})
(when (not (st/blank? domain-whitelist))
{:service-domain-whitelist domain-whitelist})
)))
{:service-domain-whitelist domain-whitelist}))))
(defn validate-all! []
(br/validate! "fqdn" ::forgejo/fqdn)
@ -91,7 +90,7 @@
(br/validate! "deploy-federated" ::forgejo/deploy-federated :optional true)
(br/validate! "issuer" ::forgejo/issuer :optional true)
(br/validate! "app-name" ::forgejo/default-app-name :optional true)
(br/validate! "domain-whitelist" ::forgejo/service-domain-whitelist :optional true)
(br/validate! "domain-whitelist" ::forgejo/service-domain-whitelist :optional true)
(br/validate! "volume-total-storage-size" ::forgejo/volume-total-storage-size :deserializer js/parseInt)
(br/validate! "auth" forgejo/auth? :deserializer edn/read-string)
(br/set-form-validated!))
@ -103,16 +102,21 @@
(defn init []
(br/append-hickory (generate-content-div))
(-> js/document
(.getElementById "generate-button")
(.addEventListener "click"
#(do (validate-all!)
(-> (cm/generate-common
(config-from-document)
(br/get-content-from-element "auth" :deserializer edn/read-string)
core/config-defaults
core/k8s-objects)
(br/set-output!)))))
(let [config-only false
auth-only false]
(-> js/document
(.getElementById "generate-button")
(.addEventListener "click"
#(do (validate-all!)
(-> (cm/generate-cm
(config-from-document)
(br/get-content-from-element "auth" :deserializer edn/read-string)
core/config-defaults
core/config-objects
core/auth-objects
config-only
auth-only)
(br/set-output!))))))
(add-validate-listener "fqdn")
(add-validate-listener "deploy-federated")
(add-validate-listener "mailer-from")
@ -120,7 +124,7 @@
(add-validate-listener "mailer-port")
(add-validate-listener "service-noreply-address")
(add-validate-listener "app-name")
(add-validate-listener "domain-whitelist")
(add-validate-listener "domain-whitelist")
(add-validate-listener "volume-total-storage-size")
(add-validate-listener "issuer")
(add-validate-listener "auth"))

4
src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc
View file

@ -14,7 +14,7 @@
(deftest should-generate-image-str
(testing "non-federated-image"
(is (= "codeberg.org/forgejo/forgejo:7.0"
(is (= "codeberg.org/forgejo/forgejo:8.0"
(cut/generate-image-str {:fqdn "test.de"
:mailer-from ""
:mailer-host "m.t.de"
@ -97,7 +97,7 @@
:spec
{:containers
[{:name "forgejo",
:image "codeberg.org/forgejo/forgejo:7.0",
:image "codeberg.org/forgejo/forgejo:8.0",
:imagePullPolicy "IfNotPresent",
:envFrom [{:configMapRef {:name "forgejo-env"}} {:secretRef {:name "forgejo-secrets"}}],
:volumeMounts [{:name "forgejo-data-volume", :mountPath "/data"}],