Merge branch 'main' into forgejo-upgrade

This commit is contained in:
Michael Jerger 2024-08-09 17:08:06 +02:00
commit a7c298a824
12 changed files with 115 additions and 91 deletions

View file

@ -46,8 +46,8 @@ Development happens at: https://repo.prod.meissa.de/meissa/c4k-forgejo
Mirrors are: Mirrors are:
* https://codeberg.org/meissa/c4k-forgejo (Issues and PR)
* https://gitlab.com/domaindrivenarchitecture/c4k-forgejo (CI) * https://gitlab.com/domaindrivenarchitecture/c4k-forgejo (CI)
* https://codeberg.org/meissa/c4k-forgejo (issues and PR)
* https://github.com/DomainDrivenArchitecture/c4k-forgejo * https://github.com/DomainDrivenArchitecture/c4k-forgejo
For more details about our repository model see: https://repo.prod.meissa.de/meissa/federate-your-repos For more details about our repository model see: https://repo.prod.meissa.de/meissa/federate-your-repos
@ -55,6 +55,6 @@ For more details about our repository model see: https://repo.prod.meissa.de/mei
## License ## License
Copyright © 2024 meissa GmbH Copyright © 2023, 2024 meissa GmbH
Licensed under the [Apache License, Version 2.0](LICENSE) (the "License") Licensed under the [Apache License, Version 2.0](LICENSE) (the "License")
Pls. find licenses of our subcomponents [here](doc/SUBCOMPONENT_LICENSE) Pls. find licenses of our subcomponents [here](doc/SUBCOMPONENT_LICENSE)

View file

@ -6,6 +6,7 @@
* 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0 * 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0
* 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0 * 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0
* 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0 * 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0
* 8.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#8-0-0
## Preparations ## Preparations
@ -17,8 +18,8 @@
### Create 2nd Repo Prod Server ### Create 2nd Repo Prod Server
1. Terraform Preparations for 2nd Server: TODO 1. Terraform Preparations for 2nd Server: TODO
1. Install c4k-forgejo Version TODO 1. Install c4k-forgejo Version `3.5.0`!
with config `"forgejo-image-version-overwrite": "1.19.3-0"` with config `"forgejo-image-version-overwrite": "1.19.3-0"` (in server-setup)
1. Stop Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` 1. Stop Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Disable Backup Cron: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` 1. Disable Backup Cron: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'`
1. Scale up Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1` 1. Scale up Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1`
@ -36,7 +37,7 @@
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` 1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `1.20.1-0` with `k edit -n forgejo deployment forgejo` 1. Set version to `1.20.1-0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors 1. Check for errors: `k logs -n forgejo forgejo-...`
## Upgrade to 1.21.1-0 ## Upgrade to 1.21.1-0
@ -44,7 +45,7 @@
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` 1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `1.21.1-0` with `k edit -n forgejo deployment forgejo` 1. Set version to `1.21.1-0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors 1. Check for errors: `k logs -n forgejo forgejo-...`
1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`. 1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`.
## Upgrade to 7.0.0 ## Upgrade to 7.0.0
@ -55,17 +56,24 @@
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` 1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `7.0.0` with `k edit -n forgejo deployment forgejo` 1. Set version to `7.0.0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors 1. Check for errors: `k logs -n forgejo forgejo-...`
## Upgrade to 7.0.5 (no breaking changes) ## Upgrade to 8.0.0 (no relevant breaking changes)
TODO: Upgrade to 8.0.0 instead after Release!
1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` 1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` 1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `7.0.5` with `k edit -n forgejo deployment forgejo` 1. Set version to `8.0.0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors 1. Check for errors: `k logs -n forgejo forgejo-...`
## Enable Federation
1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Adjust configmap: `k edit -n forgejo cm forgejo-env`
1. Change `FORGEJO__federation__ENABLED: "false"` TO `FORGEJO__federation__ENABLED: "true"`
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors: `k logs -n forgejo forgejo-...`
## Post Work ## Post Work

View file

@ -6,7 +6,7 @@ from ddadevops import *
name = "c4k-forgejo" name = "c4k-forgejo"
MODULE = "backup" MODULE = "backup"
PROJECT_ROOT_PATH = "../.." PROJECT_ROOT_PATH = "../.."
version = "3.4.5-dev" version = "3.5.1-dev"
@init @init

View file

@ -6,7 +6,7 @@ from ddadevops import *
name = 'c4k-forgejo' name = 'c4k-forgejo'
MODULE = 'federated' MODULE = 'federated'
PROJECT_ROOT_PATH = '../..' PROJECT_ROOT_PATH = '../..'
version = "3.4.5-dev" version = "3.5.1-dev"
@init @init
def initialize(project): def initialize(project):

View file

@ -2,7 +2,7 @@
"name": "c4k-forgejo", "name": "c4k-forgejo",
"description": "Generate c4k yaml for a forgejo deployment.", "description": "Generate c4k yaml for a forgejo deployment.",
"author": "meissa GmbH", "author": "meissa GmbH",
"version": "3.4.5-SNAPSHOT", "version": "3.5.1-SNAPSHOT",
"homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme",
"repository": "https://www.npmjs.com/package/c4k-forgejo", "repository": "https://www.npmjs.com/package/c4k-forgejo",
"license": "APACHE2", "license": "APACHE2",

View file

@ -1,16 +1,17 @@
(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.5-SNAPSHOT" (defproject org.domaindrivenarchitecture/c4k-forgejo "3.5.1-SNAPSHOT"
:description "forgejo c4k-installation package" :description "forgejo c4k-installation package"
:url "https://domaindrivenarchitecture.org" :url "https://domaindrivenarchitecture.org"
:license {:name "Apache License, Version 2.0" :license {:name "Apache License, Version 2.0"
:url "https://www.apache.org/licenses/LICENSE-2.0.html"} :url "https://www.apache.org/licenses/LICENSE-2.0.html"}
:dependencies [[org.clojure/clojure "1.11.3" :scope "provided"] :dependencies [[org.clojure/clojure "1.11.3" :scope "provided"]
[org.clojure/tools.reader "1.4.2"] [org.clojure/tools.reader "1.4.2"]
[org.domaindrivenarchitecture/c4k-common-clj "6.4.1"] [org.domaindrivenarchitecture/c4k-common-clj "7.0.0"]
[hickory "0.7.1" :exclusions [viebel/codox-klipse-theme]]] [hickory "0.7.1" :exclusions [viebel/codox-klipse-theme]]]
:target-path "target/%s/" :target-path "target/%s/"
:source-paths ["src/main/cljc" :source-paths ["src/main/cljc"
"src/main/clj"] "src/main/clj"]
:resource-paths ["src/main/resources"] :resource-paths ["src/main/resources"
"project.clj"]
:repositories [["snapshots" :clojars] :repositories [["snapshots" :clojars]
["releases" :clojars]] ["releases" :clojars]]
:deploy-repositories [["snapshots" {:sign-releases false :url "https://clojars.org/repo"}] :deploy-repositories [["snapshots" {:sign-releases false :url "https://clojars.org/repo"}]

View file

@ -4,7 +4,7 @@
"src/test/cljc" "src/test/cljc"
"src/test/cljs" "src/test/cljs"
"src/test/resources"] "src/test/resources"]
:dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "6.4.1"] :dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "7.0.0"]
[hickory "0.7.1"]] [hickory "0.7.1"]]
:builds {:frontend {:target :browser :builds {:frontend {:target :browser
:modules {:main {:init-fn dda.c4k-forgejo.browser/init}} :modules {:main {:init-fn dda.c4k-forgejo.browser/init}}

View file

@ -7,10 +7,11 @@
(set! *warn-on-reflection* true) (set! *warn-on-reflection* true)
(defn -main [& cmd-args] (defn -main [& cmd-args]
(uberjar/main-common (uberjar/main-cm
"c4k-forgejo" "c4k-forgejo"
core/config? core/config?
core/auth? core/auth?
core/config-defaults core/config-defaults
core/k8s-objects core/config-objects
core/auth-objects
cmd-args)) cmd-args))

View file

@ -9,7 +9,15 @@
[dda.c4k-common.postgres :as postgres] [dda.c4k-common.postgres :as postgres]
[dda.c4k-common.namespace :as ns])) [dda.c4k-common.namespace :as ns]))
(def config-defaults {:issuer "staging", :deploy-federated "false", :federation-enabled "false"}) (def config-defaults {:namespace "forgejo"
:issuer "staging"
:deploy-federated "false"
:federation-enabled "false"
:db-name "forgejo"
:pv-storage-size-gb 5
:pvc-storage-class-name ""
:postgres-image "postgres:14"
:postgres-size :2gb})
(def rate-limit-defaults {:max-rate 10, :max-concurrent-requests 5}) (def rate-limit-defaults {:max-rate 10, :max-concurrent-requests 5})
(def config? (s/keys :req-un [::forgejo/fqdn (def config? (s/keys :req-un [::forgejo/fqdn
@ -34,37 +42,39 @@
(def vol? (s/keys :req-un [::forgejo/volume-total-storage-size])) (def vol? (s/keys :req-un [::forgejo/volume-total-storage-size]))
(def postgres-config {:db-name "forgejo" (defn config-objects [config] ; ToDo: ADR for generate functions - vector or no vector?
:pv-storage-size-gb 5 (let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path)]
:pvc-storage-class-name ""
:postgres-image "postgres:14"
:postgres-size :2gb})
(defn k8s-objects [config auth] ; ToDo: ADR for generate functions - vector or no vector?
(let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path)
resolved-config (merge {:namespace "forgejo"} postgres-config config)]
(map yaml/to-string (map yaml/to-string
(filter #(not (nil? %)) (filter #(not (nil? %))
(cm/concat-vec (cm/concat-vec
(ns/generate resolved-config) (ns/generate config)
[(postgres/generate-config resolved-config) [(postgres/generate-configmap config)
(postgres/generate-secret {:namespace "forgejo"} auth) (when (contains? config :postgres-data-volume-path)
(when (contains? resolved-config :postgres-data-volume-path) (postgres/generate-persistent-volume (select-keys config [:postgres-data-volume-path :pv-storage-size-gb])))
(postgres/generate-persistent-volume (select-keys resolved-config [:postgres-data-volume-path :pv-storage-size-gb]))) (postgres/generate-pvc (merge config {:pvc-storage-class-name storage-class}))
(postgres/generate-pvc (merge resolved-config {:pvc-storage-class-name storage-class})) (postgres/generate-deployment config)
(postgres/generate-deployment resolved-config) (postgres/generate-service config)
(postgres/generate-service resolved-config) (forgejo/generate-deployment config)
(forgejo/generate-deployment resolved-config)
(forgejo/generate-service) (forgejo/generate-service)
(forgejo/generate-service-ssh) (forgejo/generate-service-ssh)
(forgejo/generate-data-volume resolved-config) (forgejo/generate-data-volume config)
(forgejo/generate-appini-env resolved-config) (forgejo/generate-appini-env config)]
(forgejo/generate-secrets auth)] ; this does not have a vector as output (forgejo/generate-ratelimit-ingress-and-cert config) ; this function has a vector as output
(forgejo/generate-ratelimit-ingress-and-cert resolved-config) ; this function has a vector as output (when (contains? config :restic-repository)
(when (contains? resolved-config :restic-repository) [(backup/generate-config config)
[(backup/generate-config resolved-config)
(backup/generate-secret auth)
(backup/generate-cron) (backup/generate-cron)
(backup/generate-backup-restore-deployment resolved-config)]) (backup/generate-backup-restore-deployment config)])
(when (:contains? resolved-config :mon-cfg) (when (contains? config :mon-cfg)
(mon/generate (:mon-cfg resolved-config) (:mon-auth auth)))))))) (mon/generate-config)))))))
(defn auth-objects [config auth]
(map yaml/to-string
(filter #(not (nil? %))
(cm/concat-vec
(ns/generate config)
[(postgres/generate-secret config auth)
(forgejo/generate-secrets auth)]
(when (contains? config :restic-repository)
[(backup/generate-secret auth)])
(when (contains? config :mon-cfg)
(mon/generate-auth (:mon-cfg config) (:mon-auth auth)))))))

View file

@ -73,7 +73,7 @@
(def federated-image-name "domaindrivenarchitecture/c4k-forgejo-federated") (def federated-image-name "domaindrivenarchitecture/c4k-forgejo-federated")
(def federated-image-version "latest") (def federated-image-version "latest")
(def non-federated-image-name "codeberg.org/forgejo/forgejo") (def non-federated-image-name "codeberg.org/forgejo/forgejo")
(def non-federated-image-version "7.0") (def non-federated-image-version "8.0")
(defn-spec generate-image-str string? (defn-spec generate-image-str string?
[config config?] [config config?]
@ -102,15 +102,15 @@
federation-enabled-bool (boolean-from-string federation-enabled)] federation-enabled-bool (boolean-from-string federation-enabled)]
(-> (->
(yaml/load-as-edn "forgejo/appini-env-configmap.yaml") (yaml/load-as-edn "forgejo/appini-env-configmap.yaml")
(cm/replace-all-matching "APPNAME" default-app-name) (cm/replace-all-matching-values-by-new-value "APPNAME" default-app-name)
(cm/replace-all-matching "FQDN" fqdn) (cm/replace-all-matching-values-by-new-value "FQDN" fqdn)
(cm/replace-all-matching "URL" (str "https://" fqdn)) (cm/replace-all-matching-values-by-new-value "URL" (str "https://" fqdn))
(cm/replace-all-matching "FROM" mailer-from) (cm/replace-all-matching-values-by-new-value "FROM" mailer-from)
(cm/replace-all-matching "MAILERHOST" mailer-host) (cm/replace-all-matching-values-by-new-value "MAILERHOST" mailer-host)
(cm/replace-all-matching "MAILERPORT" mailer-port) (cm/replace-all-matching-values-by-new-value "MAILERPORT" mailer-port)
(cm/replace-all-matching "WHITELISTDOMAINS" service-domain-whitelist) (cm/replace-all-matching-values-by-new-value "WHITELISTDOMAINS" service-domain-whitelist)
(cm/replace-all-matching "NOREPLY" service-noreply-address) (cm/replace-all-matching-values-by-new-value "NOREPLY" service-noreply-address)
(cm/replace-all-matching "IS_FEDERATED" (cm/replace-all-matching-values-by-new-value "IS_FEDERATED"
(if federation-enabled-bool (if federation-enabled-bool
"true" "true"
"false"))))) "false")))))

View file

@ -79,8 +79,7 @@
(when (not (st/blank? app-name)) (when (not (st/blank? app-name))
{:default-app-name app-name}) {:default-app-name app-name})
(when (not (st/blank? domain-whitelist)) (when (not (st/blank? domain-whitelist))
{:service-domain-whitelist domain-whitelist}) {:service-domain-whitelist domain-whitelist}))))
)))
(defn validate-all! [] (defn validate-all! []
(br/validate! "fqdn" ::forgejo/fqdn) (br/validate! "fqdn" ::forgejo/fqdn)
@ -103,16 +102,21 @@
(defn init [] (defn init []
(br/append-hickory (generate-content-div)) (br/append-hickory (generate-content-div))
(let [config-only false
auth-only false]
(-> js/document (-> js/document
(.getElementById "generate-button") (.getElementById "generate-button")
(.addEventListener "click" (.addEventListener "click"
#(do (validate-all!) #(do (validate-all!)
(-> (cm/generate-common (-> (cm/generate-cm
(config-from-document) (config-from-document)
(br/get-content-from-element "auth" :deserializer edn/read-string) (br/get-content-from-element "auth" :deserializer edn/read-string)
core/config-defaults core/config-defaults
core/k8s-objects) core/config-objects
(br/set-output!))))) core/auth-objects
config-only
auth-only)
(br/set-output!))))))
(add-validate-listener "fqdn") (add-validate-listener "fqdn")
(add-validate-listener "deploy-federated") (add-validate-listener "deploy-federated")
(add-validate-listener "mailer-from") (add-validate-listener "mailer-from")

View file

@ -14,7 +14,7 @@
(deftest should-generate-image-str (deftest should-generate-image-str
(testing "non-federated-image" (testing "non-federated-image"
(is (= "codeberg.org/forgejo/forgejo:7.0" (is (= "codeberg.org/forgejo/forgejo:8.0"
(cut/generate-image-str {:fqdn "test.de" (cut/generate-image-str {:fqdn "test.de"
:mailer-from "" :mailer-from ""
:mailer-host "m.t.de" :mailer-host "m.t.de"
@ -97,7 +97,7 @@
:spec :spec
{:containers {:containers
[{:name "forgejo", [{:name "forgejo",
:image "codeberg.org/forgejo/forgejo:7.0", :image "codeberg.org/forgejo/forgejo:8.0",
:imagePullPolicy "IfNotPresent", :imagePullPolicy "IfNotPresent",
:envFrom [{:configMapRef {:name "forgejo-env"}} {:secretRef {:name "forgejo-secrets"}}], :envFrom [{:configMapRef {:name "forgejo-env"}} {:secretRef {:name "forgejo-secrets"}}],
:volumeMounts [{:name "forgejo-data-volume", :mountPath "/data"}], :volumeMounts [{:name "forgejo-data-volume", :mountPath "/data"}],