From eb33cf5e0b2f1f4c24db46270a81df5548daf170 Mon Sep 17 00:00:00 2001 From: patdyn Date: Fri, 5 Jul 2024 09:06:43 +0200 Subject: [PATCH 01/58] [Skip-CI] Add Upgrade process for 1.19 to forgejo sem --- doc/Upgrading.md | 169 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 168 insertions(+), 1 deletion(-) diff --git a/doc/Upgrading.md b/doc/Upgrading.md index 8475aa9..b9084bf 100644 --- a/doc/Upgrading.md +++ b/doc/Upgrading.md @@ -18,4 +18,171 @@ kubectl scale deployment forgejo --replicas=1 Logging into the admin account should now show the new version. -You may want to update your c4k-forgejo resources to reflect the changes made on the cluster. \ No newline at end of file +You may want to update your c4k-forgejo resources to reflect the changes made on the cluster. + +## Upgrading from 1.19 + +### Config related issues with c4k-forgejo v3.2.2 + +- oauth2: ENABLED instead of ENABLE + - `FORGEJOoauth2ENABLED: "true"` +- 2024/07/02 13:16:17 ...g/config_provider.go:329:deprecatedSetting() [E] Deprecated config option `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` i + nstead. This fallback will be/has been removed in 1.21 + - `FORGEJOlog_0x2E_logger_0x2E_routerMODE: console, file` +- 2024/07/02 13:16:17 ...g/config_provider.go:329:deprecatedSetting() [E] Deprecated config option `[service]` `EMAIL_DOMAIN_WHITELIST` present. Use `[service] + ` `EMAIL_DOMAIN_ALLOWLIST` instead. This fallback will be/has been removed in 1.21 + - `FORGEJOserviceEMAIL_DOMAIN_ALLOWLIST:` +- 2024/07/02 13:16:17 ...g/config_provider.go:329:deprecatedSetting() [E] Deprecated config option `[mailer]` `MAILER_TYPE` present. Use `[mailer]` `PROTOCOL` + instead. This fallback will be/has been removed in v1.19.0 + - ...es/setting/mailer.go:133:loadMailerFrom() [E] Deprecated fallback `[mailer]` `PROTOCOL = smtp+startls` present. Use `[mailer]` `PROTOC + OL = smtp+starttls`` instead. This fallback will be removed in v1.19.0 + - `FORGEJOmailerPROTOCOL: smtp+starttls` + - starttls instead of startls + +### Breaking Changes + +#### 1.19.3:Aktueller Stand + +#### 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0 + +##### app.ini + +- check [queue] section - n/e +- check [repository.editor] - n/e +- check [storage] - n/e +- check ssh_keygen_path in app.ini - n/e +- is WORK_PATH set? Or app.ini writeable by forgejo server user? + - 1. no + - 2. probably + - If not, it shows in the logs starting with: `Unable to update WORK_PATH` + - Also ssh pushing will likely fail - *test ssh* + - no errors on test instance +- set logger.router.mode as described in environment-to-ini + - see: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/contrib/environment-to-ini +- check [git.reflog] and maybe move to [git.config] - n/e +- check [indexer], [mailer], [repository] - n/e + +##### tokens + +- scoped and personal access tokens were refactored + - scope may change, if we have tokens they should be rotated + +#### 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0 + +##### custom themes + +- move to `custom/public/assets/` + +##### git branches + +- `/admin` page and click run Sync missed branches from git data to databases. + +##### db - mysql + +- we use postgres + +##### ssh server + +- host cert used for auth? + +##### ssh keys + +- all team members need to check their key length, now 3072 + +##### tokens + +- finer restrictions might now return 404 errors on users in certain teams with certain restrictions + +#### 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0 + +##### webhooks + +- do we use webhooks? + +##### db + +- psql min ver is 12 - should be fine right? + +##### api + +- benutzen wir: + - [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) + - [`/repos/{owner}/{repo}/push_mirrors`](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) + - Application profiling + +##### repos + +- do we have repo descriptions? + - https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d + +##### app.ini + +- check [ui] - n/e + +### Vor dem Upgrade + +- host cert used for auth? +- benutzen wir webhooks? +- benutzen wir: + - [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) + - [`/repos/{owner}/{repo}/push_mirrors`](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) + - Application profiling +- do we have repo descriptions? + - https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d + +### Upgrade plan + +TEST kennzeichnet Aktionen die nur für den Testserver gelten und in PROD ignoriert werden. +PROD kennzeichnet Aktionen die nur für den Testserver gelten und in TEST ignoriert werden. +Generelle Übersicht zu Upgrades: https://forgejo.org/docs/latest/admin/upgrade/ + +- Forgejo server aufsetzen mit alter version +- TEST + - Alte remote id löschen + - ssh-keygen -f "/home/erik/.ssh/known_hosts" -R "repo.test.meissa.de" +- auf server ssh'en +- forgejo pod runterfahren + - `k scale deployment forgejo --replicas=0` +- install lock aus + - `k edit cm forgejo-env` + - set to `FORGEJOsecurityINSTALL_LOCK: "false"` +- forgejo pod hochfahren + - `k scale deployment forgejo --replicas=1` +- admin test oder prod admin anlegen und forgejo installieren + - `gopass show server/meissa/forgejo-test` bzw `-prod` +- forgejo pod runterfahren +- install lock an + - set to `FORGEJOsecurityINSTALL_LOCK: "true"` +- TEST + - forgejo pod hochfahren + - einloggen + - ssh keys anlegen + - ed_xyz + - rsa mit 2048 + - rsa mit 4096 + - Repos anlegen + - forgejo pod runterfahren +- PROD + - backup pod hochfahren + - `k scale deployment backup-restore --replicas=1` + - backups zurückspielen + - im backup pod vorhandene app.ini's löschen bzw umbenennen + - backup pod runterfahren + - `k scale deployment backup-restore --replicas=0` +- im deployment image ver auf 7.0.4 setzen + - `k edit deployment.apps forgejo` +- configmap updaten + - double check ob install lock an + - siehe Config related errors oben +- TEST + - backup pod hochfahren + - im backup pod vorhandene app.ini's löschen bzw umbenennen + - backup pod runterfahren + - `k scale deployment backup-restore --replicas=0` +- forgejo pod hochscalen +- Migrations happen automatically +- `/admin` page and click run Sync missed branches from git data to databases + - and **Sync missed tags ...* +- rsa keys with size 2048 can not be added anymore. However, it seems they still can be used if they are on the server +- team members having app tokens need to recreate them with proper scopes +- add analytics: https://forgejo.org/docs/latest/admin/customization/ \ No newline at end of file From b9d38bdeed155f1f207ddfc5e93d24bf60be89fb Mon Sep 17 00:00:00 2001 From: patdyn Date: Fri, 5 Jul 2024 10:29:51 +0200 Subject: [PATCH 02/58] [Skip-CI] Clarifications --- doc/Upgrading.md | 89 +++++++++++++++++++++++++----------------------- 1 file changed, 46 insertions(+), 43 deletions(-) diff --git a/doc/Upgrading.md b/doc/Upgrading.md index b9084bf..bba465d 100644 --- a/doc/Upgrading.md +++ b/doc/Upgrading.md @@ -25,18 +25,17 @@ You may want to update your c4k-forgejo resources to reflect the changes made on ### Config related issues with c4k-forgejo v3.2.2 - oauth2: ENABLED instead of ENABLE - - `FORGEJOoauth2ENABLED: "true"` + - `FORGEJO__oauth2__ENABLED: "true"` - 2024/07/02 13:16:17 ...g/config_provider.go:329:deprecatedSetting() [E] Deprecated config option `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` i nstead. This fallback will be/has been removed in 1.21 - - `FORGEJOlog_0x2E_logger_0x2E_routerMODE: console, file` + - `FORGEJO__log_0x2E_logger_0x2E_router__MODE: console, file` - 2024/07/02 13:16:17 ...g/config_provider.go:329:deprecatedSetting() [E] Deprecated config option `[service]` `EMAIL_DOMAIN_WHITELIST` present. Use `[service] ` `EMAIL_DOMAIN_ALLOWLIST` instead. This fallback will be/has been removed in 1.21 - - `FORGEJOserviceEMAIL_DOMAIN_ALLOWLIST:` + - `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST:` - 2024/07/02 13:16:17 ...g/config_provider.go:329:deprecatedSetting() [E] Deprecated config option `[mailer]` `MAILER_TYPE` present. Use `[mailer]` `PROTOCOL` instead. This fallback will be/has been removed in v1.19.0 - - ...es/setting/mailer.go:133:loadMailerFrom() [E] Deprecated fallback `[mailer]` `PROTOCOL = smtp+startls` present. Use `[mailer]` `PROTOC - OL = smtp+starttls`` instead. This fallback will be removed in v1.19.0 - - `FORGEJOmailerPROTOCOL: smtp+starttls` + - ...es/setting/mailer.go:133:loadMailerFrom() [E] Deprecated fallback `[mailer]` `PROTOCOL = smtp+startls` present. Use `[mailer]` `PROTOCOL = smtp+starttls`` instead. This fallback will be removed in v1.19.0 + - `FORGEJO__mailer__PROTOCOL: smtp+starttls` - starttls instead of startls ### Breaking Changes @@ -107,7 +106,7 @@ You may want to update your c4k-forgejo resources to reflect the changes made on - benutzen wir: - [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) - - [`/repos/{owner}/{repo}/push_mirrors`](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) + - [/repos/{owner}/{repo}/push_mirrors](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) - Application profiling ##### repos @@ -121,14 +120,14 @@ You may want to update your c4k-forgejo resources to reflect the changes made on ### Vor dem Upgrade -- host cert used for auth? -- benutzen wir webhooks? +- host cert used for auth? - nein +- benutzen wir webhooks? - nein - benutzen wir: - - [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) - - [`/repos/{owner}/{repo}/push_mirrors`](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) - - Application profiling + - [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) - ja + - [`/repos/{owner}/{repo}/push_mirrors`](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) - nein + - Application profiling - nein - do we have repo descriptions? - - https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d + - https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d - ja ### Upgrade plan @@ -136,53 +135,57 @@ TEST kennzeichnet Aktionen die nur für den Testserver gelten und in PROD ignori PROD kennzeichnet Aktionen die nur für den Testserver gelten und in TEST ignoriert werden. Generelle Übersicht zu Upgrades: https://forgejo.org/docs/latest/admin/upgrade/ -- Forgejo server aufsetzen mit alter version +- Forgejo Server aufsetzen mit c4k-forgejo v3.2.2 + - Enthält Forgejo v1.19 - TEST - Alte remote id löschen - - ssh-keygen -f "/home/erik/.ssh/known_hosts" -R "repo.test.meissa.de" -- auf server ssh'en -- forgejo pod runterfahren + - `ssh-keygen -f "/home/${USER}/.ssh/known_hosts" -R "repo.test.meissa.de"` +- Auf server ssh'en +- Forgejo pod runterfahren - `k scale deployment forgejo --replicas=0` -- install lock aus +- Install lock aus - `k edit cm forgejo-env` - - set to `FORGEJOsecurityINSTALL_LOCK: "false"` -- forgejo pod hochfahren + - Set to `FORGEJO__security__INSTALL_LOCK: "false"` +- Forgejo pod hochfahren - `k scale deployment forgejo --replicas=1` -- admin test oder prod admin anlegen und forgejo installieren +- Admin test oder prod admin anlegen und forgejo installieren - `gopass show server/meissa/forgejo-test` bzw `-prod` -- forgejo pod runterfahren -- install lock an - - set to `FORGEJOsecurityINSTALL_LOCK: "true"` +- Forgejo pod runterfahren +- Install lock an + - Set to `FORGEJO__security__INSTALL_LOCK: "true"` - TEST - - forgejo pod hochfahren - - einloggen - - ssh keys anlegen + - Forgejo pod hochfahren + - Einloggen + - Ssh keys anlegen - ed_xyz - rsa mit 2048 - rsa mit 4096 - Repos anlegen - - forgejo pod runterfahren + - Forgejo pod runterfahren - PROD - - backup pod hochfahren + - Backup pod hochfahren - `k scale deployment backup-restore --replicas=1` - - backups zurückspielen - - im backup pod vorhandene app.ini's löschen bzw umbenennen - - backup pod runterfahren + - Backups zurückspielen + - Im backup pod vorhandene app.ini's löschen bzw umbenennen + - Backup pod runterfahren - `k scale deployment backup-restore --replicas=0` -- im deployment image ver auf 7.0.4 setzen +- Im Forgejo Deployment die Image Ver auf 7.0.4 setzen - `k edit deployment.apps forgejo` -- configmap updaten - - double check ob install lock an - - siehe Config related errors oben +- Configmap updaten: + - Double check ob install lock an + - `FORGEJO__oauth2__ENABLED: "true"` + - `FORGEJO__log_0x2E_logger_0x2E_router__MODE: console, file` + - `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST:` + - `FORGEJO__mailer__PROTOCOL: smtp+starttls` - TEST - - backup pod hochfahren - - im backup pod vorhandene app.ini's löschen bzw umbenennen - - backup pod runterfahren + - Backup pod hochfahren + - Im backup pod vorhandene app.ini's löschen bzw umbenennen + - Backup pod runterfahren - `k scale deployment backup-restore --replicas=0` -- forgejo pod hochscalen +- Forgejo pod hochscalen - Migrations happen automatically - `/admin` page and click run Sync missed branches from git data to databases - and **Sync missed tags ...* -- rsa keys with size 2048 can not be added anymore. However, it seems they still can be used if they are on the server -- team members having app tokens need to recreate them with proper scopes -- add analytics: https://forgejo.org/docs/latest/admin/customization/ \ No newline at end of file +- Rsa keys with size 2048 can not be added anymore. However, it seems they still can be used if they are on the server +- Team members having app tokens need to recreate them with proper scopes +- Add analytics: https://forgejo.org/docs/latest/admin/customization/ \ No newline at end of file From 322a07de0335a072683fe940d773067a83ac0a3c Mon Sep 17 00:00:00 2001 From: patdyn Date: Fri, 5 Jul 2024 11:34:14 +0200 Subject: [PATCH 03/58] [Skip-CI] Consistent English --- doc/Upgrading.md | 81 +++++++++++++++++++++++++----------------------- 1 file changed, 43 insertions(+), 38 deletions(-) diff --git a/doc/Upgrading.md b/doc/Upgrading.md index bba465d..dc3639c 100644 --- a/doc/Upgrading.md +++ b/doc/Upgrading.md @@ -120,69 +120,74 @@ You may want to update your c4k-forgejo resources to reflect the changes made on ### Vor dem Upgrade -- host cert used for auth? - nein -- benutzen wir webhooks? - nein -- benutzen wir: - - [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) - ja - - [`/repos/{owner}/{repo}/push_mirrors`](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) - nein - - Application profiling - nein -- do we have repo descriptions? - - https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d - ja +- host cert used for auth? - no +- do we use webhooks? - no +- do we use: + - [/repos/{owner}/{repo}/releases - repoListReleases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) - no + - In the ListReleases, the `per_page` parameter has been decoupled from the `limit` parameter, we do not use the repoListReleases endpoint + - In the `ArtifactDeploymentApi` in dda-devops-build we only use the `POST` method + - The respective endpoint is [repoCreateRelease](https://code.forgejo.org/api/swagger/#/repository/repoCreateRelease) + - [`/repos/{owner}/{repo}/push_mirrors`](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) - no + - Application profiling - no +- do we have repo descriptions? - yes + - There is now a sanitizer that only allows links, emphasis, code and emojis + - See: https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d + - Our repository descriptions are mostly plaintext and links ### Upgrade plan -TEST kennzeichnet Aktionen die nur für den Testserver gelten und in PROD ignoriert werden. -PROD kennzeichnet Aktionen die nur für den Testserver gelten und in TEST ignoriert werden. -Generelle Übersicht zu Upgrades: https://forgejo.org/docs/latest/admin/upgrade/ +TEST indicates actions that only apply to the test server and are ignored in PROD. +PROD indicates actions that only apply to the test server and are ignored in TEST. +See also the overview for upgrading: https://forgejo.org/docs/latest/admin/upgrade/ -- Forgejo Server aufsetzen mit c4k-forgejo v3.2.2 - - Enthält Forgejo v1.19 +- Set up Forgejo server with c4k-forgejo v3.2.2 + - Has Forgejo v1.19 - TEST - - Alte remote id löschen + - Delete old remote ids - `ssh-keygen -f "/home/${USER}/.ssh/known_hosts" -R "repo.test.meissa.de"` -- Auf server ssh'en -- Forgejo pod runterfahren +- Ssh to server +- Forgejo pod downscale - `k scale deployment forgejo --replicas=0` -- Install lock aus +- Install lock off - `k edit cm forgejo-env` - Set to `FORGEJO__security__INSTALL_LOCK: "false"` -- Forgejo pod hochfahren +- Forgejo pod upscale - `k scale deployment forgejo --replicas=1` -- Admin test oder prod admin anlegen und forgejo installieren +- Create admin test or prod admin and install forgejo - `gopass show server/meissa/forgejo-test` bzw `-prod` -- Forgejo pod runterfahren -- Install lock an +- Forgejo pod downscale +- Install lock on - Set to `FORGEJO__security__INSTALL_LOCK: "true"` - TEST - - Forgejo pod hochfahren - - Einloggen - - Ssh keys anlegen + - Forgejo pod upscale + - Log in + - Make Ssh keys - ed_xyz - rsa mit 2048 - rsa mit 4096 - - Repos anlegen - - Forgejo pod runterfahren + - Create repos + - Forgejo pod downscale - PROD - - Backup pod hochfahren + - Backup pod upscale - `k scale deployment backup-restore --replicas=1` - - Backups zurückspielen - - Im backup pod vorhandene app.ini's löschen bzw umbenennen - - Backup pod runterfahren + - Restore backups + - Delete or rename app.ini's in the pod + - Backup pod downscale - `k scale deployment backup-restore --replicas=0` -- Im Forgejo Deployment die Image Ver auf 7.0.4 setzen +- Set image version to 7.0.4 in forgejo deployment - `k edit deployment.apps forgejo` -- Configmap updaten: - - Double check ob install lock an +- Update configmap: + - Double check install lock enabled - `FORGEJO__oauth2__ENABLED: "true"` - `FORGEJO__log_0x2E_logger_0x2E_router__MODE: console, file` - `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST:` - `FORGEJO__mailer__PROTOCOL: smtp+starttls` + - `FORGEJO__federation__ENABLED: true` - TEST - - Backup pod hochfahren - - Im backup pod vorhandene app.ini's löschen bzw umbenennen - - Backup pod runterfahren - - `k scale deployment backup-restore --replicas=0` -- Forgejo pod hochscalen + - Backup pod upscale + - Delete or rename app.ini's in the pod + - Backup pod downscale +- Forgejo pod upscale - Migrations happen automatically - `/admin` page and click run Sync missed branches from git data to databases - and **Sync missed tags ...* From 48b00899ead85e267df5fa7473c4c87e8caef6ff Mon Sep 17 00:00:00 2001 From: patdyn Date: Fri, 5 Jul 2024 11:47:32 +0200 Subject: [PATCH 04/58] [Skip-CI] Format, remove unnecessary details --- doc/Upgrading.md | 99 ++++++++++++++++++++++++------------------------ 1 file changed, 49 insertions(+), 50 deletions(-) diff --git a/doc/Upgrading.md b/doc/Upgrading.md index dc3639c..6664633 100644 --- a/doc/Upgrading.md +++ b/doc/Upgrading.md @@ -17,60 +17,60 @@ kubectl scale deployment forgejo --replicas=1 ``` Logging into the admin account should now show the new version. - You may want to update your c4k-forgejo resources to reflect the changes made on the cluster. ## Upgrading from 1.19 ### Config related issues with c4k-forgejo v3.2.2 -- oauth2: ENABLED instead of ENABLE - - `FORGEJO__oauth2__ENABLED: "true"` -- 2024/07/02 13:16:17 ...g/config_provider.go:329:deprecatedSetting() [E] Deprecated config option `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` i - nstead. This fallback will be/has been removed in 1.21 - - `FORGEJO__log_0x2E_logger_0x2E_router__MODE: console, file` -- 2024/07/02 13:16:17 ...g/config_provider.go:329:deprecatedSetting() [E] Deprecated config option `[service]` `EMAIL_DOMAIN_WHITELIST` present. Use `[service] - ` `EMAIL_DOMAIN_ALLOWLIST` instead. This fallback will be/has been removed in 1.21 - - `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST:` -- 2024/07/02 13:16:17 ...g/config_provider.go:329:deprecatedSetting() [E] Deprecated config option `[mailer]` `MAILER_TYPE` present. Use `[mailer]` `PROTOCOL` - instead. This fallback will be/has been removed in v1.19.0 - - ...es/setting/mailer.go:133:loadMailerFrom() [E] Deprecated fallback `[mailer]` `PROTOCOL = smtp+startls` present. Use `[mailer]` `PROTOCOL = smtp+starttls`` instead. This fallback will be removed in v1.19.0 - - `FORGEJO__mailer__PROTOCOL: smtp+starttls` - - starttls instead of startls - -### Breaking Changes +These errors show in the log, when just upgrading to forgejo v7.0.4 from 1.19 without changing the config. +The related config options are listed below the errors. -#### 1.19.3:Aktueller Stand +- Oauth2: ENABLED instead of ENABLE + - `FORGEJO__oauth2__ENABLED: "true"` +- [E] Deprecated config option `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. + - `FORGEJO__log_0x2E_logger_0x2E_router__MODE: console, file` +- [E] Deprecated config option `[service]` `EMAIL_DOMAIN_WHITELIST` present. Use `[service] + ` `EMAIL_DOMAIN_ALLOWLIST` instead. + - `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: YOUR_ALLOW_LIST` +- [E] Deprecated config option `[mailer]` `MAILER_TYPE` present. Use `[mailer]` `PROTOCOL` + instead. +- [E] Deprecated fallback `[mailer]` `PROTOCOL = smtp+startls` present. Use `[mailer]` `PROTOCOL = smtp+starttls`` instead. + - `FORGEJO__mailer__PROTOCOL: smtp+starttls` + +### Breaking Changes since 1.19 + +#### 1.19.3 & 1.19.4: Version installed by c4k-forgejo v3.2.2 #### 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0 ##### app.ini -- check [queue] section - n/e -- check [repository.editor] - n/e -- check [storage] - n/e -- check ssh_keygen_path in app.ini - n/e -- is WORK_PATH set? Or app.ini writeable by forgejo server user? - - 1. no - - 2. probably +- Check [queue] section - n/e +- Check [repository.editor] - n/e +- Check [storage] - n/e +- Check ssh_keygen_path in app.ini - n/e +- Is WORK_PATH set? Or app.ini writeable by forgejo server user? + - 1. No + - 2. Yes - If not, it shows in the logs starting with: `Unable to update WORK_PATH` - - Also ssh pushing will likely fail - *test ssh* - - no errors on test instance -- set logger.router.mode as described in environment-to-ini - - see: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/contrib/environment-to-ini -- check [git.reflog] and maybe move to [git.config] - n/e -- check [indexer], [mailer], [repository] - n/e + - Also ssh pushing will likely fail + - *test ssh* +- Set logger.router.mode as described in environment-to-ini + - See: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/contrib/environment-to-ini +- Check [git.reflog] and maybe move to [git.config] - n/e +- Check [indexer], [mailer], [repository] - n/e ##### tokens -- scoped and personal access tokens were refactored - - scope may change, if we have tokens they should be rotated +- Scoped and personal access tokens were refactored + - Scope may change, if we have tokens they should be rotated #### 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0 ##### custom themes -- move to `custom/public/assets/` +- Move to `custom/public/assets/` ##### git branches @@ -78,58 +78,57 @@ You may want to update your c4k-forgejo resources to reflect the changes made on ##### db - mysql -- we use postgres +- c4k uses postgres ##### ssh server -- host cert used for auth? +- We don't use host cert used for auth ##### ssh keys -- all team members need to check their key length, now 3072 +- All users need to check their key length, now 3072 ##### tokens -- finer restrictions might now return 404 errors on users in certain teams with certain restrictions +- Finer restrictions might now return 404 errors on users tokens in certain teams with certain restrictions #### 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0 ##### webhooks -- do we use webhooks? +- Do we use webhooks? ##### db -- psql min ver is 12 - should be fine right? +- Psql min ver is 12, c4k-common uses 14+ ##### api -- benutzen wir: - - [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) - - [/repos/{owner}/{repo}/push_mirrors](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) - - Application profiling +- [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) +- [/repos/{owner}/{repo}/push_mirrors](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) +- Application profiling ##### repos -- do we have repo descriptions? +- Do we have repo descriptions? - https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d ##### app.ini -- check [ui] - n/e +- Check [ui] - n/e ### Vor dem Upgrade -- host cert used for auth? - no -- do we use webhooks? - no -- do we use: +- Host cert used for auth? - no +- Do we use webhooks? - no +- Do we use: - [/repos/{owner}/{repo}/releases - repoListReleases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) - no - In the ListReleases, the `per_page` parameter has been decoupled from the `limit` parameter, we do not use the repoListReleases endpoint - In the `ArtifactDeploymentApi` in dda-devops-build we only use the `POST` method - The respective endpoint is [repoCreateRelease](https://code.forgejo.org/api/swagger/#/repository/repoCreateRelease) - [`/repos/{owner}/{repo}/push_mirrors`](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) - no - Application profiling - no -- do we have repo descriptions? - yes +- Do we have repo descriptions? - yes - There is now a sanitizer that only allows links, emphasis, code and emojis - See: https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d - Our repository descriptions are mostly plaintext and links @@ -137,7 +136,7 @@ You may want to update your c4k-forgejo resources to reflect the changes made on ### Upgrade plan TEST indicates actions that only apply to the test server and are ignored in PROD. -PROD indicates actions that only apply to the test server and are ignored in TEST. +PROD indicates actions that only apply to the prod server and are ignored in TEST. See also the overview for upgrading: https://forgejo.org/docs/latest/admin/upgrade/ - Set up Forgejo server with c4k-forgejo v3.2.2 From eaf06724e47bd3692a3f02a49f2719a4efd9822f Mon Sep 17 00:00:00 2001 From: Clemens Date: Mon, 8 Jul 2024 08:28:14 +0200 Subject: [PATCH 05/58] added option to override image-version tag --- src/main/cljc/dda/c4k_forgejo/core.cljc | 1 + src/main/cljc/dda/c4k_forgejo/forgejo.cljc | 23 +++++++---- .../cljc/dda/c4k_forgejo/forgejo_test.cljc | 39 +++++++++++++++++-- 3 files changed, 53 insertions(+), 10 deletions(-) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index ef5c407..e190e35 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -20,6 +20,7 @@ ::forgejo/deploy-federated ::forgejo/default-app-name ::forgejo/service-domain-whitelist + ::forgejo/forgejo-image-version-overwrite ::backup/restic-repository ::mon/mon-cfg])) diff --git a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc index 7d2a5fb..b9de1f8 100644 --- a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc +++ b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc @@ -38,6 +38,7 @@ (s/def ::mailer-port pred/bash-env-string?) (s/def ::service-domain-whitelist domain-list?) (s/def ::service-noreply-address string?) +(s/def ::forgejo-image-version-overwrite string?) (s/def ::mailer-user pred/bash-env-string?) (s/def ::mailer-pw pred/bash-env-string?) (s/def ::issuer pred/letsencrypt-issuer?) @@ -53,7 +54,8 @@ :opt-un [::issuer ::deploy-federated ::default-app-name - ::service-domain-whitelist])) + ::service-domain-whitelist + ::forgejo-image-version-overwrite])) (def rate-limit-config? (s/keys :req-un [::max-rate ::max-concurrent-requests])) @@ -66,8 +68,18 @@ [total] total) -(def federated-image-name "domaindrivenarchitecture/c4k-forgejo-federated:latest") -(def non-federated-image-name "codeberg.org/forgejo/forgejo:1.19") +(def federated-image-name "domaindrivenarchitecture/c4k-forgejo-federated") +(def federated-image-version "latest") +(def non-federated-image-name "codeberg.org/forgejo/forgejo") +(def non-federated-image-version "1.19") + +(defn-spec generate-image-str string? + [config config?] + (let [{:keys [deploy-federated forgejo-image-version-overwrite]} config + deploy-federated-bool (boolean-from-string deploy-federated)] + (if deploy-federated-bool + (str federated-image-name ":" (or forgejo-image-version-overwrite federated-image-version)) + (str non-federated-image-name ":" (or forgejo-image-version-overwrite non-federated-image-version))))) #?(:cljs (defmethod yaml/load-resource :forgejo [resource-name] @@ -158,10 +170,7 @@ deploy-federated-bool (boolean-from-string deploy-federated)] (-> (yaml/load-as-edn "forgejo/deployment.yaml") - (cm/replace-all-matching-values-by-new-value "IMAGE_NAME" - (if deploy-federated-bool - federated-image-name - non-federated-image-name))))) + (cm/replace-all-matching-values-by-new-value "IMAGE_NAME" (generate-image-str config))))) (defn generate-service [] diff --git a/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc b/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc index 54a6070..663793c 100644 --- a/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc +++ b/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc @@ -12,6 +12,40 @@ (st/instrument `cut/generate-ingress) (st/instrument `cut/generate-secrets) +(deftest should-generate-image-str + (testing "non-federated-image" + (is (= "codeberg.org/forgejo/forgejo:1.19" + (cut/generate-image-str {:fqdn "test.de" + :mailer-from "" + :mailer-host "m.t.de" + :mailer-port "123" + :service-noreply-address "" + :deploy-federated "false"}))) + (is (= "codeberg.org/forgejo/forgejo:1.19.3-0" + (cut/generate-image-str {:fqdn "test.de" + :mailer-from "" + :mailer-host "m.t.de" + :mailer-port "123" + :service-noreply-address "" + :deploy-federated "false" + :forgejo-image-version-overwrite "1.19.3-0"})))) + (testing "federated-image" + (is (= "domaindrivenarchitecture/c4k-forgejo-federated:latest" + (cut/generate-image-str {:fqdn "test.de" + :mailer-from "" + :mailer-host "m.t.de" + :mailer-port "123" + :service-noreply-address "" + :deploy-federated "true"}))) + (is (= "domaindrivenarchitecture/c4k-forgejo-federated:3.2.0" + (cut/generate-image-str {:fqdn "test.de" + :mailer-from "" + :mailer-host "m.t.de" + :mailer-port "123" + :service-noreply-address "" + :deploy-federated "true" + :forgejo-image-version-overwrite "3.2.0"}))))) + (deftest should-generate-appini-env (is (= {:APP_NAME-c1 "", :APP_NAME-c2 "test forgejo", @@ -35,13 +69,12 @@ :FORGEJO__service__NO_REPLY_ADDRESS-c2 "noreply@test.com"} (th/map-diff (cut/generate-appini-env {:default-app-name "" :deploy-federated "false" - :fqdn "test.de" + :fqdn "test.de" :mailer-from "" :mailer-host "m.t.de" :mailer-port "123" :service-domain-whitelist "adb.de" - :service-noreply-address "" - }) + :service-noreply-address ""}) (cut/generate-appini-env {:default-app-name "test forgejo" :deploy-federated "true" :fqdn "test.com" From 49ae63536cd69f8472f288fe530353e23a6d5882 Mon Sep 17 00:00:00 2001 From: Clemens Date: Mon, 8 Jul 2024 10:35:15 +0200 Subject: [PATCH 06/58] Began playbook for forgejo version upgrade --- doc/Playbook_UpgradeFrom1.19To7.0.5.md | 46 ++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 doc/Playbook_UpgradeFrom1.19To7.0.5.md diff --git a/doc/Playbook_UpgradeFrom1.19To7.0.5.md b/doc/Playbook_UpgradeFrom1.19To7.0.5.md new file mode 100644 index 0000000..089bd2c --- /dev/null +++ b/doc/Playbook_UpgradeFrom1.19To7.0.5.md @@ -0,0 +1,46 @@ +# Playbook Upgrade from 1.19 to 7.0.5 + +## Info: Relevant Breaking Changes: + +* 1.19.3:Current version +* 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0 +* 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0 +* 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0 + +## Preparations + +1. Stop Forgejo Prod: TODO +1. Disable Backup Cron: TODO +1. Scale up Backup-Restore Deployment: TODO +1. Execute Manual Backup: TODO + +### Create 2nd Repo Prod Server + +1. Terraform Preparations for 2nd Server: TODO +1. Install c4k-forgejo Version TODO + with config `"forgejo-image-version-overwrite": "1.19.3-0"` +1. Stop Forgejo Deployment: TODO +1. Scale up Backup-Restore Deployment: TODO +1. Restore Forgejo Backup: See [BackupAndRestore.md](BackupAndRestore.md) +1. Check for `..._INSTALL_LOCK: true` in ConfigMap `forgejo-env` +1. Scale up Forgejo Deployment and check for (startup) problems: TODO + +## Upgrade to 1.20.1-0 + +1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` +1. Adjust configmap: `k edit cm forgejo-env` + 1. Change `FORGEJO__mailer__MAILER_TYPE: smtp+startls` TO `FORGEJO__mailer__PROTOCOL: smtp+starttls` (Missed deprecation from 1.19) +1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `1.20.1-0` with `k edit deployment forgejo` +1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Check for errors + +## Upgrade to 1.21... + +TODO: +2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21 +2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[service]` `EMAIL_DOMAIN_WHITELIST` present. Use `[service]` `EMAIL_DOMAIN_ALLOWLIST` instead. This fallback will be/has been removed in 1.21 + +## Post Work + +1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens! \ No newline at end of file From 1a82d62bd9291f35b263a58c765789721bc80d38 Mon Sep 17 00:00:00 2001 From: Clemens Date: Mon, 8 Jul 2024 11:53:38 +0200 Subject: [PATCH 07/58] Added forgejo version upgrade to 1.21 in playbook --- doc/Playbook_UpgradeFrom1.19To7.0.5.md | 30 +++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/doc/Playbook_UpgradeFrom1.19To7.0.5.md b/doc/Playbook_UpgradeFrom1.19To7.0.5.md index 089bd2c..017dddb 100644 --- a/doc/Playbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Playbook_UpgradeFrom1.19To7.0.5.md @@ -29,18 +29,38 @@ 1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` 1. Adjust configmap: `k edit cm forgejo-env` + 1. Remove `FORGEJO__database__CHARSET: utf8` (This was a misconfiguration, since this option only had effect for mysql dbs) 1. Change `FORGEJO__mailer__MAILER_TYPE: smtp+startls` TO `FORGEJO__mailer__PROTOCOL: smtp+starttls` (Missed deprecation from 1.19) + 1. Change `FORGEJO__service__EMAIL_DOMAIN_WHITELIST: repo.test.meissa.de` TO `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: repo.test.meissa.de` (Fallback deprecation in 1.21) 1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` 1. Set version to `1.20.1-0` with `k edit deployment forgejo` 1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` 1. Check for errors -## Upgrade to 1.21... +## Upgrade to 1.21.1-0 -TODO: -2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21 -2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[service]` `EMAIL_DOMAIN_WHITELIST` present. Use `[service]` `EMAIL_DOMAIN_ALLOWLIST` instead. This fallback will be/has been removed in 1.21 +1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` +1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `1.21.1-0` with `k edit deployment forgejo` +1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Check for errors +1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`. + +## Upgrade to 7.0.0 + +TODO ## Post Work -1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens! \ No newline at end of file +1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens! +2. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work. + +# Known Errors + +## Error in v1.20.1-0 + +In the logs the following error can be found. This will be resolved automatically with the next upgrade (v1.21). + +``` +2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21 +``` \ No newline at end of file From 636ad0715177bb9635d1e9ef6e0e831e01675ebf Mon Sep 17 00:00:00 2001 From: Clemens Date: Mon, 8 Jul 2024 12:33:37 +0200 Subject: [PATCH 08/58] Added forgejo version upgrade to 7.0.0 in playbook --- doc/Playbook_UpgradeFrom1.19To7.0.5.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/doc/Playbook_UpgradeFrom1.19To7.0.5.md b/doc/Playbook_UpgradeFrom1.19To7.0.5.md index 017dddb..62f6ba3 100644 --- a/doc/Playbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Playbook_UpgradeFrom1.19To7.0.5.md @@ -20,6 +20,7 @@ 1. Install c4k-forgejo Version TODO with config `"forgejo-image-version-overwrite": "1.19.3-0"` 1. Stop Forgejo Deployment: TODO +1. Disable Backup Cron: TODO 1. Scale up Backup-Restore Deployment: TODO 1. Restore Forgejo Backup: See [BackupAndRestore.md](BackupAndRestore.md) 1. Check for `..._INSTALL_LOCK: true` in ConfigMap `forgejo-env` @@ -48,12 +49,21 @@ ## Upgrade to 7.0.0 -TODO +1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` +1. Adjust configmap: `k edit cm forgejo-env` + 1. Change `FORGEJO__oauth2__ENABLE: "true"` TO `FORGEJO__oauth2__ENABLED: "true"` +1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `7.0.0` with `k edit deployment forgejo` +1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Check for errors ## Post Work +1. Switch DNS to new server +1. Reenable Backup Cron: TODO +1. Execute manual Backup: TODO 1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens! -2. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work. +1. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work. # Known Errors @@ -63,4 +73,4 @@ In the logs the following error can be found. This will be resolved automaticall ``` 2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21 -``` \ No newline at end of file +``` From 8db0044895a82bba126a9cf48d7479dcbded33fa Mon Sep 17 00:00:00 2001 From: Clemens Date: Mon, 8 Jul 2024 12:49:43 +0200 Subject: [PATCH 09/58] Added forgejo version upgrade to 7.0.5 in playbook --- doc/Playbook_UpgradeFrom1.19To7.0.5.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/doc/Playbook_UpgradeFrom1.19To7.0.5.md b/doc/Playbook_UpgradeFrom1.19To7.0.5.md index 62f6ba3..290e7e6 100644 --- a/doc/Playbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Playbook_UpgradeFrom1.19To7.0.5.md @@ -57,6 +57,14 @@ 1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` 1. Check for errors +## Upgrade to 7.0.5 (no breaking changes) + +1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` +1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `7.0.5` with `k edit deployment forgejo` +1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Check for errors + ## Post Work 1. Switch DNS to new server @@ -74,3 +82,7 @@ In the logs the following error can be found. This will be resolved automaticall ``` 2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21 ``` + +# TODO: Upgrade from 7.0.5 to 7.0.5-federated + +TODO \ No newline at end of file From c94837a5bfed44934121d6738cd48a0464a8e286 Mon Sep 17 00:00:00 2001 From: Clemens Date: Mon, 8 Jul 2024 12:59:02 +0200 Subject: [PATCH 10/58] Added missing commands into playbook --- doc/Playbook_UpgradeFrom1.19To7.0.5.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/doc/Playbook_UpgradeFrom1.19To7.0.5.md b/doc/Playbook_UpgradeFrom1.19To7.0.5.md index 290e7e6..cc29c85 100644 --- a/doc/Playbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Playbook_UpgradeFrom1.19To7.0.5.md @@ -9,22 +9,22 @@ ## Preparations -1. Stop Forgejo Prod: TODO -1. Disable Backup Cron: TODO -1. Scale up Backup-Restore Deployment: TODO -1. Execute Manual Backup: TODO +1. Stop Forgejo Prod: `k scale deployment forgejo --replicas=0` +1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` +1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1` +1. Execute Manual Backup: `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh` ### Create 2nd Repo Prod Server 1. Terraform Preparations for 2nd Server: TODO 1. Install c4k-forgejo Version TODO with config `"forgejo-image-version-overwrite": "1.19.3-0"` -1. Stop Forgejo Deployment: TODO -1. Disable Backup Cron: TODO -1. Scale up Backup-Restore Deployment: TODO +1. Stop Forgejo Deployment: `k scale deployment forgejo --replicas=0` +1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` +1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1` 1. Restore Forgejo Backup: See [BackupAndRestore.md](BackupAndRestore.md) 1. Check for `..._INSTALL_LOCK: true` in ConfigMap `forgejo-env` -1. Scale up Forgejo Deployment and check for (startup) problems: TODO +1. Scale up Forgejo Deployment and check for (startup) problems: `k scale deployment forgejo --replicas=1` ## Upgrade to 1.20.1-0 @@ -68,8 +68,9 @@ ## Post Work 1. Switch DNS to new server -1. Reenable Backup Cron: TODO -1. Execute manual Backup: TODO +1. Reenable Backup Cron on new server: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : false }}'` +1. Execute manual Backup on new server: `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh` +1. Scale down Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1` 1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens! 1. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work. From d981dfc15f214c0c835997495b590ef690ccdb28 Mon Sep 17 00:00:00 2001 From: Clemens Date: Mon, 8 Jul 2024 13:36:46 +0200 Subject: [PATCH 11/58] update dependencies --- project.clj | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/project.clj b/project.clj index 8c17fec..3f8fd7f 100644 --- a/project.clj +++ b/project.clj @@ -3,9 +3,9 @@ :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" :url "https://www.apache.org/licenses/LICENSE-2.0.html"} - :dependencies [[org.clojure/clojure "1.11.2" :scope "provided"] - [org.clojure/tools.reader "1.4.1"] - [org.domaindrivenarchitecture/c4k-common-clj "6.2.3"] + :dependencies [[org.clojure/clojure "1.11.3" :scope "provided"] + [org.clojure/tools.reader "1.4.2"] + [org.domaindrivenarchitecture/c4k-common-clj "6.4.1"] [hickory "0.7.1" :exclusions [viebel/codox-klipse-theme]]] :target-path "target/%s/" :source-paths ["src/main/cljc" @@ -23,9 +23,9 @@ :main dda.c4k-forgejo.uberjar :uberjar-name "c4k-forgejo-standalone.jar" :dependencies [[org.clojure/tools.cli "1.1.230"] - [ch.qos.logback/logback-classic "1.5.3" + [ch.qos.logback/logback-classic "1.5.6" :exclusions [com.sun.mail/javax.mail]] - [org.slf4j/jcl-over-slf4j "2.0.12"] + [org.slf4j/jcl-over-slf4j "2.0.13"] [com.github.clj-easy/graal-build-time "1.0.5"]]}} :release-tasks [["test"] ["vcs" "assert-committed"] From e40861f46ffdb7a6f2637f033c16724d0b8ff3af Mon Sep 17 00:00:00 2001 From: Clemens Date: Mon, 8 Jul 2024 13:41:49 +0200 Subject: [PATCH 12/58] dependency fix --- src/main/cljc/dda/c4k_forgejo/backup.cljc | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/main/cljc/dda/c4k_forgejo/backup.cljc b/src/main/cljc/dda/c4k_forgejo/backup.cljc index 00426e4..19e22be 100644 --- a/src/main/cljc/dda/c4k_forgejo/backup.cljc +++ b/src/main/cljc/dda/c4k_forgejo/backup.cljc @@ -4,12 +4,13 @@ [dda.c4k-common.yaml :as yaml] [dda.c4k-common.base64 :as b64] [dda.c4k-common.common :as cm] + [dda.c4k-common.predicate :as p] #?(:cljs [dda.c4k-common.macros :refer-macros [inline-resources]]))) -(s/def ::aws-access-key-id cm/bash-env-string?) -(s/def ::aws-secret-access-key cm/bash-env-string?) -(s/def ::restic-password cm/bash-env-string?) -(s/def ::restic-repository cm/bash-env-string?) +(s/def ::aws-access-key-id p/bash-env-string?) +(s/def ::aws-secret-access-key p/bash-env-string?) +(s/def ::restic-password p/bash-env-string?) +(s/def ::restic-repository p/bash-env-string?) #?(:cljs (defmethod yaml/load-resource :backup [resource-name] From f43c3fd7a7595235a32abaaf232129c1174134f4 Mon Sep 17 00:00:00 2001 From: Clemens Date: Mon, 8 Jul 2024 13:55:59 +0200 Subject: [PATCH 13/58] release: 3.3.0 --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index e349315..9d094fe 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.2.3-dev" +version = "3.3.0" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index 84356b6..a57a125 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.2.3-dev" +version = "3.3.0" @init def initialize(project): diff --git a/package.json b/package.json index 8a22eef..b0b65c1 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.2.3-SNAPSHOT", + "version": "3.3.0", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index 3f8fd7f..20ae4d7 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.2.3-SNAPSHOT" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.3.0" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From 3be34862020c295201c666820cff4e79815a0461 Mon Sep 17 00:00:00 2001 From: Clemens Date: Mon, 8 Jul 2024 13:55:59 +0200 Subject: [PATCH 14/58] bump version to: 3.3.1-SNAPSHOT --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 9d094fe..b702e1d 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.3.0" +version = "3.3.1-dev" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index a57a125..3632a3d 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.3.0" +version = "3.3.1-dev" @init def initialize(project): diff --git a/package.json b/package.json index b0b65c1..bbfcef7 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.3.0", + "version": "3.3.1-SNAPSHOT", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index 20ae4d7..eac576d 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.3.0" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.3.1-SNAPSHOT" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From 545410767f32888e7e4d9db8b17c26671db73b50 Mon Sep 17 00:00:00 2001 From: Clemens Date: Tue, 9 Jul 2024 09:05:57 +0200 Subject: [PATCH 15/58] Added runbook todo and disabled forgejo-federated-image-publish --- .gitlab-ci.yml | 13 +++++++------ ...To7.0.5.md => Runbook_UpgradeFrom1.19To7.0.5.md} | 6 ++++-- 2 files changed, 11 insertions(+), 8 deletions(-) rename doc/{Playbook_UpgradeFrom1.19To7.0.5.md => Runbook_UpgradeFrom1.19To7.0.5.md} (98%) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 79ff9ac..a52445c 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -123,9 +123,10 @@ forgejo-backup-image-publish: script: - cd infrastructure/backup && pyb image publish -forgejo-federated-image-publish: - <<: *img - <<: *tag_only - stage: image - script: - - cd infrastructure/federated && pyb image publish \ No newline at end of file +# This is currently not needed +#forgejo-federated-image-publish: +# <<: *img +# <<: *tag_only +# stage: image +# script: +# - cd infrastructure/federated && pyb image publish \ No newline at end of file diff --git a/doc/Playbook_UpgradeFrom1.19To7.0.5.md b/doc/Runbook_UpgradeFrom1.19To7.0.5.md similarity index 98% rename from doc/Playbook_UpgradeFrom1.19To7.0.5.md rename to doc/Runbook_UpgradeFrom1.19To7.0.5.md index cc29c85..914f919 100644 --- a/doc/Playbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Runbook_UpgradeFrom1.19To7.0.5.md @@ -59,6 +59,8 @@ ## Upgrade to 7.0.5 (no breaking changes) +TODO: Upgrade to 8.0.0 instead after Release! + 1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` 1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` 1. Set version to `7.0.5` with `k edit deployment forgejo` @@ -74,9 +76,9 @@ 1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens! 1. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work. -# Known Errors +## Known Errors -## Error in v1.20.1-0 +### Error in v1.20.1-0 In the logs the following error can be found. This will be resolved automatically with the next upgrade (v1.21). From 0eafb03ebda2192b8df3ea45c31e506331188229 Mon Sep 17 00:00:00 2001 From: Clemens Date: Tue, 9 Jul 2024 09:08:02 +0200 Subject: [PATCH 16/58] [Skip-CI] remove deprecated todo --- doc/Runbook_UpgradeFrom1.19To7.0.5.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/doc/Runbook_UpgradeFrom1.19To7.0.5.md b/doc/Runbook_UpgradeFrom1.19To7.0.5.md index 914f919..19ca6c2 100644 --- a/doc/Runbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Runbook_UpgradeFrom1.19To7.0.5.md @@ -85,7 +85,3 @@ In the logs the following error can be found. This will be resolved automaticall ``` 2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21 ``` - -# TODO: Upgrade from 7.0.5 to 7.0.5-federated - -TODO \ No newline at end of file From 51c1d0e7577b9ddb5c52157773b3ffd8f6618241 Mon Sep 17 00:00:00 2001 From: Clemens Date: Tue, 9 Jul 2024 09:15:58 +0200 Subject: [PATCH 17/58] release: 3.3.1 --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index b702e1d..263bcf4 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.3.1-dev" +version = "3.3.1" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index 3632a3d..a1d0df3 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.3.1-dev" +version = "3.3.1" @init def initialize(project): diff --git a/package.json b/package.json index bbfcef7..d78d74c 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.3.1-SNAPSHOT", + "version": "3.3.1", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index eac576d..7ad30ad 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.3.1-SNAPSHOT" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.3.1" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From c1c15f9eaa27fdfe000e23ddb058933f0e299fce Mon Sep 17 00:00:00 2001 From: Clemens Date: Tue, 9 Jul 2024 09:15:58 +0200 Subject: [PATCH 18/58] bump version to: 3.3.2-SNAPSHOT --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 263bcf4..74f6064 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.3.1" +version = "3.3.2-dev" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index a1d0df3..7e71d71 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.3.1" +version = "3.3.2-dev" @init def initialize(project): diff --git a/package.json b/package.json index d78d74c..f0d3614 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.3.1", + "version": "3.3.2-SNAPSHOT", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index 7ad30ad..fd8cf63 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.3.1" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.3.2-SNAPSHOT" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From cf7f77848f2826ace1f541d3a0189db03ae87a13 Mon Sep 17 00:00:00 2001 From: Clemens Date: Tue, 9 Jul 2024 09:25:24 +0200 Subject: [PATCH 19/58] Added howto setup impressum --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index c31e0e0..c596a59 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,11 @@ After having deployed the yaml-file generated by the c4k-forgejo module you need * The SSH-URL for a repo has the format: "ssh://git@domain:2222/[username]/[repo].git Example: "git clone ssh://git@repo.test.meissa.de:2222/myuser/c4k-forgejo.git" +### Add Impressum + +In order to customize the UI e.g. for adding an Impressum, see the [Forgejo Docs](https://forgejo.org/docs/latest/developer/customization/#adding-links-and-tabs). +The individually needed files have to be added by hand into the directory `/data/gitea/templates/custom/` in the forgejo Pod. Since a PV is mounted under `/data`, these ui customizations are persisted. + ## Development & mirrors Development happens at: https://repo.prod.meissa.de/meissa/c4k-forgejo From 6a2cd2e7e8b3d62947732da5d5a2d28d586464c5 Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 10:34:45 +0200 Subject: [PATCH 20/58] Add ns --- src/main/cljc/dda/c4k_forgejo/core.cljc | 6 ++++-- src/main/resources/backup/backup-restore-deployment.yaml | 1 + src/main/resources/backup/config.yaml | 1 + src/main/resources/backup/cron.yaml | 1 + src/main/resources/backup/secret.yaml | 1 + src/main/resources/forgejo/appini-env-configmap.yaml | 2 +- src/main/resources/forgejo/datavolume.yaml | 2 +- src/main/resources/forgejo/deployment.yaml | 2 +- src/main/resources/forgejo/middleware-ratelimit.yaml | 1 + src/main/resources/forgejo/secrets.yaml | 1 + src/main/resources/forgejo/service-ssh.yaml | 2 +- src/main/resources/forgejo/service.yaml | 2 +- src/test/cljc/dda/c4k_forgejo/backup_test.cljc | 2 ++ src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc | 6 +++--- 14 files changed, 20 insertions(+), 10 deletions(-) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index ef5c407..99e363a 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -6,7 +6,8 @@ [dda.c4k-common.monitoring :as mon] [dda.c4k-forgejo.forgejo :as forgejo] [dda.c4k-forgejo.backup :as backup] - [dda.c4k-common.postgres :as postgres])) + [dda.c4k-common.postgres :as postgres] + [dda.c4k-common.namespace :as ns])) (def config-defaults {:issuer "staging", :deploy-federated "false"}) (def rate-limit-defaults {:max-rate 10, :max-concurrent-requests 5}) @@ -36,6 +37,7 @@ (map yaml/to-string (filter #(not (nil? %)) (cm/concat-vec + (ns/generate (merge {:namespace "forgejo"} config)) [(postgres/generate-config {:postgres-size :2gb :db-name "forgejo"}) (postgres/generate-secret auth) (when (contains? config :postgres-data-volume-path) @@ -52,7 +54,7 @@ (forgejo/generate-appini-env config) (forgejo/generate-secrets auth) (forgejo/generate-rate-limit-middleware rate-limit-defaults)] ; this does not have a vector as output - (forgejo/generate-rate-limit-ingress-and-cert config) ; this function has a vector as output + (forgejo/generate-rate-limit-ingress-and-cert (merge {:namespace "keycloak"} config)) ; this function has a vector as output (when (contains? config :restic-repository) [(backup/generate-config config) (backup/generate-secret auth) diff --git a/src/main/resources/backup/backup-restore-deployment.yaml b/src/main/resources/backup/backup-restore-deployment.yaml index 163bc14..9c34d07 100644 --- a/src/main/resources/backup/backup-restore-deployment.yaml +++ b/src/main/resources/backup/backup-restore-deployment.yaml @@ -2,6 +2,7 @@ apiVersion: apps/v1 kind: Deployment metadata: name: backup-restore + namespace: forgejo spec: replicas: 0 selector: diff --git a/src/main/resources/backup/config.yaml b/src/main/resources/backup/config.yaml index f7252a2..f1c7fe1 100644 --- a/src/main/resources/backup/config.yaml +++ b/src/main/resources/backup/config.yaml @@ -2,6 +2,7 @@ apiVersion: v1 kind: ConfigMap metadata: name: backup-config + namespace: forgejo labels: app.kubernetes.io/name: backup app.kubernetes.io/part-of: forgejo diff --git a/src/main/resources/backup/cron.yaml b/src/main/resources/backup/cron.yaml index 6f89bf4..2996efa 100644 --- a/src/main/resources/backup/cron.yaml +++ b/src/main/resources/backup/cron.yaml @@ -2,6 +2,7 @@ apiVersion: batch/v1 kind: CronJob metadata: name: forgejo-backup + namespace: forgejo labels: app.kubernetes.part-of: forgejo spec: diff --git a/src/main/resources/backup/secret.yaml b/src/main/resources/backup/secret.yaml index c5809e0..f4c8fc0 100644 --- a/src/main/resources/backup/secret.yaml +++ b/src/main/resources/backup/secret.yaml @@ -2,6 +2,7 @@ apiVersion: v1 kind: Secret metadata: name: backup-secret + namespace: forgejo type: Opaque data: aws-access-key-id: aws-access-key-id diff --git a/src/main/resources/forgejo/appini-env-configmap.yaml b/src/main/resources/forgejo/appini-env-configmap.yaml index 935de1c..348a62a 100644 --- a/src/main/resources/forgejo/appini-env-configmap.yaml +++ b/src/main/resources/forgejo/appini-env-configmap.yaml @@ -2,7 +2,7 @@ apiVersion: v1 kind: ConfigMap metadata: name: forgejo-env - namespace: default + namespace: forgejo data: #[admin] FORGEJO__admin__DEFAULT_EMAIL_NOTIFICATIONS: "enabled" # Default configuration for email notifications for users (user configurable). Options: enabled, onmention, disabled diff --git a/src/main/resources/forgejo/datavolume.yaml b/src/main/resources/forgejo/datavolume.yaml index 44c8fd0..f874ff3 100644 --- a/src/main/resources/forgejo/datavolume.yaml +++ b/src/main/resources/forgejo/datavolume.yaml @@ -2,7 +2,7 @@ apiVersion: v1 kind: PersistentVolumeClaim metadata: name: forgejo-data-pvc - namespace: default + namespace: forgejo labels: app: forgejo spec: diff --git a/src/main/resources/forgejo/deployment.yaml b/src/main/resources/forgejo/deployment.yaml index 81d5dcb..a254a2d 100644 --- a/src/main/resources/forgejo/deployment.yaml +++ b/src/main/resources/forgejo/deployment.yaml @@ -2,7 +2,7 @@ apiVersion: apps/v1 kind: Deployment metadata: name: forgejo - namespace: default + namespace: forgejo labels: app: forgejo spec: diff --git a/src/main/resources/forgejo/middleware-ratelimit.yaml b/src/main/resources/forgejo/middleware-ratelimit.yaml index 0f6c49d..918dd95 100644 --- a/src/main/resources/forgejo/middleware-ratelimit.yaml +++ b/src/main/resources/forgejo/middleware-ratelimit.yaml @@ -2,6 +2,7 @@ apiVersion: traefik.containo.us/v1alpha1 kind: Middleware metadata: name: ratelimit + namespace: forgejo spec: rateLimit: # Config options for rate limiting: https://doc.traefik.io/traefik/middlewares/http/ratelimit/ average: AVG diff --git a/src/main/resources/forgejo/secrets.yaml b/src/main/resources/forgejo/secrets.yaml index 0c2a224..fbf0eea 100644 --- a/src/main/resources/forgejo/secrets.yaml +++ b/src/main/resources/forgejo/secrets.yaml @@ -2,6 +2,7 @@ apiVersion: v1 kind: Secret metadata: name: forgejo-secrets + namespace: forgejo data: FORGEJO__database__USER: DBUSER FORGEJO__database__PASSWD: DBPW diff --git a/src/main/resources/forgejo/service-ssh.yaml b/src/main/resources/forgejo/service-ssh.yaml index 1694958..2856c85 100644 --- a/src/main/resources/forgejo/service-ssh.yaml +++ b/src/main/resources/forgejo/service-ssh.yaml @@ -2,7 +2,7 @@ kind: Service apiVersion: v1 metadata: name: forgejo-ssh-service - namespace: default + namespace: forgejo annotations: metallb.universe.tf/allow-shared-ip: "shared-ip-service-group" metallb.universe.tf/address-pool: public diff --git a/src/main/resources/forgejo/service.yaml b/src/main/resources/forgejo/service.yaml index e8fed3f..493b5a3 100644 --- a/src/main/resources/forgejo/service.yaml +++ b/src/main/resources/forgejo/service.yaml @@ -2,7 +2,7 @@ kind: Service apiVersion: v1 metadata: name: forgejo-service - namespace: default + namespace: forgejo spec: selector: app: forgejo diff --git a/src/test/cljc/dda/c4k_forgejo/backup_test.cljc b/src/test/cljc/dda/c4k_forgejo/backup_test.cljc index af9eb55..c30f8bb 100644 --- a/src/test/cljc/dda/c4k_forgejo/backup_test.cljc +++ b/src/test/cljc/dda/c4k_forgejo/backup_test.cljc @@ -13,6 +13,7 @@ :kind "ConfigMap", :metadata {:name "backup-config", + :namespace "forgejo", :labels #:app.kubernetes.io{:name "backup", :part-of "forgejo"}}, :data {:restic-repository "s3:s3.amazonaws.com/backup/federated-repo"}} @@ -23,6 +24,7 @@ :kind "ConfigMap", :metadata {:name "backup-config", + :namespace "forgejo", :labels #:app.kubernetes.io{:name "backup", :part-of "forgejo"}}, :data {:restic-repository "s3:s3.amazonaws.com/backup/repo"}} diff --git a/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc b/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc index 54a6070..f3df49b 100644 --- a/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc +++ b/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc @@ -55,7 +55,7 @@ (testing "non-federated" (is (= {:apiVersion "apps/v1", :kind "Deployment", - :metadata {:name "forgejo", :namespace "default", :labels {:app "forgejo"}}, + :metadata {:name "forgejo", :namespace "forgejo", :labels {:app "forgejo"}}, :spec {:replicas 1, :selector {:matchLabels {:app "forgejo"}}, @@ -82,7 +82,7 @@ (testing "federated-deployment" (is (= {:apiVersion "apps/v1", :kind "Deployment", - :metadata {:name "forgejo", :namespace "default", :labels {:app "forgejo"}}, + :metadata {:name "forgejo", :namespace "forgejo", :labels {:app "forgejo"}}, :spec {:replicas 1, :selector {:matchLabels {:app "forgejo"}}, @@ -134,7 +134,7 @@ (deftest should-generate-middleware-ratelimit (is (= {:apiVersion "traefik.containo.us/v1alpha1", :kind "Middleware", - :metadata {:name "ratelimit"}, + :metadata {:name "ratelimit", :namespace "forgejo"}, :spec {:rateLimit {:average 10, :burst 5}}} (cut/generate-rate-limit-middleware {:max-rate 10, :max-concurrent-requests 5})))) From 42eb9206901add54b924805b44e6c664a9df09d0 Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 10:40:52 +0200 Subject: [PATCH 21/58] Update cljs deps --- shadow-cljs.edn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/shadow-cljs.edn b/shadow-cljs.edn index a95e276..6b45f88 100644 --- a/shadow-cljs.edn +++ b/shadow-cljs.edn @@ -4,7 +4,7 @@ "src/test/cljc" "src/test/cljs" "src/test/resources"] - :dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "6.1.3"] + :dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "6.4.1"] [hickory "0.7.1"]] :builds {:frontend {:target :browser :modules {:main {:init-fn dda.c4k-forgejo.browser/init}} From 92d56691a2987fe877aac06ef5a38d478533c648 Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 10:55:49 +0200 Subject: [PATCH 22/58] release: 3.4.0 --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 74f6064..c7fd1fb 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.3.2-dev" +version = "3.4.0" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index 7e71d71..7a7be58 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.3.2-dev" +version = "3.4.0" @init def initialize(project): diff --git a/package.json b/package.json index f0d3614..03319eb 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.3.2-SNAPSHOT", + "version": "3.4.0", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index fd8cf63..ec39cae 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.3.2-SNAPSHOT" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.0" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From 9093748893d87e10692651fcc3cade381611be8f Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 10:55:49 +0200 Subject: [PATCH 23/58] bump version to: 3.4.1-SNAPSHOT --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index c7fd1fb..dd15f9b 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.4.0" +version = "3.4.1-dev" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index 7a7be58..bd9c397 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.4.0" +version = "3.4.1-dev" @init def initialize(project): diff --git a/package.json b/package.json index 03319eb..010f49b 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.4.0", + "version": "3.4.1-SNAPSHOT", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index ec39cae..360f7e0 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.0" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.1-SNAPSHOT" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From 13aa8ca70085ce8ad2a04e38447143adb1dbb02a Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 11:43:39 +0200 Subject: [PATCH 24/58] Fix erroneous keycloak --- src/main/cljc/dda/c4k_forgejo/core.cljc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index 868937b..c2a19aa 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -55,7 +55,7 @@ (forgejo/generate-appini-env config) (forgejo/generate-secrets auth) (forgejo/generate-rate-limit-middleware rate-limit-defaults)] ; this does not have a vector as output - (forgejo/generate-rate-limit-ingress-and-cert (merge {:namespace "keycloak"} config)) ; this function has a vector as output + (forgejo/generate-rate-limit-ingress-and-cert (merge {:namespace "forgejo"} config)) ; this function has a vector as output (when (contains? config :restic-repository) [(backup/generate-config config) (backup/generate-secret auth) From a8e9e6f108f834a8f5236facc9245839e81510cc Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 11:45:38 +0200 Subject: [PATCH 25/58] release: 3.4.1 --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index dd15f9b..56c5f42 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.4.1-dev" +version = "3.4.1" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index bd9c397..934adc0 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.4.1-dev" +version = "3.4.1" @init def initialize(project): diff --git a/package.json b/package.json index 010f49b..3cba186 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.4.1-SNAPSHOT", + "version": "3.4.1", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index 360f7e0..a886049 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.1-SNAPSHOT" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.1" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From b618da8bed4d6d69a7ab0234819af46eb6de7bc3 Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 11:45:38 +0200 Subject: [PATCH 26/58] bump version to: 3.4.2-SNAPSHOT --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 56c5f42..9851f14 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.4.1" +version = "3.4.2-dev" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index 934adc0..eb3c186 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.4.1" +version = "3.4.2-dev" @init def initialize(project): diff --git a/package.json b/package.json index 3cba186..bfb5e0a 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.4.1", + "version": "3.4.2-SNAPSHOT", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index a886049..128488f 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.1" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.2-SNAPSHOT" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From b5b45f8c1c089550d17ae4f13f1057186a0e5fcc Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 12:04:02 +0200 Subject: [PATCH 27/58] Add missing namespace kw --- src/main/cljc/dda/c4k_forgejo/core.cljc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index c2a19aa..b181dd2 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -39,7 +39,9 @@ (filter #(not (nil? %)) (cm/concat-vec (ns/generate (merge {:namespace "forgejo"} config)) - [(postgres/generate-config {:postgres-size :2gb :db-name "forgejo"}) + [(postgres/generate-config {:postgres-size :2gb + :db-name "forgejo" + :namespace "forgejo"}) (postgres/generate-secret auth) (when (contains? config :postgres-data-volume-path) (postgres/generate-persistent-volume (select-keys config [:postgres-data-volume-path :pv-storage-size-gb]))) From 574cc0f76b32bc683ad82e725c4a89c6f83deca0 Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 12:04:14 +0200 Subject: [PATCH 28/58] Use non deprecated functions --- src/main/cljc/dda/c4k_forgejo/forgejo.cljc | 30 +++++++++++----------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc index b9de1f8..10f35f4 100644 --- a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc +++ b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc @@ -100,15 +100,15 @@ deploy-federated-bool (boolean-from-string deploy-federated)] (-> (yaml/load-as-edn "forgejo/appini-env-configmap.yaml") - (cm/replace-all-matching-values-by-new-value "APPNAME" default-app-name) - (cm/replace-all-matching-values-by-new-value "FQDN" fqdn) - (cm/replace-all-matching-values-by-new-value "URL" (str "https://" fqdn)) - (cm/replace-all-matching-values-by-new-value "FROM" mailer-from) - (cm/replace-all-matching-values-by-new-value "MAILERHOST" mailer-host) - (cm/replace-all-matching-values-by-new-value "MAILERPORT" mailer-port) - (cm/replace-all-matching-values-by-new-value "WHITELISTDOMAINS" service-domain-whitelist) - (cm/replace-all-matching-values-by-new-value "NOREPLY" service-noreply-address) - (cm/replace-all-matching-values-by-new-value "IS_FEDERATED" + (cm/replace-all-matching "APPNAME" default-app-name) + (cm/replace-all-matching "FQDN" fqdn) + (cm/replace-all-matching "URL" (str "https://" fqdn)) + (cm/replace-all-matching "FROM" mailer-from) + (cm/replace-all-matching "MAILERHOST" mailer-host) + (cm/replace-all-matching "MAILERPORT" mailer-port) + (cm/replace-all-matching "WHITELISTDOMAINS" service-domain-whitelist) + (cm/replace-all-matching "NOREPLY" service-noreply-address) + (cm/replace-all-matching "IS_FEDERATED" (if deploy-federated-bool "true" "false"))))) @@ -121,10 +121,10 @@ mailer-pw]} auth] (-> (yaml/load-as-edn "forgejo/secrets.yaml") - (cm/replace-all-matching-values-by-new-value "DBUSER" (b64/encode postgres-db-user)) - (cm/replace-all-matching-values-by-new-value "DBPW" (b64/encode postgres-db-password)) - (cm/replace-all-matching-values-by-new-value "MAILERUSER" (b64/encode mailer-user)) - (cm/replace-all-matching-values-by-new-value "MAILERPW" (b64/encode mailer-pw))))) + (cm/replace-all-matching "DBUSER" (b64/encode postgres-db-user)) + (cm/replace-all-matching "DBPW" (b64/encode postgres-db-password)) + (cm/replace-all-matching "MAILERUSER" (b64/encode mailer-user)) + (cm/replace-all-matching "MAILERPW" (b64/encode mailer-pw))))) (defn generate-ingress-and-cert [config] @@ -162,7 +162,7 @@ data-storage-size (data-storage-by-volume-size volume-total-storage-size)] (-> (yaml/load-as-edn "forgejo/datavolume.yaml") - (cm/replace-all-matching-values-by-new-value "DATASTORAGESIZE" (str (str data-storage-size) "Gi"))))) + (cm/replace-all-matching "DATASTORAGESIZE" (str (str data-storage-size) "Gi"))))) (defn-spec generate-deployment pred/map-or-seq? [config config?] @@ -170,7 +170,7 @@ deploy-federated-bool (boolean-from-string deploy-federated)] (-> (yaml/load-as-edn "forgejo/deployment.yaml") - (cm/replace-all-matching-values-by-new-value "IMAGE_NAME" (generate-image-str config))))) + (cm/replace-all-matching "IMAGE_NAME" (generate-image-str config))))) (defn generate-service [] From 665008c1aa23ebcea22060c0abb4683e6782484c Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 12:07:26 +0200 Subject: [PATCH 29/58] release: 3.4.2 --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 9851f14..2b94d3b 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.4.2-dev" +version = "3.4.2" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index eb3c186..bb96c18 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.4.2-dev" +version = "3.4.2" @init def initialize(project): diff --git a/package.json b/package.json index bfb5e0a..4da6ad8 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.4.2-SNAPSHOT", + "version": "3.4.2", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index 128488f..4fa50f4 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.2-SNAPSHOT" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.2" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From 076bfd4d72c749dbbfca13396cca70fc15225430 Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 12:07:26 +0200 Subject: [PATCH 30/58] bump version to: 3.4.3-SNAPSHOT --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 2b94d3b..eabcf0c 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.4.2" +version = "3.4.3-dev" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index bb96c18..aac0fe3 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.4.2" +version = "3.4.3-dev" @init def initialize(project): diff --git a/package.json b/package.json index 4da6ad8..de5b64a 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.4.2", + "version": "3.4.3-SNAPSHOT", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index 4fa50f4..b4dfdb7 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.2" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.3-SNAPSHOT" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From 6122e9139b59a3575e6bdcd01a58ce7e85f9df3e Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 14:29:07 +0200 Subject: [PATCH 31/58] Lift postgres config from k8s-objects Add merge namespace in let. --- src/main/cljc/dda/c4k_forgejo/core.cljc | 47 +++++++++++++------------ 1 file changed, 25 insertions(+), 22 deletions(-) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index b181dd2..7d42c62 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -33,35 +33,38 @@ (def vol? (s/keys :req-un [::forgejo/volume-total-storage-size])) +(def postgres-config {:db-name "forgejo" + :pv-storage-size-gb 5 + :pvc-storage-class-name "" + :postgres-image "postgres:14" + :postgres-size :2gb}) + (defn k8s-objects [config auth] ; ToDo: ADR for generate functions - vector or no vector? - (let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path)] + (let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path) + resolved-config (merge {:namespace "forgejo"} postgres-config config)] (map yaml/to-string (filter #(not (nil? %)) (cm/concat-vec - (ns/generate (merge {:namespace "forgejo"} config)) - [(postgres/generate-config {:postgres-size :2gb - :db-name "forgejo" - :namespace "forgejo"}) - (postgres/generate-secret auth) - (when (contains? config :postgres-data-volume-path) - (postgres/generate-persistent-volume (select-keys config [:postgres-data-volume-path :pv-storage-size-gb]))) - (postgres/generate-pvc {:pv-storage-size-gb 5 - :pvc-storage-class-name storage-class}) - (postgres/generate-deployment {:postgres-image "postgres:14" - :postgres-size :2gb}) - (postgres/generate-service config) - (forgejo/generate-deployment config) + (ns/generate resolved-config) + [(postgres/generate-config resolved-config) + (postgres/generate-secret {:namespace "forgejo"} auth) + (when (contains? resolved-config :postgres-data-volume-path) + (postgres/generate-persistent-volume (select-keys resolved-config [:postgres-data-volume-path :pv-storage-size-gb]))) + (postgres/generate-pvc (merge resolved-config {:pvc-storage-class-name storage-class})) + (postgres/generate-deployment resolved-config) + (postgres/generate-service resolved-config) + (forgejo/generate-deployment resolved-config) (forgejo/generate-service) (forgejo/generate-service-ssh) - (forgejo/generate-data-volume config) - (forgejo/generate-appini-env config) + (forgejo/generate-data-volume resolved-config) + (forgejo/generate-appini-env resolved-config) (forgejo/generate-secrets auth) (forgejo/generate-rate-limit-middleware rate-limit-defaults)] ; this does not have a vector as output - (forgejo/generate-rate-limit-ingress-and-cert (merge {:namespace "forgejo"} config)) ; this function has a vector as output - (when (contains? config :restic-repository) - [(backup/generate-config config) + (forgejo/generate-rate-limit-ingress-and-cert resolved-config) ; this function has a vector as output + (when (contains? resolved-config :restic-repository) + [(backup/generate-config resolved-config) (backup/generate-secret auth) (backup/generate-cron) - (backup/generate-backup-restore-deployment config)]) - (when (:contains? config :mon-cfg) - (mon/generate (:mon-cfg config) (:mon-auth auth)))))))) + (backup/generate-backup-restore-deployment resolved-config)]) + (when (:contains? resolved-config :mon-cfg) + (mon/generate (:mon-cfg resolved-config) (:mon-auth auth)))))))) From 7ccdf13af8e8dbc614dfabdbb64596271b2ce525 Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 14:51:47 +0200 Subject: [PATCH 32/58] release: 3.4.3 --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index eabcf0c..20eed90 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.4.3-dev" +version = "3.4.3" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index aac0fe3..7d8e7b9 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.4.3-dev" +version = "3.4.3" @init def initialize(project): diff --git a/package.json b/package.json index de5b64a..d2ce703 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.4.3-SNAPSHOT", + "version": "3.4.3", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index b4dfdb7..475007d 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.3-SNAPSHOT" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.3" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From 01914f8d16128d5bc83032af86414b8006c6331c Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 14:51:47 +0200 Subject: [PATCH 33/58] bump version to: 3.4.4-SNAPSHOT --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 20eed90..4503c5f 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.4.3" +version = "3.4.4-dev" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index 7d8e7b9..8001247 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.4.3" +version = "3.4.4-dev" @init def initialize(project): diff --git a/package.json b/package.json index d2ce703..814ba93 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.4.3", + "version": "3.4.4-SNAPSHOT", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index 475007d..a08839f 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.3" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.4-SNAPSHOT" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From ecbe0feae45a5515a59ebb5ea338f2f49b649b19 Mon Sep 17 00:00:00 2001 From: patdyn Date: Tue, 9 Jul 2024 16:22:45 +0200 Subject: [PATCH 34/58] [Skip-CI] Add todos --- src/main/cljc/dda/c4k_forgejo/core.cljc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index 7d42c62..0283c02 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -59,6 +59,8 @@ (forgejo/generate-data-volume resolved-config) (forgejo/generate-appini-env resolved-config) (forgejo/generate-secrets auth) + ; TODO: generate-rate-limit-middleware does not use c4k-common -> refactor this + ; TODO: generate-rate-limit-ingress-and-cert should probably use cm/generate-simple-ingress (forgejo/generate-rate-limit-middleware rate-limit-defaults)] ; this does not have a vector as output (forgejo/generate-rate-limit-ingress-and-cert resolved-config) ; this function has a vector as output (when (contains? resolved-config :restic-repository) From ba649f4c2809bc6790c7154f2241477e33bdc8c4 Mon Sep 17 00:00:00 2001 From: patdyn Date: Wed, 10 Jul 2024 09:51:32 +0200 Subject: [PATCH 35/58] Use ratelimit from common --- src/main/cljc/dda/c4k_forgejo/core.cljc | 7 +--- src/main/cljc/dda/c4k_forgejo/forgejo.cljc | 38 +++++-------------- .../forgejo/middleware-ratelimit.yaml | 9 ----- .../cljc/dda/c4k_forgejo/forgejo_test.cljc | 23 ----------- 4 files changed, 12 insertions(+), 65 deletions(-) delete mode 100644 src/main/resources/forgejo/middleware-ratelimit.yaml diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index 0283c02..9b39ce1 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -58,11 +58,8 @@ (forgejo/generate-service-ssh) (forgejo/generate-data-volume resolved-config) (forgejo/generate-appini-env resolved-config) - (forgejo/generate-secrets auth) - ; TODO: generate-rate-limit-middleware does not use c4k-common -> refactor this - ; TODO: generate-rate-limit-ingress-and-cert should probably use cm/generate-simple-ingress - (forgejo/generate-rate-limit-middleware rate-limit-defaults)] ; this does not have a vector as output - (forgejo/generate-rate-limit-ingress-and-cert resolved-config) ; this function has a vector as output + (forgejo/generate-secrets auth)] ; this does not have a vector as output + (forgejo/generate-ratelimit-ingress-and-cert resolved-config) ; this function has a vector as output (when (contains? resolved-config :restic-repository) [(backup/generate-config resolved-config) (backup/generate-secret auth) diff --git a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc index 10f35f4..7d1c86a 100644 --- a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc +++ b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc @@ -126,35 +126,17 @@ (cm/replace-all-matching "MAILERUSER" (b64/encode mailer-user)) (cm/replace-all-matching "MAILERPW" (b64/encode mailer-pw))))) -(defn generate-ingress-and-cert - [config] - (let [{:keys [fqdn]} config] - (ing/generate-ingress-and-cert - (merge - {:service-name "forgejo-service" - :service-port 3000 - :fqdns [fqdn]} - config)))) - -(defn-spec generate-rate-limit-ingress-and-cert pred/map-or-seq? +(defn-spec generate-ratelimit-ingress-and-cert seq? [config config?] - (-> - (generate-ingress-and-cert config) ; returns a vector - (#(assoc-in % ; Attention: heavily relying on the output order of ing/generate-ingress-and-cert - [1 :metadata :annotations :traefik.ingress.kubernetes.io/router.middlewares] - (str - (-> (second %) :metadata :annotations :traefik.ingress.kubernetes.io/router.middlewares) - ", default-ratelimit@kubernetescrd"))))) - - -; using :average and :burst seems sensible, :period may be interesting for fine tuning later on -(defn-spec generate-rate-limit-middleware pred/map-or-seq? - [config rate-limit-config?] - (let [{:keys [max-rate max-concurrent-requests]} config] - (-> - (yaml/load-as-edn "forgejo/middleware-ratelimit.yaml") - (cm/replace-key-value :average max-rate) - (cm/replace-key-value :burst max-concurrent-requests)))) + (let [{:keys [fqdn max-rate max-concurrent-requests namespace]} config] + (ing/generate-simple-ingress (merge + {:service-name "forgejo-service" + :service-port 3000 + :fqdns [fqdn] + :average-rate max-rate + :burst-rate max-concurrent-requests + :namespace namespace} + config)))) (defn-spec generate-data-volume pred/map-or-seq? [config vol?] diff --git a/src/main/resources/forgejo/middleware-ratelimit.yaml b/src/main/resources/forgejo/middleware-ratelimit.yaml deleted file mode 100644 index 918dd95..0000000 --- a/src/main/resources/forgejo/middleware-ratelimit.yaml +++ /dev/null @@ -1,9 +0,0 @@ -apiVersion: traefik.containo.us/v1alpha1 -kind: Middleware -metadata: - name: ratelimit - namespace: forgejo -spec: - rateLimit: # Config options for rate limiting: https://doc.traefik.io/traefik/middlewares/http/ratelimit/ - average: AVG - burst: BRS \ No newline at end of file diff --git a/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc b/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc index e8837bd..51f3021 100644 --- a/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc +++ b/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc @@ -163,26 +163,3 @@ :storage-c2 "15Gi"} (th/map-diff (cut/generate-data-volume {:volume-total-storage-size 1}) (cut/generate-data-volume {:volume-total-storage-size 15}))))) - -(deftest should-generate-middleware-ratelimit - (is (= {:apiVersion "traefik.containo.us/v1alpha1", - :kind "Middleware", - :metadata {:name "ratelimit", :namespace "forgejo"}, - :spec {:rateLimit {:average 10, :burst 5}}} - (cut/generate-rate-limit-middleware {:max-rate 10, :max-concurrent-requests 5})))) - -(deftest should-generate-middleware-ratelimit-ingress-and-cert - (is (= {:traefik.ingress.kubernetes.io/router.entrypoints "web, websecure", - :traefik.ingress.kubernetes.io/router.middlewares - "default-redirect-https@kubernetescrd, default-ratelimit@kubernetescrd", - :metallb.universe.tf/address-pool "public"} - (-> (second - (cut/generate-rate-limit-ingress-and-cert - {:fqdn "test.de" - :mailer-from "" - :mailer-host "m.t.de" - :mailer-port "123" - :service-noreply-address "" - :average 10 - :burst 5})) - :metadata :annotations)))) From 786c06cc0ad10a7dbc672492b0a53aa54059955c Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 10 Jul 2024 10:02:16 +0200 Subject: [PATCH 36/58] release: 3.4.4 --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 4503c5f..46e2429 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.4.4-dev" +version = "3.4.4" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index 8001247..b0ca20d 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.4.4-dev" +version = "3.4.4" @init def initialize(project): diff --git a/package.json b/package.json index 814ba93..91aeb98 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.4.4-SNAPSHOT", + "version": "3.4.4", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index a08839f..4de460c 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.4-SNAPSHOT" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.4" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From 11123e253f420401152524982dfa7c07c2cf8b07 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 10 Jul 2024 10:02:16 +0200 Subject: [PATCH 37/58] bump version to: 3.4.5-SNAPSHOT --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 46e2429..2a43c08 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.4.4" +version = "3.4.5-dev" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index b0ca20d..30cefe9 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.4.4" +version = "3.4.5-dev" @init def initialize(project): diff --git a/package.json b/package.json index 91aeb98..e4039aa 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.4.4", + "version": "3.4.5-SNAPSHOT", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index 4de460c..ed3592e 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.4" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.5-SNAPSHOT" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From d5d4dd5b4300d159f0f60bed85af8ec60a5c8330 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 17 Jul 2024 08:29:14 +0200 Subject: [PATCH 38/58] fix -v option --- project.clj | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/project.clj b/project.clj index ed3592e..39bdde4 100644 --- a/project.clj +++ b/project.clj @@ -10,7 +10,8 @@ :target-path "target/%s/" :source-paths ["src/main/cljc" "src/main/clj"] - :resource-paths ["src/main/resources"] + :resource-paths ["src/main/resources" + "project.clj"] :repositories [["snapshots" :clojars] ["releases" :clojars]] :deploy-repositories [["snapshots" {:sign-releases false :url "https://clojars.org/repo"}] From d3dd3ca5efbbf2b85e7894aa315edd6b3d363219 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 17 Jul 2024 14:18:08 +0200 Subject: [PATCH 39/58] split auth and config --- project.clj | 2 +- src/main/clj/dda/c4k_forgejo/uberjar.clj | 5 ++- src/main/cljc/dda/c4k_forgejo/core.cljc | 51 +++++++++++++--------- src/main/cljs/dda/c4k_forgejo/browser.cljs | 36 ++++++++------- 4 files changed, 55 insertions(+), 39 deletions(-) diff --git a/project.clj b/project.clj index 39bdde4..a2fe8aa 100644 --- a/project.clj +++ b/project.clj @@ -5,7 +5,7 @@ :url "https://www.apache.org/licenses/LICENSE-2.0.html"} :dependencies [[org.clojure/clojure "1.11.3" :scope "provided"] [org.clojure/tools.reader "1.4.2"] - [org.domaindrivenarchitecture/c4k-common-clj "6.4.1"] + [org.domaindrivenarchitecture/c4k-common-clj "6.4.2-SNAPSHOT"] ; TODO: Release version with refactorings and update here [hickory "0.7.1" :exclusions [viebel/codox-klipse-theme]]] :target-path "target/%s/" :source-paths ["src/main/cljc" diff --git a/src/main/clj/dda/c4k_forgejo/uberjar.clj b/src/main/clj/dda/c4k_forgejo/uberjar.clj index 14338d6..42b2098 100644 --- a/src/main/clj/dda/c4k_forgejo/uberjar.clj +++ b/src/main/clj/dda/c4k_forgejo/uberjar.clj @@ -7,10 +7,11 @@ (set! *warn-on-reflection* true) (defn -main [& cmd-args] - (uberjar/main-common + (uberjar/main-cm "c4k-forgejo" core/config? core/auth? core/config-defaults - core/k8s-objects + core/config-objects + core/auth-objects cmd-args)) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index 9b39ce1..ca0c3a5 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -1,25 +1,25 @@ (ns dda.c4k-forgejo.core - (:require - [clojure.spec.alpha :as s] - [dda.c4k-common.yaml :as yaml] - [dda.c4k-common.common :as cm] - [dda.c4k-common.monitoring :as mon] - [dda.c4k-forgejo.forgejo :as forgejo] - [dda.c4k-forgejo.backup :as backup] - [dda.c4k-common.postgres :as postgres] - [dda.c4k-common.namespace :as ns])) + (:require + [clojure.spec.alpha :as s] + [dda.c4k-common.yaml :as yaml] + [dda.c4k-common.common :as cm] + [dda.c4k-common.monitoring :as mon] + [dda.c4k-forgejo.forgejo :as forgejo] + [dda.c4k-forgejo.backup :as backup] + [dda.c4k-common.postgres :as postgres] + [dda.c4k-common.namespace :as ns])) (def config-defaults {:issuer "staging", :deploy-federated "false"}) (def rate-limit-defaults {:max-rate 10, :max-concurrent-requests 5}) -(def config? (s/keys :req-un [::forgejo/fqdn - ::forgejo/mailer-from - ::forgejo/mailer-host +(def config? (s/keys :req-un [::forgejo/fqdn + ::forgejo/mailer-from + ::forgejo/mailer-host ::forgejo/mailer-port ::forgejo/service-noreply-address] - :opt-un [::forgejo/issuer + :opt-un [::forgejo/issuer ::forgejo/deploy-federated - ::forgejo/default-app-name + ::forgejo/default-app-name ::forgejo/service-domain-whitelist ::forgejo/forgejo-image-version-overwrite ::backup/restic-repository @@ -39,7 +39,7 @@ :postgres-image "postgres:14" :postgres-size :2gb}) -(defn k8s-objects [config auth] ; ToDo: ADR for generate functions - vector or no vector? +(defn config-objects [config] ; ToDo: ADR for generate functions - vector or no vector? (let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path) resolved-config (merge {:namespace "forgejo"} postgres-config config)] (map yaml/to-string @@ -47,7 +47,6 @@ (cm/concat-vec (ns/generate resolved-config) [(postgres/generate-config resolved-config) - (postgres/generate-secret {:namespace "forgejo"} auth) (when (contains? resolved-config :postgres-data-volume-path) (postgres/generate-persistent-volume (select-keys resolved-config [:postgres-data-volume-path :pv-storage-size-gb]))) (postgres/generate-pvc (merge resolved-config {:pvc-storage-class-name storage-class})) @@ -57,13 +56,25 @@ (forgejo/generate-service) (forgejo/generate-service-ssh) (forgejo/generate-data-volume resolved-config) - (forgejo/generate-appini-env resolved-config) - (forgejo/generate-secrets auth)] ; this does not have a vector as output + (forgejo/generate-appini-env resolved-config)] (forgejo/generate-ratelimit-ingress-and-cert resolved-config) ; this function has a vector as output (when (contains? resolved-config :restic-repository) [(backup/generate-config resolved-config) - (backup/generate-secret auth) (backup/generate-cron) (backup/generate-backup-restore-deployment resolved-config)]) (when (:contains? resolved-config :mon-cfg) - (mon/generate (:mon-cfg resolved-config) (:mon-auth auth)))))))) + (mon/generate-config))))))) + +(defn auth-objects [config auth] ; ToDo: ADR for generate functions - vector or no vector? + (let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path) + resolved-config (merge {:namespace "forgejo"} postgres-config config)] + (map yaml/to-string + (filter #(not (nil? %)) + (cm/concat-vec + (ns/generate resolved-config) + [(postgres/generate-secret {:namespace "forgejo"} auth) + (forgejo/generate-secrets auth)] + (when (contains? resolved-config :restic-repository) + [(backup/generate-secret auth)]) + (when (:contains? resolved-config :mon-cfg) + (mon/generate-auth (:mon-cfg resolved-config) (:mon-auth auth)))))))) diff --git a/src/main/cljs/dda/c4k_forgejo/browser.cljs b/src/main/cljs/dda/c4k_forgejo/browser.cljs index 70caf09..ba59420 100644 --- a/src/main/cljs/dda/c4k_forgejo/browser.cljs +++ b/src/main/cljs/dda/c4k_forgejo/browser.cljs @@ -4,7 +4,7 @@ [clojure.tools.reader.edn :as edn] [dda.c4k-forgejo.core :as core] [dda.c4k-forgejo.forgejo :as forgejo] - [dda.c4k-common.browser :as br] + [dda.c4k-common.browser :as br] [dda.c4k-common.common :as cm])) (defn generate-group @@ -73,14 +73,13 @@ :mailer-host (br/get-content-from-element "mailer-host") :mailer-port (br/get-content-from-element "mailer-port") :service-noreply-address (br/get-content-from-element "service-noreply-address") - :volume-total-storage-size (br/get-content-from-element "volume-total-storage-size" :deserializer js/parseInt)} + :volume-total-storage-size (br/get-content-from-element "volume-total-storage-size" :deserializer js/parseInt)} (when (not (st/blank? issuer)) {:issuer issuer}) (when (not (st/blank? app-name)) {:default-app-name app-name}) (when (not (st/blank? domain-whitelist)) - {:service-domain-whitelist domain-whitelist}) - ))) + {:service-domain-whitelist domain-whitelist})))) (defn validate-all! [] (br/validate! "fqdn" ::forgejo/fqdn) @@ -91,7 +90,7 @@ (br/validate! "deploy-federated" ::forgejo/deploy-federated :optional true) (br/validate! "issuer" ::forgejo/issuer :optional true) (br/validate! "app-name" ::forgejo/default-app-name :optional true) - (br/validate! "domain-whitelist" ::forgejo/service-domain-whitelist :optional true) + (br/validate! "domain-whitelist" ::forgejo/service-domain-whitelist :optional true) (br/validate! "volume-total-storage-size" ::forgejo/volume-total-storage-size :deserializer js/parseInt) (br/validate! "auth" forgejo/auth? :deserializer edn/read-string) (br/set-form-validated!)) @@ -103,16 +102,21 @@ (defn init [] (br/append-hickory (generate-content-div)) - (-> js/document - (.getElementById "generate-button") - (.addEventListener "click" - #(do (validate-all!) - (-> (cm/generate-common - (config-from-document) - (br/get-content-from-element "auth" :deserializer edn/read-string) - core/config-defaults - core/k8s-objects) - (br/set-output!))))) + (let [config-only false + auth-only false] + (-> js/document + (.getElementById "generate-button") + (.addEventListener "click" + #(do (validate-all!) + (-> (cm/generate-cm + (config-from-document) + (br/get-content-from-element "auth" :deserializer edn/read-string) + core/config-defaults + core/config-objects + core/auth-objects + config-only + auth-only) + (br/set-output!)))))) (add-validate-listener "fqdn") (add-validate-listener "deploy-federated") (add-validate-listener "mailer-from") @@ -120,7 +124,7 @@ (add-validate-listener "mailer-port") (add-validate-listener "service-noreply-address") (add-validate-listener "app-name") - (add-validate-listener "domain-whitelist") + (add-validate-listener "domain-whitelist") (add-validate-listener "volume-total-storage-size") (add-validate-listener "issuer") (add-validate-listener "auth")) \ No newline at end of file From 0055eb343533c6f972a3c62bece1e1ef0db3e432 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 17 Jul 2024 15:33:30 +0200 Subject: [PATCH 40/58] adjusted configs --- src/main/cljc/dda/c4k_forgejo/core.cljc | 73 ++++++++++++------------- 1 file changed, 36 insertions(+), 37 deletions(-) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index ca0c3a5..3a895ac 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -9,7 +9,14 @@ [dda.c4k-common.postgres :as postgres] [dda.c4k-common.namespace :as ns])) -(def config-defaults {:issuer "staging", :deploy-federated "false"}) +(def config-defaults {:namespace "forgejo" + :issuer "staging" + :deploy-federated "false" + :db-name "forgejo" + :pv-storage-size-gb 5 + :pvc-storage-class-name "" + :postgres-image "postgres:14" + :postgres-size :2gb}) (def rate-limit-defaults {:max-rate 10, :max-concurrent-requests 5}) (def config? (s/keys :req-un [::forgejo/fqdn @@ -33,48 +40,40 @@ (def vol? (s/keys :req-un [::forgejo/volume-total-storage-size])) -(def postgres-config {:db-name "forgejo" - :pv-storage-size-gb 5 - :pvc-storage-class-name "" - :postgres-image "postgres:14" - :postgres-size :2gb}) - (defn config-objects [config] ; ToDo: ADR for generate functions - vector or no vector? - (let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path) - resolved-config (merge {:namespace "forgejo"} postgres-config config)] + (let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path)] (map yaml/to-string (filter #(not (nil? %)) (cm/concat-vec - (ns/generate resolved-config) - [(postgres/generate-config resolved-config) - (when (contains? resolved-config :postgres-data-volume-path) - (postgres/generate-persistent-volume (select-keys resolved-config [:postgres-data-volume-path :pv-storage-size-gb]))) - (postgres/generate-pvc (merge resolved-config {:pvc-storage-class-name storage-class})) - (postgres/generate-deployment resolved-config) - (postgres/generate-service resolved-config) - (forgejo/generate-deployment resolved-config) + (ns/generate config) + [(postgres/generate-config-configmap config) + (when (contains? config :postgres-data-volume-path) + (postgres/generate-persistent-volume (select-keys config [:postgres-data-volume-path :pv-storage-size-gb]))) + (postgres/generate-pvc (merge config {:pvc-storage-class-name storage-class})) + (postgres/generate-deployment config) + (postgres/generate-service config) + (forgejo/generate-deployment config) (forgejo/generate-service) (forgejo/generate-service-ssh) - (forgejo/generate-data-volume resolved-config) - (forgejo/generate-appini-env resolved-config)] - (forgejo/generate-ratelimit-ingress-and-cert resolved-config) ; this function has a vector as output - (when (contains? resolved-config :restic-repository) - [(backup/generate-config resolved-config) + (forgejo/generate-data-volume config) + (forgejo/generate-appini-env config)] + (forgejo/generate-ratelimit-ingress-and-cert config) ; this function has a vector as output + (when (contains? config :restic-repository) + [(backup/generate-config config) (backup/generate-cron) - (backup/generate-backup-restore-deployment resolved-config)]) - (when (:contains? resolved-config :mon-cfg) + (backup/generate-backup-restore-deployment config)]) + (when (contains? config :mon-cfg) (mon/generate-config))))))) -(defn auth-objects [config auth] ; ToDo: ADR for generate functions - vector or no vector? - (let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path) - resolved-config (merge {:namespace "forgejo"} postgres-config config)] - (map yaml/to-string - (filter #(not (nil? %)) - (cm/concat-vec - (ns/generate resolved-config) - [(postgres/generate-secret {:namespace "forgejo"} auth) - (forgejo/generate-secrets auth)] - (when (contains? resolved-config :restic-repository) - [(backup/generate-secret auth)]) - (when (:contains? resolved-config :mon-cfg) - (mon/generate-auth (:mon-cfg resolved-config) (:mon-auth auth)))))))) +; REVIEW gec: In general, how do we handle config and auth for auth-objects? +(defn auth-objects [config] ; ToDo: ADR for generate functions - vector or no vector? + (map yaml/to-string + (filter #(not (nil? %)) + (cm/concat-vec + (ns/generate config) + [(postgres/generate-secret config config) ; "config config" seems not right + (forgejo/generate-secrets config)] + (when (contains? config :restic-repository) + [(backup/generate-secret config)]) + (when (contains? config :mon-cfg) + (mon/generate-auth (:mon-cfg config) (:mon-auth config))))))) ; Here also "config config" seems not right From 2c3a03108187223419fed5b796d34a5caa7a9304 Mon Sep 17 00:00:00 2001 From: Clemens Date: Thu, 18 Jul 2024 08:55:00 +0200 Subject: [PATCH 41/58] adjust auth-objects signature --- src/main/cljc/dda/c4k_forgejo/core.cljc | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index 3a895ac..e893af2 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -65,15 +65,14 @@ (when (contains? config :mon-cfg) (mon/generate-config))))))) -; REVIEW gec: In general, how do we handle config and auth for auth-objects? -(defn auth-objects [config] ; ToDo: ADR for generate functions - vector or no vector? +(defn auth-objects [config auth] (map yaml/to-string (filter #(not (nil? %)) (cm/concat-vec (ns/generate config) - [(postgres/generate-secret config config) ; "config config" seems not right + [(postgres/generate-secret config auth) (forgejo/generate-secrets config)] (when (contains? config :restic-repository) [(backup/generate-secret config)]) (when (contains? config :mon-cfg) - (mon/generate-auth (:mon-cfg config) (:mon-auth config))))))) ; Here also "config config" seems not right + (mon/generate-auth (:mon-cfg config) (:mon-auth auth))))))) From 0d13edc8d3a1e42ff4409a2e7079ca37d7ac0fba Mon Sep 17 00:00:00 2001 From: Clemens Date: Thu, 18 Jul 2024 09:31:42 +0200 Subject: [PATCH 42/58] fix auth calls --- src/main/cljc/dda/c4k_forgejo/core.cljc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index e893af2..999a0a5 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -71,8 +71,8 @@ (cm/concat-vec (ns/generate config) [(postgres/generate-secret config auth) - (forgejo/generate-secrets config)] + (forgejo/generate-secrets auth)] (when (contains? config :restic-repository) - [(backup/generate-secret config)]) + [(backup/generate-secret auth)]) (when (contains? config :mon-cfg) (mon/generate-auth (:mon-cfg config) (:mon-auth auth))))))) From 3b10016fae99aa8c07db00c2ac795e17d246b674 Mon Sep 17 00:00:00 2001 From: Clemens Date: Thu, 18 Jul 2024 09:58:35 +0200 Subject: [PATCH 43/58] added todo --- shadow-cljs.edn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/shadow-cljs.edn b/shadow-cljs.edn index 6b45f88..cce08cf 100644 --- a/shadow-cljs.edn +++ b/shadow-cljs.edn @@ -4,7 +4,7 @@ "src/test/cljc" "src/test/cljs" "src/test/resources"] - :dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "6.4.1"] + :dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "6.4.1"] ; TODO: Release version with refactorings and update here [hickory "0.7.1"]] :builds {:frontend {:target :browser :modules {:main {:init-fn dda.c4k-forgejo.browser/init}} From c5fcec4985ce50a3c1c5ec80735267c6f6524a83 Mon Sep 17 00:00:00 2001 From: Clemens Date: Fri, 19 Jul 2024 10:59:32 +0200 Subject: [PATCH 44/58] adjust postgres function call --- src/main/cljc/dda/c4k_forgejo/core.cljc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index 999a0a5..5614ed4 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -46,7 +46,7 @@ (filter #(not (nil? %)) (cm/concat-vec (ns/generate config) - [(postgres/generate-config-configmap config) + [(postgres/generate-configmap config) (when (contains? config :postgres-data-volume-path) (postgres/generate-persistent-volume (select-keys config [:postgres-data-volume-path :pv-storage-size-gb]))) (postgres/generate-pvc (merge config {:pvc-storage-class-name storage-class})) From 97dace2030381e4a9ade773ce497fb9cc57f6d99 Mon Sep 17 00:00:00 2001 From: Clemens Date: Fri, 19 Jul 2024 11:27:03 +0200 Subject: [PATCH 45/58] updated deps --- project.clj | 2 +- shadow-cljs.edn | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/project.clj b/project.clj index a2fe8aa..4e65cf5 100644 --- a/project.clj +++ b/project.clj @@ -5,7 +5,7 @@ :url "https://www.apache.org/licenses/LICENSE-2.0.html"} :dependencies [[org.clojure/clojure "1.11.3" :scope "provided"] [org.clojure/tools.reader "1.4.2"] - [org.domaindrivenarchitecture/c4k-common-clj "6.4.2-SNAPSHOT"] ; TODO: Release version with refactorings and update here + [org.domaindrivenarchitecture/c4k-common-clj "7.0.0"] [hickory "0.7.1" :exclusions [viebel/codox-klipse-theme]]] :target-path "target/%s/" :source-paths ["src/main/cljc" diff --git a/shadow-cljs.edn b/shadow-cljs.edn index cce08cf..aafbb0b 100644 --- a/shadow-cljs.edn +++ b/shadow-cljs.edn @@ -4,7 +4,7 @@ "src/test/cljc" "src/test/cljs" "src/test/resources"] - :dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "6.4.1"] ; TODO: Release version with refactorings and update here + :dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "7.0.0"] [hickory "0.7.1"]] :builds {:frontend {:target :browser :modules {:main {:init-fn dda.c4k-forgejo.browser/init}} From 670a45966d4e8c964fbe246eaa762ae45fed7c66 Mon Sep 17 00:00:00 2001 From: patdyn Date: Wed, 10 Jul 2024 14:00:31 +0200 Subject: [PATCH 46/58] [Skip-CI] Add Analytics doc --- doc/Runbook_UpgradeFrom1.19To7.0.5.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/doc/Runbook_UpgradeFrom1.19To7.0.5.md b/doc/Runbook_UpgradeFrom1.19To7.0.5.md index 19ca6c2..4816595 100644 --- a/doc/Runbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Runbook_UpgradeFrom1.19To7.0.5.md @@ -85,3 +85,23 @@ In the logs the following error can be found. This will be resolved automaticall ``` 2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21 ``` + +# Add Shynet Analytics + +1. Log into shynet & create new Service + 1. Copy the generated html snippet and save it somewhere you remember +1. SSH into prod server +1. Make the necessary folders and files in forgejo data dir: + 1. `kubectl exec -n forgejo -it forgejo-... -- bash` + 1. `mkdir -p /data/gitea/templates/custom` + 1. `touch /data/gitea/templates/custom/footer.tmpl` +1. Open the `footer.tmpl` and paste the saved snippet +1. Restart the pod + 1. `k scale -n forgejo deployment forgejo --replicas=0` + 1. `k scale -n forgejo deployment forgejo --replicas=1` +1. Add Information about analytics: Clone Datenschutz Repo + 1. `git clone ssh://git@repo.prod.meissa.de:2222/meissa/Datenschutz.git` +1. Merge forgejo-upgrade into main + 1. `git merge forgejo-upgrade` +1. Push to origin + 1. `git push` From a66f398d71d0a962c403c74f32119343f396cdc5 Mon Sep 17 00:00:00 2001 From: Clemens Date: Tue, 9 Jul 2024 11:10:40 +0200 Subject: [PATCH 47/58] updated to forgejo version 7.0 --- src/main/cljc/dda/c4k_forgejo/forgejo.cljc | 2 +- src/main/resources/forgejo/appini-env-configmap.yaml | 7 +++---- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc index 7d1c86a..06e14f6 100644 --- a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc +++ b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc @@ -71,7 +71,7 @@ (def federated-image-name "domaindrivenarchitecture/c4k-forgejo-federated") (def federated-image-version "latest") (def non-federated-image-name "codeberg.org/forgejo/forgejo") -(def non-federated-image-version "1.19") +(def non-federated-image-version "7.0") (defn-spec generate-image-str string? [config config?] diff --git a/src/main/resources/forgejo/appini-env-configmap.yaml b/src/main/resources/forgejo/appini-env-configmap.yaml index 348a62a..247d2c2 100644 --- a/src/main/resources/forgejo/appini-env-configmap.yaml +++ b/src/main/resources/forgejo/appini-env-configmap.yaml @@ -16,7 +16,6 @@ data: FORGEJO__database__NAME: forgejo FORGEJO__database__LOG_SQL: "false" FORGEJO__database__SSL_MODE: disable - FORGEJO__database__CHARSET: utf8 #[DEFAULT] APP_NAME: APPNAME @@ -37,12 +36,12 @@ data: #[mailer] FORGEJO__mailer__ENABLED: "true" FORGEJO__mailer__FROM: FROM - FORGEJO__mailer__MAILER_TYPE: smtp+startls + FORGEJO__mailer__PROTOCOL: smtp+starttls FORGEJO__mailer__SMTP_ADDR: MAILERHOST FORGEJO__mailer__SMTP_PORT: MAILERPORT #[oauth2] - FORGEJO__oauth2__ENABLE: "true" + FORGEJO__oauth2__ENABLED: "true" #[openid] FORGEJO__openid__ENABLE_OPENID: "true" @@ -76,7 +75,7 @@ data: FORGEJO__service__REQUIRE_SIGNIN_VIEW: "false" FORGEJO__service__REGISTER_EMAIL_CONFIRM: "true" FORGEJO__service__ENABLE_NOTIFY_MAIL: "true" - FORGEJO__service__EMAIL_DOMAIN_WHITELIST: WHITELISTDOMAINS + FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: WHITELISTDOMAINS FORGEJO__service__ALLOW_ONLY_EXTERNAL_REGISTRATION: "false" FORGEJO__service__ENABLE_BASIC_AUTHENTICATION: "true" FORGEJO__service__ENABLE_CAPTCHA: "false" From 3f0ce02da3c69e619714b4b822503582bb291be7 Mon Sep 17 00:00:00 2001 From: Clemens Date: Tue, 9 Jul 2024 11:26:39 +0200 Subject: [PATCH 48/58] Added option for dedicated federation-enables and fixed tests --- src/main/cljc/dda/c4k_forgejo/core.cljc | 4 ++- src/main/cljc/dda/c4k_forgejo/forgejo.cljc | 30 +++++++++---------- .../cljc/dda/c4k_forgejo/forgejo_test.cljc | 12 ++++---- 3 files changed, 24 insertions(+), 22 deletions(-) diff --git a/src/main/cljc/dda/c4k_forgejo/core.cljc b/src/main/cljc/dda/c4k_forgejo/core.cljc index 5614ed4..303cbc3 100644 --- a/src/main/cljc/dda/c4k_forgejo/core.cljc +++ b/src/main/cljc/dda/c4k_forgejo/core.cljc @@ -12,6 +12,7 @@ (def config-defaults {:namespace "forgejo" :issuer "staging" :deploy-federated "false" + :federation-enabled "false" :db-name "forgejo" :pv-storage-size-gb 5 :pvc-storage-class-name "" @@ -26,7 +27,8 @@ ::forgejo/service-noreply-address] :opt-un [::forgejo/issuer ::forgejo/deploy-federated - ::forgejo/default-app-name + ::forgejo/federation-enabled + ::forgejo/default-app-name ::forgejo/service-domain-whitelist ::forgejo/forgejo-image-version-overwrite ::backup/restic-repository diff --git a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc index 06e14f6..abda99c 100644 --- a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc +++ b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc @@ -33,6 +33,7 @@ (s/def ::default-app-name string?) (s/def ::fqdn pred/fqdn-string?) (s/def ::deploy-federated boolean-string?) +(s/def ::federation-enabled boolean-string?) (s/def ::mailer-from pred/bash-env-string?) (s/def ::mailer-host pred/bash-env-string?) (s/def ::mailer-port pred/bash-env-string?) @@ -53,6 +54,7 @@ ::service-noreply-address] :opt-un [::issuer ::deploy-federated + ::federation-enabled ::default-app-name ::service-domain-whitelist ::forgejo-image-version-overwrite])) @@ -88,7 +90,7 @@ (defn generate-appini-env [config] (let [{:keys [default-app-name - deploy-federated + federation-enabled fqdn mailer-from mailer-host @@ -97,19 +99,19 @@ service-noreply-address] :or {default-app-name "forgejo instance" service-domain-whitelist fqdn}} config - deploy-federated-bool (boolean-from-string deploy-federated)] + federation-enabled-bool (boolean-from-string federation-enabled)] (-> (yaml/load-as-edn "forgejo/appini-env-configmap.yaml") - (cm/replace-all-matching "APPNAME" default-app-name) - (cm/replace-all-matching "FQDN" fqdn) - (cm/replace-all-matching "URL" (str "https://" fqdn)) - (cm/replace-all-matching "FROM" mailer-from) - (cm/replace-all-matching "MAILERHOST" mailer-host) - (cm/replace-all-matching "MAILERPORT" mailer-port) - (cm/replace-all-matching "WHITELISTDOMAINS" service-domain-whitelist) - (cm/replace-all-matching "NOREPLY" service-noreply-address) - (cm/replace-all-matching "IS_FEDERATED" - (if deploy-federated-bool + (cm/replace-all-matching-values-by-new-value "APPNAME" default-app-name) + (cm/replace-all-matching-values-by-new-value "FQDN" fqdn) + (cm/replace-all-matching-values-by-new-value "URL" (str "https://" fqdn)) + (cm/replace-all-matching-values-by-new-value "FROM" mailer-from) + (cm/replace-all-matching-values-by-new-value "MAILERHOST" mailer-host) + (cm/replace-all-matching-values-by-new-value "MAILERPORT" mailer-port) + (cm/replace-all-matching-values-by-new-value "WHITELISTDOMAINS" service-domain-whitelist) + (cm/replace-all-matching-values-by-new-value "NOREPLY" service-noreply-address) + (cm/replace-all-matching-values-by-new-value "IS_FEDERATED" + (if federation-enabled-bool "true" "false"))))) @@ -148,11 +150,9 @@ (defn-spec generate-deployment pred/map-or-seq? [config config?] - (let [{:keys [deploy-federated]} config - deploy-federated-bool (boolean-from-string deploy-federated)] (-> (yaml/load-as-edn "forgejo/deployment.yaml") - (cm/replace-all-matching "IMAGE_NAME" (generate-image-str config))))) + (cm/replace-all-matching "IMAGE_NAME" (generate-image-str config)))) (defn generate-service [] diff --git a/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc b/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc index 51f3021..d7b801b 100644 --- a/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc +++ b/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc @@ -14,7 +14,7 @@ (deftest should-generate-image-str (testing "non-federated-image" - (is (= "codeberg.org/forgejo/forgejo:1.19" + (is (= "codeberg.org/forgejo/forgejo:7.0" (cut/generate-image-str {:fqdn "test.de" :mailer-from "" :mailer-host "m.t.de" @@ -63,12 +63,12 @@ :FORGEJO__server__ROOT_URL-c2 "https://test.com", :FORGEJO__server__SSH_DOMAIN-c1 "test.de", :FORGEJO__server__SSH_DOMAIN-c2 "test.com", - :FORGEJO__service__EMAIL_DOMAIN_WHITELIST-c1 "adb.de", - :FORGEJO__service__EMAIL_DOMAIN_WHITELIST-c2 "test.com,test.net", + :FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST-c1 "adb.de", + :FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST-c2 "test.com,test.net", :FORGEJO__service__NO_REPLY_ADDRESS-c1 "", :FORGEJO__service__NO_REPLY_ADDRESS-c2 "noreply@test.com"} (th/map-diff (cut/generate-appini-env {:default-app-name "" - :deploy-federated "false" + :federation-enabled "false" :fqdn "test.de" :mailer-from "" :mailer-host "m.t.de" @@ -76,7 +76,7 @@ :service-domain-whitelist "adb.de" :service-noreply-address ""}) (cut/generate-appini-env {:default-app-name "test forgejo" - :deploy-federated "true" + :federation-enabled "true" :fqdn "test.com" :mailer-from "test@test.com" :mailer-host "mail.test.com" @@ -97,7 +97,7 @@ :spec {:containers [{:name "forgejo", - :image "codeberg.org/forgejo/forgejo:1.19", + :image "codeberg.org/forgejo/forgejo:7.0", :imagePullPolicy "IfNotPresent", :envFrom [{:configMapRef {:name "forgejo-env"}} {:secretRef {:name "forgejo-secrets"}}], :volumeMounts [{:name "forgejo-data-volume", :mountPath "/data"}], From 6a291d962ae67acf3ec9ab4c8f6fc1512ebb81c7 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 10 Jul 2024 11:39:46 +0200 Subject: [PATCH 49/58] added namespace to runbook commands --- doc/Runbook_UpgradeFrom1.19To7.0.5.md | 58 +++++++++++++-------------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/doc/Runbook_UpgradeFrom1.19To7.0.5.md b/doc/Runbook_UpgradeFrom1.19To7.0.5.md index 4816595..45cbf69 100644 --- a/doc/Runbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Runbook_UpgradeFrom1.19To7.0.5.md @@ -9,70 +9,70 @@ ## Preparations -1. Stop Forgejo Prod: `k scale deployment forgejo --replicas=0` -1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` -1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1` -1. Execute Manual Backup: `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh` +1. Stop Forgejo Prod: `k scale -n forgejo deployment forgejo --replicas=0` +1. Disable Backup Cron: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` +1. Scale up Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1` +1. Execute Manual Backup: `kubectl exec -n forgejo -it backup-restore-... -- /usr/local/bin/backup.sh` ### Create 2nd Repo Prod Server 1. Terraform Preparations for 2nd Server: TODO 1. Install c4k-forgejo Version TODO with config `"forgejo-image-version-overwrite": "1.19.3-0"` -1. Stop Forgejo Deployment: `k scale deployment forgejo --replicas=0` -1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` -1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1` +1. Stop Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Disable Backup Cron: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` +1. Scale up Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1` 1. Restore Forgejo Backup: See [BackupAndRestore.md](BackupAndRestore.md) 1. Check for `..._INSTALL_LOCK: true` in ConfigMap `forgejo-env` -1. Scale up Forgejo Deployment and check for (startup) problems: `k scale deployment forgejo --replicas=1` +1. Scale up Forgejo Deployment and check for (startup) problems: `k scale -n forgejo deployment forgejo --replicas=1` ## Upgrade to 1.20.1-0 -1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` -1. Adjust configmap: `k edit cm forgejo-env` +1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Adjust configmap: `k edit -n forgejo cm forgejo-env` 1. Remove `FORGEJO__database__CHARSET: utf8` (This was a misconfiguration, since this option only had effect for mysql dbs) 1. Change `FORGEJO__mailer__MAILER_TYPE: smtp+startls` TO `FORGEJO__mailer__PROTOCOL: smtp+starttls` (Missed deprecation from 1.19) 1. Change `FORGEJO__service__EMAIL_DOMAIN_WHITELIST: repo.test.meissa.de` TO `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: repo.test.meissa.de` (Fallback deprecation in 1.21) -1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` -1. Set version to `1.20.1-0` with `k edit deployment forgejo` -1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `1.20.1-0` with `k edit -n forgejo deployment forgejo` +1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Check for errors ## Upgrade to 1.21.1-0 -1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` -1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` -1. Set version to `1.21.1-0` with `k edit deployment forgejo` -1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `1.21.1-0` with `k edit -n forgejo deployment forgejo` +1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Check for errors 1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`. ## Upgrade to 7.0.0 -1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` -1. Adjust configmap: `k edit cm forgejo-env` +1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Adjust configmap: `k edit -n forgejo cm forgejo-env` 1. Change `FORGEJO__oauth2__ENABLE: "true"` TO `FORGEJO__oauth2__ENABLED: "true"` -1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` -1. Set version to `7.0.0` with `k edit deployment forgejo` -1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `7.0.0` with `k edit -n forgejo deployment forgejo` +1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Check for errors ## Upgrade to 7.0.5 (no breaking changes) TODO: Upgrade to 8.0.0 instead after Release! -1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` -1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` -1. Set version to `7.0.5` with `k edit deployment forgejo` -1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `7.0.5` with `k edit -n forgejo deployment forgejo` +1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Check for errors ## Post Work 1. Switch DNS to new server -1. Reenable Backup Cron on new server: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : false }}'` -1. Execute manual Backup on new server: `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh` -1. Scale down Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1` +1. Reenable Backup Cron on new server: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : false }}'` +1. Execute manual Backup on new server: `kubectl exec -n forgejo -it backup-restore-... -- /usr/local/bin/backup.sh` +1. Scale down Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1` 1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens! 1. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work. From 5c521e287727e3826e1c6b8085dc0c42fdf63dea Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 31 Jul 2024 10:02:36 +0200 Subject: [PATCH 50/58] Added v8.0.0 upgrade to runbook --- doc/Runbook_UpgradeFrom1.19To7.0.5.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/doc/Runbook_UpgradeFrom1.19To7.0.5.md b/doc/Runbook_UpgradeFrom1.19To7.0.5.md index 4816595..c2b23d9 100644 --- a/doc/Runbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Runbook_UpgradeFrom1.19To7.0.5.md @@ -6,6 +6,7 @@ * 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0 * 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0 * 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0 +* 8.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#8-0-0 ## Preparations @@ -57,9 +58,7 @@ 1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` 1. Check for errors -## Upgrade to 7.0.5 (no breaking changes) - -TODO: Upgrade to 8.0.0 instead after Release! +## Upgrade to 8.0.0 (no relevant breaking changes) 1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` 1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` From 26dba0b756ff350cd9186716e7254cc0291f3ce9 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 10 Jul 2024 11:39:46 +0200 Subject: [PATCH 51/58] added namespace to runbook commands --- doc/Runbook_UpgradeFrom1.19To7.0.5.md | 58 +++++++++++++-------------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/doc/Runbook_UpgradeFrom1.19To7.0.5.md b/doc/Runbook_UpgradeFrom1.19To7.0.5.md index c2b23d9..85d4405 100644 --- a/doc/Runbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Runbook_UpgradeFrom1.19To7.0.5.md @@ -10,68 +10,68 @@ ## Preparations -1. Stop Forgejo Prod: `k scale deployment forgejo --replicas=0` -1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` -1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1` -1. Execute Manual Backup: `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh` +1. Stop Forgejo Prod: `k scale -n forgejo deployment forgejo --replicas=0` +1. Disable Backup Cron: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` +1. Scale up Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1` +1. Execute Manual Backup: `kubectl exec -n forgejo -it backup-restore-... -- /usr/local/bin/backup.sh` ### Create 2nd Repo Prod Server 1. Terraform Preparations for 2nd Server: TODO 1. Install c4k-forgejo Version TODO with config `"forgejo-image-version-overwrite": "1.19.3-0"` -1. Stop Forgejo Deployment: `k scale deployment forgejo --replicas=0` -1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` -1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1` +1. Stop Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Disable Backup Cron: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` +1. Scale up Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1` 1. Restore Forgejo Backup: See [BackupAndRestore.md](BackupAndRestore.md) 1. Check for `..._INSTALL_LOCK: true` in ConfigMap `forgejo-env` -1. Scale up Forgejo Deployment and check for (startup) problems: `k scale deployment forgejo --replicas=1` +1. Scale up Forgejo Deployment and check for (startup) problems: `k scale -n forgejo deployment forgejo --replicas=1` ## Upgrade to 1.20.1-0 -1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` -1. Adjust configmap: `k edit cm forgejo-env` +1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Adjust configmap: `k edit -n forgejo cm forgejo-env` 1. Remove `FORGEJO__database__CHARSET: utf8` (This was a misconfiguration, since this option only had effect for mysql dbs) 1. Change `FORGEJO__mailer__MAILER_TYPE: smtp+startls` TO `FORGEJO__mailer__PROTOCOL: smtp+starttls` (Missed deprecation from 1.19) 1. Change `FORGEJO__service__EMAIL_DOMAIN_WHITELIST: repo.test.meissa.de` TO `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: repo.test.meissa.de` (Fallback deprecation in 1.21) -1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` -1. Set version to `1.20.1-0` with `k edit deployment forgejo` -1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `1.20.1-0` with `k edit -n forgejo deployment forgejo` +1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Check for errors ## Upgrade to 1.21.1-0 -1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` -1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` -1. Set version to `1.21.1-0` with `k edit deployment forgejo` -1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `1.21.1-0` with `k edit -n forgejo deployment forgejo` +1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Check for errors 1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`. ## Upgrade to 7.0.0 -1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` -1. Adjust configmap: `k edit cm forgejo-env` +1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Adjust configmap: `k edit -n forgejo cm forgejo-env` 1. Change `FORGEJO__oauth2__ENABLE: "true"` TO `FORGEJO__oauth2__ENABLED: "true"` -1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` -1. Set version to `7.0.0` with `k edit deployment forgejo` -1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `7.0.0` with `k edit -n forgejo deployment forgejo` +1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Check for errors ## Upgrade to 8.0.0 (no relevant breaking changes) -1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0` -1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` -1. Set version to `7.0.5` with `k edit deployment forgejo` -1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1` +1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Set version to `7.0.5` with `k edit -n forgejo deployment forgejo` +1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Check for errors ## Post Work 1. Switch DNS to new server -1. Reenable Backup Cron on new server: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : false }}'` -1. Execute manual Backup on new server: `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh` -1. Scale down Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1` +1. Reenable Backup Cron on new server: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : false }}'` +1. Execute manual Backup on new server: `kubectl exec -n forgejo -it backup-restore-... -- /usr/local/bin/backup.sh` +1. Scale down Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1` 1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens! 1. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work. From e1e032697dfae398dbb2540a5dfee05339922aa8 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 31 Jul 2024 10:16:47 +0200 Subject: [PATCH 52/58] Added cmd for pod logs to Runbook --- doc/Runbook_UpgradeFrom1.19To7.0.5.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/Runbook_UpgradeFrom1.19To7.0.5.md b/doc/Runbook_UpgradeFrom1.19To7.0.5.md index 85d4405..89fcc28 100644 --- a/doc/Runbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Runbook_UpgradeFrom1.19To7.0.5.md @@ -37,7 +37,7 @@ 1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` 1. Set version to `1.20.1-0` with `k edit -n forgejo deployment forgejo` 1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` -1. Check for errors +1. Check for errors: `k logs -n forgejo forgejo-...` ## Upgrade to 1.21.1-0 @@ -45,7 +45,7 @@ 1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` 1. Set version to `1.21.1-0` with `k edit -n forgejo deployment forgejo` 1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` -1. Check for errors +1. Check for errors: `k logs -n forgejo forgejo-...` 1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`. ## Upgrade to 7.0.0 @@ -56,15 +56,15 @@ 1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` 1. Set version to `7.0.0` with `k edit -n forgejo deployment forgejo` 1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` -1. Check for errors +1. Check for errors: `k logs -n forgejo forgejo-...` ## Upgrade to 8.0.0 (no relevant breaking changes) 1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` 1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` -1. Set version to `7.0.5` with `k edit -n forgejo deployment forgejo` +1. Set version to `8.0.0` with `k edit -n forgejo deployment forgejo` 1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` -1. Check for errors +1. Check for errors: `k logs -n forgejo forgejo-...` ## Post Work From 6bab8fcc3919f97909947202d0314f82c1e5bce7 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 31 Jul 2024 11:22:13 +0200 Subject: [PATCH 53/58] release: 3.5.0 --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 2a43c08..3a46ff8 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.4.5-dev" +version = "3.5.0" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index 30cefe9..b9fbce4 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.4.5-dev" +version = "3.5.0" @init def initialize(project): diff --git a/package.json b/package.json index e4039aa..f028f3f 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.4.5-SNAPSHOT", + "version": "3.5.0", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index 4e65cf5..836a89a 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.5-SNAPSHOT" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.5.0" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From b21317268c0da0f9fdc5cc6574b4a2f0ddf16a29 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 31 Jul 2024 11:22:13 +0200 Subject: [PATCH 54/58] bump version to: 3.5.1-SNAPSHOT --- infrastructure/backup/build.py | 2 +- infrastructure/federated/build.py | 2 +- package.json | 2 +- project.clj | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infrastructure/backup/build.py b/infrastructure/backup/build.py index 3a46ff8..5a13e15 100644 --- a/infrastructure/backup/build.py +++ b/infrastructure/backup/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = "c4k-forgejo" MODULE = "backup" PROJECT_ROOT_PATH = "../.." -version = "3.5.0" +version = "3.5.1-dev" @init diff --git a/infrastructure/federated/build.py b/infrastructure/federated/build.py index b9fbce4..81a9c55 100644 --- a/infrastructure/federated/build.py +++ b/infrastructure/federated/build.py @@ -6,7 +6,7 @@ from ddadevops import * name = 'c4k-forgejo' MODULE = 'federated' PROJECT_ROOT_PATH = '../..' -version = "3.5.0" +version = "3.5.1-dev" @init def initialize(project): diff --git a/package.json b/package.json index f028f3f..0f1eb50 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "c4k-forgejo", "description": "Generate c4k yaml for a forgejo deployment.", "author": "meissa GmbH", - "version": "3.5.0", + "version": "3.5.1-SNAPSHOT", "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme", "repository": "https://www.npmjs.com/package/c4k-forgejo", "license": "APACHE2", diff --git a/project.clj b/project.clj index 836a89a..c977c83 100644 --- a/project.clj +++ b/project.clj @@ -1,4 +1,4 @@ -(defproject org.domaindrivenarchitecture/c4k-forgejo "3.5.0" +(defproject org.domaindrivenarchitecture/c4k-forgejo "3.5.1-SNAPSHOT" :description "forgejo c4k-installation package" :url "https://domaindrivenarchitecture.org" :license {:name "Apache License, Version 2.0" From ba2b5157d447e4a43f4d515981fb04829297b9d0 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 31 Jul 2024 11:29:00 +0200 Subject: [PATCH 55/58] [Skip-CI] added c4k-forgejo base version for upgrade to runbook --- doc/Runbook_UpgradeFrom1.19To7.0.5.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/Runbook_UpgradeFrom1.19To7.0.5.md b/doc/Runbook_UpgradeFrom1.19To7.0.5.md index 89fcc28..f6c0073 100644 --- a/doc/Runbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Runbook_UpgradeFrom1.19To7.0.5.md @@ -18,8 +18,8 @@ ### Create 2nd Repo Prod Server 1. Terraform Preparations for 2nd Server: TODO -1. Install c4k-forgejo Version TODO - with config `"forgejo-image-version-overwrite": "1.19.3-0"` +1. Install c4k-forgejo Version `3.5.0`! + with config `"forgejo-image-version-overwrite": "1.19.3-0"` (in server-setup) 1. Stop Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` 1. Disable Backup Cron: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'` 1. Scale up Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1` From c9f6d54ce1ca107d3d74804126a4e73c76dbdf07 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 31 Jul 2024 11:30:57 +0200 Subject: [PATCH 56/58] update forgejo image version to 8.0 --- src/main/cljc/dda/c4k_forgejo/forgejo.cljc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc index abda99c..0766a02 100644 --- a/src/main/cljc/dda/c4k_forgejo/forgejo.cljc +++ b/src/main/cljc/dda/c4k_forgejo/forgejo.cljc @@ -73,7 +73,7 @@ (def federated-image-name "domaindrivenarchitecture/c4k-forgejo-federated") (def federated-image-version "latest") (def non-federated-image-name "codeberg.org/forgejo/forgejo") -(def non-federated-image-version "7.0") +(def non-federated-image-version "8.0") (defn-spec generate-image-str string? [config config?] From b133f89ea4ca65b650b63e97d667d0fd086ec8e5 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 31 Jul 2024 11:37:23 +0200 Subject: [PATCH 57/58] fix tests --- src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc b/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc index d7b801b..25d64cd 100644 --- a/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc +++ b/src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc @@ -14,7 +14,7 @@ (deftest should-generate-image-str (testing "non-federated-image" - (is (= "codeberg.org/forgejo/forgejo:7.0" + (is (= "codeberg.org/forgejo/forgejo:8.0" (cut/generate-image-str {:fqdn "test.de" :mailer-from "" :mailer-host "m.t.de" @@ -97,7 +97,7 @@ :spec {:containers [{:name "forgejo", - :image "codeberg.org/forgejo/forgejo:7.0", + :image "codeberg.org/forgejo/forgejo:8.0", :imagePullPolicy "IfNotPresent", :envFrom [{:configMapRef {:name "forgejo-env"}} {:secretRef {:name "forgejo-secrets"}}], :volumeMounts [{:name "forgejo-data-volume", :mountPath "/data"}], From be80628785138206b9aec832514981c9ed668313 Mon Sep 17 00:00:00 2001 From: Clemens Date: Wed, 31 Jul 2024 11:59:37 +0200 Subject: [PATCH 58/58] [Skip-CI] Added "enable federation" to runbook --- doc/Runbook_UpgradeFrom1.19To7.0.5.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/doc/Runbook_UpgradeFrom1.19To7.0.5.md b/doc/Runbook_UpgradeFrom1.19To7.0.5.md index f6c0073..4401d7b 100644 --- a/doc/Runbook_UpgradeFrom1.19To7.0.5.md +++ b/doc/Runbook_UpgradeFrom1.19To7.0.5.md @@ -66,6 +66,15 @@ 1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` 1. Check for errors: `k logs -n forgejo forgejo-...` +## Enable Federation + +1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0` +1. Adjust configmap: `k edit -n forgejo cm forgejo-env` + 1. Change `FORGEJO__federation__ENABLED: "false"` TO `FORGEJO__federation__ENABLED: "true"` +1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini` +1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1` +1. Check for errors: `k logs -n forgejo forgejo-...` + ## Post Work 1. Switch DNS to new server