README.md

@@ -1,7 +1,7 @@
 # convention 4 kubernetes: c4k-forgejo
 [![Clojars Project](https://img.shields.io/clojars/v/org.domaindrivenarchitecture/c4k-forgejo.svg)](https://clojars.org/org.domaindrivenarchitecture/c4k-forgejo) [![pipeline status](https://gitlab.com/domaindrivenarchitecture/c4k-forgejo/badges/master/pipeline.svg)](https://gitlab.com/domaindrivenarchitecture/c4k-forgejo/-/commits/main) 
 
-[<img src="https://domaindrivenarchitecture.org/img/delta-chat.svg" width=20 alt="DeltaChat"> chat over e-mail](mailto:buero@meissa-gmbh.de?subject=community-chat) | [<img src="https://meissa.de/images/parts/contact/mastodon36_hue9b2464f10b18e134322af482b9c915e_5501_filter_14705073121015236177.png" width=20 alt="M"> meissa@social.meissa-gmbh.de](https://social.meissa-gmbh.de/@meissa) | [Blog](https://domaindrivenarchitecture.org) | [Website](https://meissa.de)
+[<img src="https://domaindrivenarchitecture.org/img/delta-chat.svg" width=20 alt="DeltaChat"> chat over e-mail](mailto:buero@meissa-gmbh.de?subject=community-chat) | [<img src="https://meissa-gmbh.de/img/community/Mastodon_Logotype.svg" width=20 alt="team@social.meissa-gmbh.de"> team@social.meissa-gmbh.de](https://social.meissa-gmbh.de/@team) | [Website & Blog](https://domaindrivenarchitecture.org)
 
 ## Purpose
 
@@ -46,8 +46,8 @@ Development happens at: https://repo.prod.meissa.de/meissa/c4k-forgejo
 
 Mirrors are:
 
-* https://codeberg.org/meissa/c4k-forgejo (Issues and PR)
 * https://gitlab.com/domaindrivenarchitecture/c4k-forgejo (CI)
+* https://codeberg.org/meissa/c4k-forgejo (issues and PR)
 * https://github.com/DomainDrivenArchitecture/c4k-forgejo
 
 For more details about our repository model see: https://repo.prod.meissa.de/meissa/federate-your-repos
@@ -55,6 +55,6 @@ For more details about our repository model see: https://repo.prod.meissa.de/mei
 
 ## License
 
-Copyright © 2023, 2024 meissa GmbH
+Copyright © 2023 meissa GmbH
 Licensed under the [Apache License, Version 2.0](LICENSE) (the "License")
 Pls. find licenses of our subcomponents [here](doc/SUBCOMPONENT_LICENSE)

doc/BackupAndRestore.md

@@ -10,32 +10,32 @@
 ## Manual init the restic repository for the first time
 
 1. apply backup-and-restore pod:   
-   `kubectl -n forgejo scale deployment backup-restore --replicas=1`
+   `kubectl scale deployment backup-restore --replicas=1`
 2. exec into pod and execute restore pod (press tab to get your exact pod name)   
-   `kubectl -n forgejo exec -it backup-restore-... -- /usr/local/bin/init.bb`
+   `kubectl exec -it backup-restore-... -- /usr/local/bin/init.sh`
 3. remove backup-and-restore pod:   
-   `kubectl -n forgejo scale deployment backup-restore --replicas=0`
+   `kubectl scale deployment backup-restore --replicas=0`
 
 
 ## Manual backup the restic repository for the first time
 
 1. apply backup-and-restore pod:   
-  `kubectl -n forgejo scale deployment backup-restore --replicas=1`
+  `kubectl scale deployment backup-restore --replicas=1`
 2. exec into pod and execute backup pod (press tab to get your exact pod name)   
-   `kubectl -n forgejo exec -it backup-restore-... -- /usr/local/bin/backup.bb`
+   `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh`
 3. remove backup-and-restore pod:   
-   `kubectl -n forgejo scale deployment backup-restore --replicas=0`
+   `kubectl scale deployment backup-restore --replicas=0`
 
 
 ## Manual restore
 
 1. apply backup-and-restore pod:   
-  `kubectl -n forgejo scale deployment backup-restore --replicas=1`
+  `kubectl scale deployment backup-restore --replicas=1`
 2. Scale down forgejo deployment:
-   `kubectl -n forgejo scale deployment forgejo --replicas=0`
+   `kubectl scale deployment forgejo --replicas=0`
 3. exec into pod and execute restore pod (press tab to get your exact pod name)   
-   `kubectl -n forgejo exec -it backup-restore-... -- /usr/local/bin/restore.bb`
+   `kubectl exec -it backup-restore-... -- /usr/local/bin/restore.sh`
 4. Start forgejo again:
-   `kubectl -n forgejo scale deployment forgejo --replicas=1`
+   `kubectl scale deployment forgejo --replicas=1`
 5. remove backup-and-restore pod:   
-   `kubectl -n forgejo scale deployment backup-restore --replicas=0`
+   `kubectl scale deployment backup-restore --replicas=0`

doc/Releasing.md

@@ -0,0 +1,41 @@
+# Release process
+
+## ... for testing (snapshots)
+
+Make sure your clojars.org credentials are correctly set in your ~/.lein/profiles.clj file.
+
+``` bash
+git add .
+git commit
+```
+
+``` bash
+lein deploy # or lein deploy clojars
+```
+
+## ... for stable release
+
+Make sure tags are protected in gitlab:
+Repository Settings -> Protected Tags -> set \*.\*.\* as tag and save.
+
+``` bash
+git checkout main # for old projects replace main with master
+git add .
+git commit 
+```
+
+Execute tests
+
+``` bash
+shadow-cljs compile test
+node target/node-tests.js
+lein test
+```
+
+Release with type (NONE, PATCH, MINOR, MAJOR):
+``` bash
+RELEASE_TYPE=[TYPE] pyb prepare_release after_publish
+
+```
+
+Done.

doc/Runbook_UpgradeFrom1.19.md

@@ -1,135 +0,0 @@
-# Playbook Upgrade from 1.19 to 7.0.5
-
-## Info: Relevant Breaking Changes:
-
-* 1.19.3: First version under consideration
-* 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0
-* 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0
-* 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0
-* 8.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#8-0-0
-
-## Preparations
-
-1. Stop Forgejo Prod: `k scale deployment forgejo --replicas=0`
-1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'`
-1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1`
-1. Execute Manual Backup: `kubectl exec -n forgejo -it backup-restore-... -- /usr/local/bin/backup.sh`
-
-### Create 2nd Repo Prod Server
-
-1. Terraform Preparations for 2nd Server: TODO
-1. Install c4k-forgejo Version `3.5.0`!
-   with config `"forgejo-image-version-overwrite": "1.19.3-0"` (in server-setup)
-1. Stop Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
-1. Disable Backup Cron: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'`
-1. Scale up Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1`
-1. Restore Forgejo Backup: See [BackupAndRestore.md](BackupAndRestore.md)
-1. Check for `..._INSTALL_LOCK: true` in ConfigMap `forgejo-env`
-1. Scale up Forgejo Deployment and check for (startup) problems: `k scale -n forgejo deployment forgejo --replicas=1`
-
-## Upgrade to 1.20.1-0
-
-1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
-1. Adjust configmap: `k edit -n forgejo cm forgejo-env`
-    1. Remove `FORGEJO__database__CHARSET: utf8` (This was a misconfiguration, since this option only had effect for mysql dbs)
-    1. Change `FORGEJO__mailer__MAILER_TYPE: smtp+startls` TO `FORGEJO__mailer__PROTOCOL: smtp+starttls` (Missed deprecation from 1.19)
-    1. Change `FORGEJO__service__EMAIL_DOMAIN_WHITELIST: repo.test.meissa.de` TO `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: repo.test.meissa.de` (Fallback deprecation in 1.21)
-1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
-1. Set version to `1.20.1-0` with `k edit -n forgejo deployment forgejo`
-1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
-1. Check for errors: `k logs -n forgejo forgejo-...`
-
-## Upgrade to 1.21.1-0
-
-1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
-1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
-1. Set version to `1.21.1-0` with `k edit -n forgejo deployment forgejo`
-1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
-1. Check for errors: `k logs -n forgejo forgejo-...`
-1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`.
-
-## Upgrade to 7.0.0
-
-1. Check DB Version.
-    1. MariaDB or MySQL needs to be 8.0 or higher.
-    2. Postgres needs to be 12 or higher
-1. API Endpoints
-    1. Check if the [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) API endpoint is used
-        1. as the per_page param is not used for [limit](https://codeberg.org/forgejo/forgejo/commit/0aab2d38a7d91bc8caff332e452364468ce52d9a) anymore
-    2. Check if [/repos/{owner}/{repo}/push_mirrors](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) and [/repos/{owner}/{repo}/push_mirrors](https://code.forgejo.org/api/swagger/#/repository/repoAddPushMirror) API endpoints are used
-        1. The date format of created and last_update fields are now [timestamps](https://codeberg.org/forgejo/forgejo/commit/0ee7cbf725f45650136be45f8e0f74d395f73b5c)
-    3. [pprof](https://forgejo.org/docs/v7.0/admin/config-cheat-sheet/#server-server) endpoint changed labels
-        1. graceful-lifecycle to gracefulLifecycle
-        2. process-type to processType
-        3. process-description to processDescription This allows for those endpoints to be scraped by services requiring prometheus style labels such as grafana-agent.
-1. The Gitea themes were renamed and the \[ui\].THEMES setting must be changed as follows:
-    1. gitea is replaced by gitea-light
-    2. arc-green is replaced by gitea-dark
-    3. auto is replaced by gitea-auto
-1. Migration warning
-    2. If the logs show a line like the following, run `doctor convert` to fix it.
-        3. Current database is using a case-insensitive collation "utf8mb4_general_ci"
-    4. Large instances may experience slow migrations when the database is upgraded to support SHA-256 git repositories.
-1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
-1. Adjust configmap: `k edit -n forgejo cm forgejo-env`
-    1. Change `FORGEJO__oauth2__ENABLE: "true"` TO `FORGEJO__oauth2__ENABLED: "true"`
-1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
-1. Set version to `7.0.0` with `k edit -n forgejo deployment forgejo`
-1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
-1. Check for errors: `k logs -n forgejo forgejo-...`
-
-## Upgrade to 8.0.3 (no relevant breaking changes)
-
-1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
-1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
-1. Set version to `8.0.3` with `k edit -n forgejo deployment forgejo`
-1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
-1. Check for errors: `k logs -n forgejo forgejo-...`
-
-## Enable Federation
-
-1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
-1. Adjust configmap: `k edit -n forgejo cm forgejo-env`
-    1. Change `FORGEJO__federation__ENABLED: "false"` TO `FORGEJO__federation__ENABLED: "true"`
-1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
-1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
-1. Check for errors: `k logs -n forgejo forgejo-...`
-
-## Post Work
-
-1. Switch DNS to new server
-1. Reenable Backup Cron on new server: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : false }}'`
-1. Execute manual Backup on new server: `kubectl exec -n forgejo -it backup-restore-... -- /usr/local/bin/backup.sh`
-1. Scale down Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1`
-1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens!
-1. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work.
-
-## Known Errors
-
-### Error in v1.20.1-0
-
-In the logs the following error can be found. This will be resolved automatically with the next upgrade (v1.21).
-
-```
-2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21
-```
-
-# Add Shynet Analytics
-
-1. Log into shynet & create new Service
-    1. Copy the generated html snippet and save it somewhere you remember
-1. SSH into prod server
-1. Make the necessary folders and files in forgejo data dir:
-    1. `kubectl exec -n forgejo -it forgejo-... -- bash`
-    1. `mkdir -p /data/gitea/templates/custom`
-    1. `touch /data/gitea/templates/custom/footer.tmpl`
-1. Open the `footer.tmpl` and paste the saved snippet
-1. Restart the pod
-    1. `k scale -n forgejo deployment forgejo --replicas=0`
-    1. `k scale -n forgejo deployment forgejo --replicas=1`
-1. Add Information about analytics: Clone Datenschutz Repo
-    1. `git clone ssh://git@repo.prod.meissa.de:2222/meissa/Datenschutz.git`
-1. Merge forgejo-upgrade into main
-    1. `git merge forgejo-upgrade`
-1. Push to origin
-    1. `git push`

doc/Runbook_UpgradeFrom1.19To7.0.5.md

@@ -0,0 +1,87 @@
+# Playbook Upgrade from 1.19 to 7.0.5
+
+## Info: Relevant Breaking Changes:
+
+* 1.19.3:Current version
+* 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0
+* 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0
+* 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0
+
+## Preparations
+
+1. Stop Forgejo Prod: `k scale deployment forgejo --replicas=0`
+1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'`
+1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1`
+1. Execute Manual Backup: `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh`
+
+### Create 2nd Repo Prod Server
+
+1. Terraform Preparations for 2nd Server: TODO
+1. Install c4k-forgejo Version TODO   
+   with config `"forgejo-image-version-overwrite": "1.19.3-0"`
+1. Stop Forgejo Deployment: `k scale deployment forgejo --replicas=0`
+1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'`
+1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1`
+1. Restore Forgejo Backup: See [BackupAndRestore.md](BackupAndRestore.md)
+1. Check for `..._INSTALL_LOCK: true` in ConfigMap `forgejo-env`
+1. Scale up Forgejo Deployment and check for (startup) problems: `k scale deployment forgejo --replicas=1`
+
+## Upgrade to 1.20.1-0
+
+1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0`
+1. Adjust configmap: `k edit cm forgejo-env`
+    1. Remove `FORGEJO__database__CHARSET: utf8` (This was a misconfiguration, since this option only had effect for mysql dbs)
+    1. Change `FORGEJO__mailer__MAILER_TYPE: smtp+startls` TO `FORGEJO__mailer__PROTOCOL: smtp+starttls` (Missed deprecation from 1.19)
+    1. Change `FORGEJO__service__EMAIL_DOMAIN_WHITELIST: repo.test.meissa.de` TO `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: repo.test.meissa.de` (Fallback deprecation in 1.21)
+1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
+1. Set version to `1.20.1-0` with `k edit deployment forgejo`
+1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1`
+1. Check for errors
+
+## Upgrade to 1.21.1-0
+
+1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0`
+1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
+1. Set version to `1.21.1-0` with `k edit deployment forgejo`
+1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1`
+1. Check for errors
+1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`.
+
+## Upgrade to 7.0.0
+
+1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0`
+1. Adjust configmap: `k edit cm forgejo-env`
+    1. Change `FORGEJO__oauth2__ENABLE: "true"` TO `FORGEJO__oauth2__ENABLED: "true"`
+1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
+1. Set version to `7.0.0` with `k edit deployment forgejo`
+1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1`
+1. Check for errors
+
+## Upgrade to 7.0.5 (no breaking changes)
+
+TODO: Upgrade to 8.0.0 instead after Release!
+
+1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0`
+1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
+1. Set version to `7.0.5` with `k edit deployment forgejo`
+1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1`
+1. Check for errors
+
+## Post Work
+
+1. Switch DNS to new server
+1. Reenable Backup Cron on new server: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : false }}'`
+1. Execute manual Backup on new server: `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh`
+1. Scale down Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1`
+1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens!
+1. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work.
+
+## Known Errors
+
+### Error in v1.20.1-0
+
+In the logs the following error can be found. This will be resolved automatically with the next upgrade (v1.21).
+
+```
+2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21
+```

doc/Upgrading.md

@@ -0,0 +1,195 @@
+# Upgrading process
+
+## adhoc (on kubernetes cluster)
+
+Ssh into your kubernetes cluster running the forgejo instance.  
+
+``` bash
+kubectl edit configmap forgejo-env
+# make sure INSTALL_LOCK under security is set to true to disable the installation screen
+# save and exit
+kubectl edit deployments forgejo
+# search for your current forgejo version, e.g. 1.19
+# replace with new version
+# save and exit
+kubectl scale deployment forgejo --replicas=0
+kubectl scale deployment forgejo --replicas=1
+```
+
+Logging into the admin account should now show the new version.
+You may want to update your c4k-forgejo resources to reflect the changes made on the cluster.
+
+## Upgrading from 1.19
+
+### Config related issues with c4k-forgejo v3.2.2
+
+These errors show in the log, when just upgrading to forgejo v7.0.4 from 1.19 without changing the config.
+The related config options are listed below the errors.
+
+- Oauth2: ENABLED instead of ENABLE
+	- `FORGEJO__oauth2__ENABLED: "true"`  
+- [E] Deprecated config option `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead.
+	- `FORGEJO__log_0x2E_logger_0x2E_router__MODE: console, file`  
+- [E] Deprecated config option `[service]` `EMAIL_DOMAIN_WHITELIST` present. Use `[service]
+  ` `EMAIL_DOMAIN_ALLOWLIST` instead.
+	- `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: YOUR_ALLOW_LIST`  
+- [E] Deprecated config option `[mailer]` `MAILER_TYPE` present. Use `[mailer]` `PROTOCOL` 
+  instead. 
+- [E] Deprecated fallback `[mailer]` `PROTOCOL = smtp+startls` present. Use `[mailer]` `PROTOCOL = smtp+starttls`` instead.
+	- `FORGEJO__mailer__PROTOCOL: smtp+starttls`  
+
+### Breaking Changes since 1.19
+
+#### 1.19.3 & 1.19.4: Version installed by c4k-forgejo v3.2.2
+
+#### 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0
+
+##### app.ini
+
+- Check [queue] section - n/e
+- Check [repository.editor] - n/e
+- Check [storage] - n/e
+- Check ssh_keygen_path in app.ini - n/e
+- Is WORK_PATH set? Or app.ini writeable by forgejo server user?
+    - 1. No
+    - 2. Yes
+		- If not, it shows in the logs starting with: `Unable to update WORK_PATH`
+		- Also ssh pushing will likely fail 
+			- *test ssh*
+- Set logger.router.mode as described in environment-to-ini
+	- See: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/contrib/environment-to-ini
+- Check [git.reflog] and maybe move to [git.config] - n/e
+- Check [indexer], [mailer], [repository] - n/e
+
+##### tokens
+
+- Scoped and personal access tokens were refactored
+	- Scope may change, if we have tokens they should be rotated
+
+#### 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0
+
+##### custom themes
+
+- Move to `custom/public/assets/`
+
+##### git branches
+
+- `/admin` page and click run Sync missed branches from git data to databases.
+
+##### db - mysql
+
+- c4k uses postgres
+
+##### ssh server
+
+- We don't use host cert used for auth
+
+##### ssh keys
+
+- All users need to check their key length, now 3072
+
+##### tokens
+
+- Finer restrictions might now return 404 errors on users tokens in certain teams with certain restrictions
+
+#### 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0
+
+##### webhooks
+
+- Do we use webhooks?
+
+##### db
+
+- Psql min ver is 12, c4k-common uses 14+
+
+##### api
+
+- [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases)
+- [/repos/{owner}/{repo}/push_mirrors](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors)
+- Application profiling
+
+##### repos
+
+- Do we have repo descriptions?
+	- https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d
+
+##### app.ini
+
+- Check [ui] - n/e
+
+### Vor dem Upgrade
+
+- Host cert used for auth? - no
+- Do we use webhooks? - no
+- Do we use:
+	- [/repos/{owner}/{repo}/releases - repoListReleases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) - no
+		- In the ListReleases, the `per_page` parameter has been decoupled from the `limit` parameter, we do not use the repoListReleases endpoint
+		- In the `ArtifactDeploymentApi` in dda-devops-build we only use the `POST` method
+			- The respective endpoint is [repoCreateRelease](https://code.forgejo.org/api/swagger/#/repository/repoCreateRelease)
+	- [`/repos/{owner}/{repo}/push_mirrors`](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) - no
+	- Application profiling - no
+- Do we have repo descriptions? - yes
+	- There is now a sanitizer that only allows links, emphasis, code and emojis
+		- See: https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d
+	- Our repository descriptions are mostly plaintext and links
+
+### Upgrade plan
+
+TEST indicates actions that only apply to the test server and are ignored in PROD.
+PROD indicates actions that only apply to the prod server and are ignored in TEST.
+See also the overview for upgrading: https://forgejo.org/docs/latest/admin/upgrade/
+
+- Set up Forgejo server with c4k-forgejo v3.2.2
+	- Has Forgejo v1.19
+- TEST
+	- Delete old remote ids
+		- `ssh-keygen -f "/home/${USER}/.ssh/known_hosts" -R "repo.test.meissa.de"`
+- Ssh to server
+- Forgejo pod downscale
+	- `k scale deployment forgejo --replicas=0`
+- Install lock off
+	- `k edit cm forgejo-env`
+	- Set to `FORGEJO__security__INSTALL_LOCK: "false"`
+- Forgejo pod upscale
+	- `k scale deployment forgejo --replicas=1`
+- Create admin test or prod admin and install forgejo
+	- `gopass show server/meissa/forgejo-test` bzw `-prod`
+- Forgejo pod downscale
+- Install lock on
+	- Set to `FORGEJO__security__INSTALL_LOCK: "true"`
+- TEST
+	- Forgejo pod upscale
+	- Log in
+	- Make Ssh keys
+		- ed_xyz
+		- rsa mit 2048
+		- rsa mit 4096
+	- Create repos
+	- Forgejo pod downscale
+- PROD
+	- Backup pod upscale
+		- `k scale deployment backup-restore --replicas=1`
+	- Restore backups
+	- Delete or rename app.ini's in the pod
+	- Backup pod downscale
+		- `k scale deployment backup-restore --replicas=0`
+- Set image version to 7.0.4 in forgejo deployment
+	- `k edit deployment.apps forgejo`
+- Update configmap:
+	- Double check install lock enabled
+	- `FORGEJO__oauth2__ENABLED: "true"`  
+	- `FORGEJO__log_0x2E_logger_0x2E_router__MODE: console, file`  
+	- `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST:`  
+	- `FORGEJO__mailer__PROTOCOL: smtp+starttls`  
+	- `FORGEJO__federation__ENABLED: true`  
+- TEST
+	- Backup pod upscale
+	- Delete or rename app.ini's in the pod
+	- Backup pod downscale
+- Forgejo pod upscale
+- Migrations happen automatically
+- `/admin` page and click run Sync missed branches from git data to databases
+	- and **Sync missed tags ...*
+- Rsa keys with size 2048 can not be added anymore. However, it seems they still can be used if they are on the server
+- Team members having app tokens need to recreate them with proper scopes
+- Add analytics: https://forgejo.org/docs/latest/admin/customization/

infrastructure/backup/build.py

@@ -6,7 +6,7 @@ from ddadevops import *
 name = "c4k-forgejo"
 MODULE = "backup"
 PROJECT_ROOT_PATH = "../.."
-version = "4.0.1-dev"
+version = "3.4.3"
 
 
 @init

infrastructure/backup/image/Dockerfile

@@ -1,4 +1,5 @@
 FROM domaindrivenarchitecture/dda-backup:latest
 
+# Prepare Entrypoint Script
 ADD resources /tmp
-RUN /tmp/install.bb
+RUN /tmp/install.sh

infrastructure/backup/image/resources/backup.bb

@@ -1,46 +0,0 @@
-#!/usr/bin/env bb
-
-(require
- '[dda.backup.core :as bc]
- '[dda.backup.restic :as rc]
- '[dda.backup.postgresql :as pg]
- '[dda.backup.backup :as bak])
-
-(def restic-repo {:password-file (bc/env-or-file "RESTIC_PASSWORD_FILE")
-                  :restic-repository (bc/env-or-file "RESTIC_REPOSITORY")})
-
-(def file-config (merge restic-repo {:backup-path "files"
-                                     :execution-directory "/var/backups/"
-                                     :files ["gitea/" "git/repositories/"]}))
-
-
-(def db-config (merge restic-repo {:backup-path "pg-database"
-                                   :pg-host (bc/env-or-file "POSTGRES_SERVICE")
-                                   :pg-port (bc/env-or-file "POSTGRES_PORT")
-                                   :pg-db (bc/env-or-file "POSTGRES_DB")
-                                   :pg-user (bc/env-or-file "POSTGRES_USER")
-                                   :pg-password (bc/env-or-file "POSTGRES_PASSWORD")}))
-
-(def aws-config {:aws-access-key-id (bc/env-or-file "AWS_ACCESS_KEY_ID")
-                 :aws-secret-access-key (bc/env-or-file "AWS_SECRET_ACCESS_KEY")})
-
-(def dry-run {:dry-run true :debug true})
-
-(defn prepare!
-  []
-  (bc/create-aws-credentials! aws-config)
-  (pg/create-pg-pass! db-config))
-
-(defn restic-repo-init!
-  []
-  (rc/init! file-config)
-  (rc/init! db-config))
-
-(defn restic-backup!
-  []
-  (bak/backup-file! file-config)
-  (bak/backup-db! db-config))
-
-(prepare!)
-(restic-repo-init!)
-(restic-backup!)

infrastructure/backup/image/resources/backup.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+
+set -Eexo pipefail
+
+function main() {
+    file_env AWS_ACCESS_KEY_ID
+    file_env AWS_SECRET_ACCESS_KEY
+    file_env RESTIC_DAYS_TO_KEEP 30
+    file_env RESTIC_MONTHS_TO_KEEP 12
+
+    backup-db-dump
+    backup-fs-from-directory '/var/backups/' 'gitea/' 'git/repositories/'
+}
+
+source /usr/local/lib/functions.sh
+source /usr/local/lib/pg-functions.sh
+source /usr/local/lib/file-functions.sh
+
+main

infrastructure/backup/image/resources/bb-backup.edn

@@ -1,3 +0,0 @@
-{:deps {org.clojure/spec.alpha {:mvn/version "0.4.233"}
-        orchestra/orchestra {:mvn/version "2021.01.01-1"}
-        org.domaindrivenarchitecture/dda-backup {:local/root "/usr/local/lib/dda-backup"}}}

infrastructure/backup/image/resources/bb.edn

@@ -1,3 +0,0 @@
-{:deps {org.clojure/spec.alpha {:mvn/version "0.4.233"}
-        orchestra/orchestra {:mvn/version "2021.01.01-1"}
-        org.domaindrivenarchitecture/dda-build {:mvn/version "0.1.1-SNAPSHOT"}}}

infrastructure/backup/image/resources/entrypoint-start-and-wait.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+
+set -Eexo pipefail
+
+function main() {
+    create-pg-pass
+
+    while true; do
+        sleep 1m
+    done
+}
+
+source /usr/local/lib/functions.sh
+source /usr/local/lib/pg-functions.sh
+main

infrastructure/backup/image/resources/entrypoint.sh

@@ -0,0 +1,13 @@
+#!/bin/bash
+
+set -Eexo pipefail
+
+function main() {
+    create-pg-pass
+
+	/usr/local/bin/backup.sh
+}
+
+source /usr/local/lib/functions.sh
+source /usr/local/lib/pg-functions.sh
+main

infrastructure/backup/image/resources/init.bb

@@ -1,3 +0,0 @@
-#!/usr/bin/env bb
-
-(println "initialized")

infrastructure/backup/image/resources/init.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+
+set -Eexo pipefail
+
+function main() {
+    file_env AWS_ACCESS_KEY_ID
+    file_env AWS_SECRET_ACCESS_KEY
+
+    init-database-repo
+    init-file-repo
+}
+
+source /usr/local/lib/functions.sh
+source /usr/local/lib/pg-functions.sh
+source /usr/local/lib/file-functions.sh
+main

infrastructure/backup/image/resources/install.bb

@@ -1,14 +0,0 @@
-#!/usr/bin/env bb
-
-(require 
- '[dda.image.ubuntu :as ub]
- '[dda.image.install :as in])
-
-
-(ub/upgrade-system!)
-(in/install! "bb-backup.edn" :target-name "bb.edn" :mod "0400")
-(in/install! "backup.bb")
-(in/install! "restore.bb")
-(in/install! "list-snapshots.bb")
-(in/install! "wait.bb")
-(ub/cleanup-container!)

infrastructure/backup/image/resources/install.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+
+set -exo pipefail
+
+function main()
+{
+    upgradeSystem
+    
+    install -m 0700 /tmp/entrypoint.sh /
+    install -m 0700 /tmp/entrypoint-start-and-wait.sh /
+
+    install -m 0700 /tmp/init.sh /usr/local/bin/
+    install -m 0700 /tmp/backup.sh /usr/local/bin/
+    install -m 0700 /tmp/restore.sh /usr/local/bin/
+    install -m 0700 /tmp/restic-snapshots.sh /usr/local/bin/
+
+    cleanupDocker
+} > /dev/null
+
+source /tmp/install_functions_debian.sh
+DEBIAN_FRONTEND=noninteractive DEBCONF_NOWARNINGS=yes main

infrastructure/backup/image/resources/list-snapshots.bb

@@ -1,28 +0,0 @@
-#!/usr/bin/env bb
-
-(require
- '[dda.backup.core :as bc]
- '[dda.backup.restic :as rc])
-
-(def restic-repo {:password-file (bc/env-or-file "RESTIC_PASSWORD_FILE")
-                  :restic-repository (bc/env-or-file "RESTIC_REPOSITORY")})
-
-(def file-config (merge restic-repo {:backup-path "files"}))
-
-
-(def db-config (merge restic-repo {:backup-path "pg-database"}))
-
-(def aws-config {:aws-access-key-id (bc/env-or-file "AWS_ACCESS_KEY_ID")
-                 :aws-secret-access-key (bc/env-or-file "AWS_SECRET_ACCESS_KEY")})
-
-(defn prepare!
-  []
-  (bc/create-aws-credentials! aws-config))
-
-(defn list-snapshots!
-  []
-  (rc/list-snapshots! file-config)
-  (rc/list-snapshots! db-config))
-
-(prepare!)
-(list-snapshots!)

infrastructure/backup/image/resources/restic-snapshots.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+
+set -exo pipefail
+
+function main() {
+    file_env AWS_ACCESS_KEY_ID
+    file_env AWS_SECRET_ACCESS_KEY
+
+    restic -r ${RESTIC_REPOSITORY}/files snapshots
+    restic -r ${RESTIC_REPOSITORY}/pg-database snapshots
+}
+
+source /usr/local/lib/functions.sh
+source /usr/local/lib/file-functions.sh
+
+main

infrastructure/backup/image/resources/restore.bb

@@ -1,46 +0,0 @@
-#!/usr/bin/env bb
-
-(require '[babashka.tasks :as tasks]
-         '[dda.backup.core :as bc]
-         '[dda.backup.postgresql :as pg]
-         '[dda.backup.restore :as rs])
-
-(def restic-repo {:password-file (bc/env-or-file "RESTIC_PASSWORD_FILE")
-                  :restic-repository (bc/env-or-file "RESTIC_REPOSITORY")})
-
-(def file-config (merge restic-repo {:backup-path "files"
-                                     :restore-target-directory "/var/backups/restore"
-                                     :snapshot-id "latest"}))
-
-
-(def db-config (merge restic-repo {:backup-path "pg-database"
-                                   :pg-host (bc/env-or-file "POSTGRES_SERVICE")
-                                   :pg-port (bc/env-or-file "POSTGRES_PORT")
-                                   :pg-db (bc/env-or-file "POSTGRES_DB")
-                                   :pg-user (bc/env-or-file "POSTGRES_USER")
-                                   :pg-password (bc/env-or-file "POSTGRES_PASSWORD")
-                                   :snapshot-id "latest"}))
-
-(def aws-config {:aws-access-key-id (bc/env-or-file "AWS_ACCESS_KEY_ID")
-                 :aws-secret-access-key (bc/env-or-file "AWS_SECRET_ACCESS_KEY")})
-
-(def dry-run {:dry-run true :debug true})
-
-(defn prepare!
-  []
-  (pg/create-pg-pass! db-config)
-  (bc/create-aws-credentials! aws-config))
-
-(defn restic-restore!
-  []
-  (rs/restore-file! file-config)
-  (tasks/shell ["bash" "-c" "rm -rf /var/backups/gitea/*"])
-  (tasks/shell ["bash" "-c" "rm -rf /var/backups/git/repositories/*"])
-  (tasks/shell ["mv" "/var/backups/restore/gitea" "/var/backups/"])
-  (tasks/shell ["mv" "/var/backups/restore/git/repositories" "/var/backups/git/"])
-  (tasks/shell ["chown" "-R" "1000:1000" "/var/backups"])
-  (pg/drop-create-db! (merge db-config {:debug true}))
-  (rs/restore-db! (merge db-config {:debug true})))
-
-(prepare!)
-(restic-restore!)

infrastructure/backup/image/resources/restore.sh

@@ -0,0 +1,37 @@
+#!/bin/bash
+
+set -Eexo pipefail
+
+function main() {
+
+    file_env AWS_ACCESS_KEY_ID
+    file_env AWS_SECRET_ACCESS_KEY
+
+    file_env POSTGRES_DB
+    file_env POSTGRES_PASSWORD
+    file_env POSTGRES_USER
+
+    # Restore latest snapshot into /var/backups/restore
+    restore-directory '/var/backups/restore'
+
+    rm -rf /var/backups/gitea/*
+    rm -rf /var/backups/git/repositories/*
+    cp -r /var/backups/restore/gitea /var/backups/ #ToDo: mv instead of cp or rm -rf after
+    cp -r /var/backups/restore/git/repositories /var/backups/git/ #ToDo: mv instead of cp or rm -rf after
+    
+    # adjust file permissions for the git user
+    chown -R 1000:1000 /var/backups
+
+    # TODO: Regenerate Git Hooks? Do we need this?
+    #/usr/local/bin/gitea -c '/data/gitea/conf/app.ini' admin regenerate hooks
+
+    # Restore db
+    drop-create-db
+    restore-db
+}
+
+source /usr/local/lib/functions.sh
+source /usr/local/lib/pg-functions.sh
+source /usr/local/lib/file-functions.sh
+
+main

infrastructure/backup/image/resources/wait.bb

@@ -1,27 +0,0 @@
-#!/usr/bin/env bb
-
-(require
- '[dda.backup.core :as bc]
- '[dda.backup.postgresql :as pg])
-
-
-(def restic-repo {:password-file (bc/env-or-file "RESTIC_PASSWORD_FILE")
-                  :restic-repository (bc/env-or-file "RESTIC_REPOSITORY")})
-
-(def db-config (merge restic-repo {:backup-path "pg-database"
-                                   :pg-host (bc/env-or-file "POSTGRES_SERVICE")
-                                   :pg-port (bc/env-or-file "POSTGRES_PORT")
-                                   :pg-db (bc/env-or-file "POSTGRES_DB")
-                                   :pg-user (bc/env-or-file "POSTGRES_USER")
-                                   :pg-password (bc/env-or-file "POSTGRES_PASSWORD")}))
-
-(defn prepare!
-  []
-  (pg/create-pg-pass! db-config))
-
-(defn wait! []
-  (while true
-    (Thread/sleep 1000)))
-
-(prepare!)
-(wait!)

infrastructure/backup/test/Dockerfile

@@ -1,4 +0,0 @@
-FROM c4k-forgejo-backup:latest
-
-ADD resources /tmp/
-RUN ENV_PASSWORD=env-password FILE_PASSWORD_FILE=/tmp/file_password /tmp/test.bb

infrastructure/backup/test/resources/bb.edn

@@ -1,3 +0,0 @@
-{:deps {org.clojure/spec.alpha {:mvn/version "0.4.233"}
-        orchestra/orchestra {:mvn/version "2021.01.01-1"}
-        org.domaindrivenarchitecture/dda-backup {:local/root "/usr/local/lib/dda-backup"}}}

infrastructure/backup/test/resources/test.bb

@@ -1,62 +0,0 @@
-#!/usr/bin/env bb
-
-(require '[babashka.tasks :as tasks]
-         '[dda.backup.core :as bc]
-         '[dda.backup.restic :as rc]
-         '[dda.backup.postgresql :as pg]
-         '[dda.backup.backup :as bak]
-         '[dda.backup.restore :as rs])
-
-(def restic-repo {:password-file "restic-pwd"
-                  :restic-repository "restic-repo"})
-
-(def file-config (merge restic-repo {:backup-path "files"
-                                     :files ["test-backup"]
-                                     :restore-target-directory "test-restore"}))
-
-
-(def db-config (merge restic-repo {:backup-path "db"
-                                   :pg-db "mydb"
-                                   :pg-user "user"
-                                   :pg-password "password"}))
-
-(def dry-run {:dry-run true :debug true})
-
-(defn prepare!
-  []
-  (spit "/tmp/file_password" "file-password")
-  (println (bc/env-or-file "FILE_PASSWORD"))
-  (println (bc/env-or-file "ENV_PASSWORD"))
-  (spit "restic-pwd" "ThePassword")
-  (tasks/shell "mkdir" "-p" "test-backup")
-  (spit "test-backup/file" "I was here")
-  (tasks/shell "mkdir" "-p" "test-restore")
-  (pg/create-pg-pass! db-config))
-
-(defn restic-repo-init!
-  []
-  (rc/init! file-config)
-  (rc/init! (merge db-config dry-run)))
-
-(defn restic-backup!
-  []
-  (bak/backup-file! file-config)
-  (bak/backup-db! (merge db-config dry-run)))
-
-(defn list-snapshots!
-  []
-  (rc/list-snapshots! file-config)
-  (rc/list-snapshots! (merge db-config dry-run)))
-
-
-(defn restic-restore!
-  []
-  (rs/restore-file! file-config)
-  (pg/drop-create-db! (merge db-config dry-run))
-  (rs/restore-db! (merge db-config dry-run)))
-
-(prepare!)
-(restic-repo-init!)
-(restic-backup!)
-(list-snapshots!)
-(restic-restore!)

infrastructure/federated/build.py

@@ -6,7 +6,7 @@ from ddadevops import *
 name = 'c4k-forgejo'
 MODULE = 'federated'
 PROJECT_ROOT_PATH = '../..'
-version = "4.0.1-dev"
+version = "3.4.3"
 
 @init
 def initialize(project):

package.json

@@ -2,7 +2,7 @@
     "name": "c4k-forgejo",
     "description": "Generate c4k yaml for a forgejo deployment.",
     "author": "meissa GmbH",
-    "version": "4.0.1-SNAPSHOT",
+    "version": "3.4.3",
     "homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme",
     "repository": "https://www.npmjs.com/package/c4k-forgejo",
     "license": "APACHE2",

project.clj

@@ -1,17 +1,16 @@
-(defproject org.domaindrivenarchitecture/c4k-forgejo "4.0.1-SNAPSHOT"
+(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.3"
   :description "forgejo c4k-installation package"
   :url "https://domaindrivenarchitecture.org"
   :license {:name "Apache License, Version 2.0"
             :url "https://www.apache.org/licenses/LICENSE-2.0.html"}
-  :dependencies [[org.clojure/clojure "1.11.4" :scope "provided"]
+  :dependencies [[org.clojure/clojure "1.11.3" :scope "provided"]
-                 [org.clojure/tools.reader "1.5.0"]
+                 [org.clojure/tools.reader "1.4.2"]
-                 [org.domaindrivenarchitecture/c4k-common-clj "8.0.0"]
+                 [org.domaindrivenarchitecture/c4k-common-clj "6.4.1"]
                  [hickory "0.7.1" :exclusions [viebel/codox-klipse-theme]]]
   :target-path "target/%s/"
   :source-paths ["src/main/cljc"
                  "src/main/clj"]
-  :resource-paths ["src/main/resources"
+  :resource-paths ["src/main/resources"]
-                   "project.clj"]
   :repositories [["snapshots" :clojars]
                  ["releases" :clojars]]
   :deploy-repositories [["snapshots" {:sign-releases false :url "https://clojars.org/repo"}]
@@ -24,9 +23,9 @@
                        :main dda.c4k-forgejo.uberjar
                        :uberjar-name "c4k-forgejo-standalone.jar"
                        :dependencies [[org.clojure/tools.cli "1.1.230"]
-                                      [ch.qos.logback/logback-classic "1.5.7"
+                                      [ch.qos.logback/logback-classic "1.5.6"
                                        :exclusions [com.sun.mail/javax.mail]]
-                                      [org.slf4j/jcl-over-slf4j "2.0.16"]
+                                      [org.slf4j/jcl-over-slf4j "2.0.13"]
                                       [com.github.clj-easy/graal-build-time "1.0.5"]]}}
   :release-tasks [["test"]
                   ["vcs" "assert-committed"]

shadow-cljs.edn

@@ -4,7 +4,7 @@
                 "src/test/cljc"
                 "src/test/cljs"
                 "src/test/resources"]
- :dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "8.0.0"]
+ :dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "6.4.1"]
                 [hickory "0.7.1"]]
  :builds {:frontend {:target :browser
                      :modules {:main {:init-fn dda.c4k-forgejo.browser/init}}

src/main/clj/dda/c4k_forgejo/uberjar.clj

@@ -7,11 +7,10 @@
 (set! *warn-on-reflection* true)
 
 (defn -main [& cmd-args]
-  (uberjar/main-cm
+  (uberjar/main-common 
    "c4k-forgejo"
    core/config?
    core/auth?
    core/config-defaults
-   core/config-objects
+   core/k8s-objects
-   core/auth-objects
    cmd-args))

src/main/cljc/dda/c4k_forgejo/core.cljc

@@ -1,33 +1,24 @@
 (ns dda.c4k-forgejo.core
-  (:require
+ (:require
-   [clojure.spec.alpha :as s]
+  [clojure.spec.alpha :as s]
-   [dda.c4k-common.yaml :as yaml]
+  [dda.c4k-common.yaml :as yaml]
-   [dda.c4k-common.common :as cm]
+  [dda.c4k-common.common :as cm]
-   [dda.c4k-common.monitoring :as mon]
+  [dda.c4k-common.monitoring :as mon]
-   [dda.c4k-forgejo.forgejo :as forgejo]
+  [dda.c4k-forgejo.forgejo :as forgejo]
-   [dda.c4k-forgejo.backup :as backup]
+  [dda.c4k-forgejo.backup :as backup]
-   [dda.c4k-common.postgres :as postgres]
+  [dda.c4k-common.postgres :as postgres]
-   [dda.c4k-common.namespace :as ns]))
+  [dda.c4k-common.namespace :as ns]))
 
-(def config-defaults {:namespace "forgejo"
+(def config-defaults {:issuer "staging", :deploy-federated "false"})
-                      :issuer "staging"
-                      :deploy-federated "false"
-                      :federation-enabled "false"
-                      :db-name "forgejo"
-                      :pv-storage-size-gb 5
-                      :pvc-storage-class-name ""
-                      :postgres-image "postgres:14"
-                      :postgres-size :2gb})
 (def rate-limit-defaults {:max-rate 10, :max-concurrent-requests 5})
 
-(def config? (s/keys :req-un [::forgejo/fqdn
+(def config? (s/keys :req-un [::forgejo/fqdn 
-                              ::forgejo/mailer-from
+                              ::forgejo/mailer-from 
-                              ::forgejo/mailer-host
+                              ::forgejo/mailer-host 
                               ::forgejo/mailer-port
                               ::forgejo/service-noreply-address]
-                     :opt-un [::forgejo/issuer
+                     :opt-un [::forgejo/issuer 
                               ::forgejo/deploy-federated
-                              ::forgejo/federation-enabled
                               ::forgejo/default-app-name 
                               ::forgejo/service-domain-whitelist
                               ::forgejo/forgejo-image-version-overwrite
@@ -42,39 +33,38 @@
 
 (def vol? (s/keys :req-un [::forgejo/volume-total-storage-size]))
 
-(defn config-objects [config] ; ToDo: ADR for generate functions - vector or no vector?
+(def postgres-config {:db-name "forgejo"
-  (let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path)]
+                      :pv-storage-size-gb 5
+                      :pvc-storage-class-name ""
+                      :postgres-image "postgres:14"
+                      :postgres-size :2gb})
+
+(defn k8s-objects [config auth] ; ToDo: ADR for generate functions - vector or no vector?
+  (let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path)
+        resolved-config (merge {:namespace "forgejo"} postgres-config config)]
     (map yaml/to-string
          (filter #(not (nil? %))
                  (cm/concat-vec
-                  (ns/generate config)
+                  (ns/generate resolved-config)
-                  [(postgres/generate-configmap config)
+                  [(postgres/generate-config resolved-config)
-                   (when (contains? config :postgres-data-volume-path)
+                   (postgres/generate-secret {:namespace "forgejo"} auth)
-                     (postgres/generate-persistent-volume (select-keys config [:postgres-data-volume-path :pv-storage-size-gb])))
+                   (when (contains? resolved-config :postgres-data-volume-path)
-                   (postgres/generate-pvc (merge config {:pvc-storage-class-name storage-class}))
+                     (postgres/generate-persistent-volume (select-keys resolved-config [:postgres-data-volume-path :pv-storage-size-gb])))
-                   (postgres/generate-deployment config)
+                   (postgres/generate-pvc (merge resolved-config {:pvc-storage-class-name storage-class}))
-                   (postgres/generate-service config)
+                   (postgres/generate-deployment resolved-config)
-                   (forgejo/generate-deployment config)
+                   (postgres/generate-service resolved-config)
+                   (forgejo/generate-deployment resolved-config)
                    (forgejo/generate-service)
                    (forgejo/generate-service-ssh)
-                   (forgejo/generate-data-volume config)
+                   (forgejo/generate-data-volume resolved-config)
-                   (forgejo/generate-appini-env config)]
+                   (forgejo/generate-appini-env resolved-config)
-                  (forgejo/generate-ratelimit-ingress-and-cert config) ; this function has a vector as output
+                   (forgejo/generate-secrets auth)
-                  (when (contains? config :restic-repository)
+                   (forgejo/generate-rate-limit-middleware rate-limit-defaults)] ; this does not have a vector as output
-                    [(backup/generate-config config)
+                  (forgejo/generate-rate-limit-ingress-and-cert resolved-config) ; this function has a vector as output
+                  (when (contains? resolved-config :restic-repository)
+                    [(backup/generate-config resolved-config)
+                     (backup/generate-secret auth)
                      (backup/generate-cron)
-                     (backup/generate-backup-restore-deployment config)])
+                     (backup/generate-backup-restore-deployment resolved-config)])
-                  (when (contains? config :mon-cfg)
+                  (when (:contains? resolved-config :mon-cfg)
-                    (mon/generate-config)))))))
+                    (mon/generate (:mon-cfg resolved-config) (:mon-auth auth))))))))
-
-(defn auth-objects [config auth]
-  (map yaml/to-string
-       (filter #(not (nil? %))
-               (cm/concat-vec
-                (ns/generate config)
-                [(postgres/generate-secret config auth)
-                 (forgejo/generate-secrets auth)]
-                (when (contains? config :restic-repository)
-                  [(backup/generate-secret auth)])
-                (when (contains? config :mon-cfg)
-                  (mon/generate-auth (:mon-cfg config) (:mon-auth auth)))))))

src/main/cljc/dda/c4k_forgejo/forgejo.cljc

@@ -33,7 +33,6 @@
 (s/def ::default-app-name string?)
 (s/def ::fqdn pred/fqdn-string?)
 (s/def ::deploy-federated boolean-string?)
-(s/def ::federation-enabled boolean-string?)
 (s/def ::mailer-from pred/bash-env-string?)
 (s/def ::mailer-host pred/bash-env-string?)
 (s/def ::mailer-port pred/bash-env-string?)
@@ -54,7 +53,6 @@
                               ::service-noreply-address]
                      :opt-un [::issuer
                               ::deploy-federated
-                              ::federation-enabled
                               ::default-app-name
                               ::service-domain-whitelist
                               ::forgejo-image-version-overwrite]))
@@ -69,11 +67,11 @@
 (defn data-storage-by-volume-size
   [total]
   total)
-;;TODO: remove unneccessaries, fedaration is merged
+
 (def federated-image-name "domaindrivenarchitecture/c4k-forgejo-federated")
 (def federated-image-version "latest")
 (def non-federated-image-name "codeberg.org/forgejo/forgejo")
-(def non-federated-image-version "8.0.3")
+(def non-federated-image-version "1.19")
 
 (defn-spec generate-image-str string?
   [config config?]
@@ -90,7 +88,7 @@
 (defn generate-appini-env
   [config]
   (let [{:keys [default-app-name
-                federation-enabled
+                deploy-federated
                 fqdn
                 mailer-from
                 mailer-host
@@ -99,19 +97,19 @@
                 service-noreply-address]
          :or {default-app-name "forgejo instance"
               service-domain-whitelist fqdn}} config
-        federation-enabled-bool (boolean-from-string federation-enabled)]
+        deploy-federated-bool (boolean-from-string deploy-federated)]
     (->
      (yaml/load-as-edn "forgejo/appini-env-configmap.yaml")
-     (cm/replace-all-matching-values-by-new-value "APPNAME" default-app-name)
+     (cm/replace-all-matching "APPNAME" default-app-name)
-     (cm/replace-all-matching-values-by-new-value "FQDN" fqdn)
+     (cm/replace-all-matching "FQDN" fqdn)
-     (cm/replace-all-matching-values-by-new-value "URL" (str "https://" fqdn))
+     (cm/replace-all-matching "URL" (str "https://" fqdn))
-     (cm/replace-all-matching-values-by-new-value "FROM" mailer-from)
+     (cm/replace-all-matching "FROM" mailer-from)
-     (cm/replace-all-matching-values-by-new-value "MAILERHOST" mailer-host)
+     (cm/replace-all-matching "MAILERHOST" mailer-host)
-     (cm/replace-all-matching-values-by-new-value "MAILERPORT" mailer-port)
+     (cm/replace-all-matching "MAILERPORT" mailer-port)
-     (cm/replace-all-matching-values-by-new-value "WHITELISTDOMAINS" service-domain-whitelist)
+     (cm/replace-all-matching "WHITELISTDOMAINS" service-domain-whitelist)
-     (cm/replace-all-matching-values-by-new-value "NOREPLY" service-noreply-address)
+     (cm/replace-all-matching "NOREPLY" service-noreply-address)
-     (cm/replace-all-matching-values-by-new-value "IS_FEDERATED" 
+     (cm/replace-all-matching "IS_FEDERATED" 
-                                                  (if federation-enabled-bool
+                                                  (if deploy-federated-bool
                                                     "true"
                                                     "false")))))
 
@@ -128,17 +126,35 @@
      (cm/replace-all-matching "MAILERUSER" (b64/encode mailer-user))
      (cm/replace-all-matching "MAILERPW" (b64/encode mailer-pw)))))
 
-(defn-spec generate-ratelimit-ingress-and-cert seq?
+(defn generate-ingress-and-cert
+  [config]
+  (let [{:keys [fqdn]} config]
+    (ing/generate-ingress-and-cert
+     (merge
+      {:service-name "forgejo-service"
+       :service-port 3000
+       :fqdns [fqdn]}
+      config))))
+
+(defn-spec generate-rate-limit-ingress-and-cert pred/map-or-seq?
   [config config?]
-  (let [{:keys [fqdn max-rate max-concurrent-requests namespace]} config]
+  (->
-    (ing/generate-simple-ingress (merge
+   (generate-ingress-and-cert config) ; returns a vector
-                                  {:service-name "forgejo-service"
+   (#(assoc-in % ; Attention: heavily relying on the output order of ing/generate-ingress-and-cert
-                                   :service-port 3000
+               [1 :metadata :annotations :traefik.ingress.kubernetes.io/router.middlewares]
-                                   :fqdns [fqdn]
+               (str
-                                   :average-rate max-rate
+                (-> (second %) :metadata :annotations :traefik.ingress.kubernetes.io/router.middlewares)
-                                   :burst-rate max-concurrent-requests
+                ", default-ratelimit@kubernetescrd")))))
-                                   :namespace namespace}
+
-                                  config))))
+
+; using :average and :burst seems sensible, :period may be interesting for fine tuning later on
+(defn-spec generate-rate-limit-middleware pred/map-or-seq?
+  [config rate-limit-config?]
+  (let [{:keys [max-rate max-concurrent-requests]} config]
+  (->
+   (yaml/load-as-edn "forgejo/middleware-ratelimit.yaml")
+   (cm/replace-key-value :average max-rate)
+   (cm/replace-key-value :burst max-concurrent-requests))))
 
 (defn-spec generate-data-volume pred/map-or-seq?
   [config vol?]
@@ -150,9 +166,11 @@
 
 (defn-spec generate-deployment pred/map-or-seq?
   [config config?]
+  (let [{:keys [deploy-federated]} config
+        deploy-federated-bool (boolean-from-string deploy-federated)]
     (->
      (yaml/load-as-edn "forgejo/deployment.yaml")
-     (cm/replace-all-matching "IMAGE_NAME" (generate-image-str config))))
+     (cm/replace-all-matching "IMAGE_NAME" (generate-image-str config)))))
 
 (defn generate-service
   []

src/main/cljs/dda/c4k_forgejo/browser.cljs

@@ -4,7 +4,7 @@
    [clojure.tools.reader.edn :as edn]
    [dda.c4k-forgejo.core :as core]
    [dda.c4k-forgejo.forgejo :as forgejo]
-   [dda.c4k-common.browser :as br]
+   [dda.c4k-common.browser :as br]   
    [dda.c4k-common.common :as cm]))
 
 (defn generate-group
@@ -73,13 +73,14 @@
       :mailer-host (br/get-content-from-element "mailer-host")
       :mailer-port (br/get-content-from-element "mailer-port")
       :service-noreply-address (br/get-content-from-element "service-noreply-address")
-      :volume-total-storage-size (br/get-content-from-element "volume-total-storage-size" :deserializer js/parseInt)}
+      :volume-total-storage-size (br/get-content-from-element "volume-total-storage-size" :deserializer js/parseInt)}     
      (when (not (st/blank? issuer))
        {:issuer issuer})
      (when (not (st/blank? app-name))
        {:default-app-name app-name})
      (when (not (st/blank? domain-whitelist))
-       {:service-domain-whitelist domain-whitelist}))))
+       {:service-domain-whitelist domain-whitelist})
+     )))
 
 (defn validate-all! []
   (br/validate! "fqdn" ::forgejo/fqdn)
@@ -90,7 +91,7 @@
   (br/validate! "deploy-federated" ::forgejo/deploy-federated :optional true)
   (br/validate! "issuer" ::forgejo/issuer :optional true)
   (br/validate! "app-name" ::forgejo/default-app-name :optional true)
-  (br/validate! "domain-whitelist" ::forgejo/service-domain-whitelist :optional true)
+  (br/validate! "domain-whitelist" ::forgejo/service-domain-whitelist :optional true)  
   (br/validate! "volume-total-storage-size" ::forgejo/volume-total-storage-size :deserializer js/parseInt)
   (br/validate! "auth" forgejo/auth? :deserializer edn/read-string)
   (br/set-form-validated!))
@@ -102,21 +103,16 @@
 
 (defn init []
   (br/append-hickory (generate-content-div))
-  (let [config-only false
+  (-> js/document
-        auth-only false]
+      (.getElementById "generate-button")
-    (-> js/document
+      (.addEventListener "click"
-        (.getElementById "generate-button")
+                         #(do (validate-all!)
-        (.addEventListener "click"
+                              (-> (cm/generate-common
-                           #(do (validate-all!)
+                                   (config-from-document)
-                                (-> (cm/generate-cm
+                                   (br/get-content-from-element "auth" :deserializer edn/read-string)
-                                     (config-from-document)
+                                   core/config-defaults
-                                     (br/get-content-from-element "auth" :deserializer edn/read-string)
+                                   core/k8s-objects)
-                                     core/config-defaults
+                               (br/set-output!)))))
-                                     core/config-objects
-                                     core/auth-objects
-                                     config-only
-                                     auth-only)
-                                    (br/set-output!))))))
   (add-validate-listener "fqdn")
   (add-validate-listener "deploy-federated")
   (add-validate-listener "mailer-from")
@@ -124,7 +120,7 @@
   (add-validate-listener "mailer-port")
   (add-validate-listener "service-noreply-address")
   (add-validate-listener "app-name")
-  (add-validate-listener "domain-whitelist")
+  (add-validate-listener "domain-whitelist")  
   (add-validate-listener "volume-total-storage-size")
   (add-validate-listener "issuer")
   (add-validate-listener "auth"))

src/main/resources/backup/backup-restore-deployment.yaml

@@ -21,7 +21,7 @@ spec:
       - image: domaindrivenarchitecture/c4k-forgejo-backup
         name: backup-app
         imagePullPolicy: IfNotPresent
-        command: ["wait.bb"]
+        command: ["/entrypoint-start-and-wait.sh"]
         env:
         - name: POSTGRES_USER
           valueFrom:

src/main/resources/backup/cron.yaml

@@ -17,7 +17,7 @@ spec:
           - name: backup-app
             image: domaindrivenarchitecture/c4k-forgejo-backup
             imagePullPolicy: IfNotPresent
-            command: ["backup.bb"]
+            command: ["/entrypoint.sh"]
             env:
             - name: POSTGRES_USER
               valueFrom:

src/main/resources/forgejo/appini-env-configmap.yaml

@@ -16,6 +16,7 @@ data:
   FORGEJO__database__NAME: forgejo
   FORGEJO__database__LOG_SQL: "false"
   FORGEJO__database__SSL_MODE: disable
+  FORGEJO__database__CHARSET: utf8
 
   #[DEFAULT]
   APP_NAME: APPNAME
@@ -36,12 +37,12 @@ data:
   #[mailer]
   FORGEJO__mailer__ENABLED: "true"
   FORGEJO__mailer__FROM: FROM
-  FORGEJO__mailer__PROTOCOL: smtp+starttls
+  FORGEJO__mailer__MAILER_TYPE: smtp+startls
   FORGEJO__mailer__SMTP_ADDR: MAILERHOST
   FORGEJO__mailer__SMTP_PORT: MAILERPORT
 
   #[oauth2]
-  FORGEJO__oauth2__ENABLED: "true"
+  FORGEJO__oauth2__ENABLE: "true"
 
   #[openid]
   FORGEJO__openid__ENABLE_OPENID: "true"
@@ -75,7 +76,7 @@ data:
   FORGEJO__service__REQUIRE_SIGNIN_VIEW: "false"
   FORGEJO__service__REGISTER_EMAIL_CONFIRM: "true"
   FORGEJO__service__ENABLE_NOTIFY_MAIL: "true"
-  FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: WHITELISTDOMAINS 
+  FORGEJO__service__EMAIL_DOMAIN_WHITELIST: WHITELISTDOMAINS 
   FORGEJO__service__ALLOW_ONLY_EXTERNAL_REGISTRATION: "false"
   FORGEJO__service__ENABLE_BASIC_AUTHENTICATION: "true"
   FORGEJO__service__ENABLE_CAPTCHA: "false"

src/main/resources/forgejo/middleware-ratelimit.yaml

@@ -0,0 +1,9 @@
+apiVersion: traefik.containo.us/v1alpha1
+kind: Middleware
+metadata:
+  name: ratelimit
+  namespace: forgejo
+spec:
+  rateLimit: # Config options for rate limiting: https://doc.traefik.io/traefik/middlewares/http/ratelimit/
+    average: AVG
+    burst: BRS

src/test/cljc/dda/c4k_forgejo/forgejo_test.cljc

@@ -14,7 +14,7 @@
 
 (deftest should-generate-image-str
   (testing "non-federated-image"
-    (is (= "codeberg.org/forgejo/forgejo:8.0.3"
+    (is (= "codeberg.org/forgejo/forgejo:1.19"
            (cut/generate-image-str {:fqdn "test.de"
                                     :mailer-from ""
                                     :mailer-host "m.t.de"
@@ -63,12 +63,12 @@
           :FORGEJO__server__ROOT_URL-c2 "https://test.com",
           :FORGEJO__server__SSH_DOMAIN-c1 "test.de",
           :FORGEJO__server__SSH_DOMAIN-c2 "test.com",
-          :FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST-c1 "adb.de",
+          :FORGEJO__service__EMAIL_DOMAIN_WHITELIST-c1 "adb.de",
-          :FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST-c2 "test.com,test.net",
+          :FORGEJO__service__EMAIL_DOMAIN_WHITELIST-c2 "test.com,test.net",
           :FORGEJO__service__NO_REPLY_ADDRESS-c1 "",
           :FORGEJO__service__NO_REPLY_ADDRESS-c2 "noreply@test.com"}
          (th/map-diff (cut/generate-appini-env {:default-app-name ""
-                                                :federation-enabled "false"
+                                                :deploy-federated "false"
                                                 :fqdn "test.de"
                                                 :mailer-from ""
                                                 :mailer-host "m.t.de"
@@ -76,7 +76,7 @@
                                                 :service-domain-whitelist "adb.de"
                                                 :service-noreply-address ""})
                       (cut/generate-appini-env {:default-app-name "test forgejo"
-                                                :federation-enabled "true"
+                                                :deploy-federated "true"
                                                 :fqdn "test.com"
                                                 :mailer-from "test@test.com"
                                                 :mailer-host "mail.test.com"
@@ -97,7 +97,7 @@
               :spec
               {:containers
                [{:name "forgejo",
-                 :image "codeberg.org/forgejo/forgejo:8.0.3",
+                 :image "codeberg.org/forgejo/forgejo:1.19",
                  :imagePullPolicy "IfNotPresent",
                  :envFrom [{:configMapRef {:name "forgejo-env"}} {:secretRef {:name "forgejo-secrets"}}],
                  :volumeMounts [{:name "forgejo-data-volume", :mountPath "/data"}],
@@ -163,3 +163,26 @@
           :storage-c2 "15Gi"}
          (th/map-diff (cut/generate-data-volume {:volume-total-storage-size 1})
                       (cut/generate-data-volume {:volume-total-storage-size 15})))))
+
+(deftest should-generate-middleware-ratelimit
+  (is (= {:apiVersion "traefik.containo.us/v1alpha1",
+          :kind "Middleware",
+          :metadata {:name "ratelimit", :namespace "forgejo"},
+          :spec {:rateLimit {:average 10, :burst 5}}}
+         (cut/generate-rate-limit-middleware {:max-rate 10, :max-concurrent-requests 5}))))
+
+(deftest should-generate-middleware-ratelimit-ingress-and-cert
+  (is (= {:traefik.ingress.kubernetes.io/router.entrypoints "web, websecure",
+          :traefik.ingress.kubernetes.io/router.middlewares
+          "default-redirect-https@kubernetescrd, default-ratelimit@kubernetescrd",
+          :metallb.universe.tf/address-pool "public"}
+         (-> (second
+              (cut/generate-rate-limit-ingress-and-cert
+               {:fqdn "test.de"
+                :mailer-from ""
+                :mailer-host "m.t.de"
+                :mailer-port "123"
+                :service-noreply-address ""
+                :average 10
+                :burst 5}))
+             :metadata :annotations))))