Compare commits

..

No commits in common. "main" and "3.4.1" have entirely different histories.
main ... 3.4.1

39 changed files with 619 additions and 502 deletions

View file

@ -1,7 +1,7 @@
# convention 4 kubernetes: c4k-forgejo
[![Clojars Project](https://img.shields.io/clojars/v/org.domaindrivenarchitecture/c4k-forgejo.svg)](https://clojars.org/org.domaindrivenarchitecture/c4k-forgejo) [![pipeline status](https://gitlab.com/domaindrivenarchitecture/c4k-forgejo/badges/master/pipeline.svg)](https://gitlab.com/domaindrivenarchitecture/c4k-forgejo/-/commits/main)
[<img src="https://domaindrivenarchitecture.org/img/delta-chat.svg" width=20 alt="DeltaChat"> chat over e-mail](mailto:buero@meissa-gmbh.de?subject=community-chat) | [<img src="https://meissa.de/images/parts/contact/mastodon36_hue9b2464f10b18e134322af482b9c915e_5501_filter_14705073121015236177.png" width=20 alt="M"> meissa@social.meissa-gmbh.de](https://social.meissa-gmbh.de/@meissa) | [Blog](https://domaindrivenarchitecture.org) | [Website](https://meissa.de)
[<img src="https://domaindrivenarchitecture.org/img/delta-chat.svg" width=20 alt="DeltaChat"> chat over e-mail](mailto:buero@meissa-gmbh.de?subject=community-chat) | [<img src="https://meissa-gmbh.de/img/community/Mastodon_Logotype.svg" width=20 alt="team@social.meissa-gmbh.de"> team@social.meissa-gmbh.de](https://social.meissa-gmbh.de/@team) | [Website & Blog](https://domaindrivenarchitecture.org)
## Purpose
@ -46,8 +46,8 @@ Development happens at: https://repo.prod.meissa.de/meissa/c4k-forgejo
Mirrors are:
* https://codeberg.org/meissa/c4k-forgejo (Issues and PR)
* https://gitlab.com/domaindrivenarchitecture/c4k-forgejo (CI)
* https://codeberg.org/meissa/c4k-forgejo (issues and PR)
* https://github.com/DomainDrivenArchitecture/c4k-forgejo
For more details about our repository model see: https://repo.prod.meissa.de/meissa/federate-your-repos
@ -55,6 +55,6 @@ For more details about our repository model see: https://repo.prod.meissa.de/mei
## License
Copyright © 2023, 2024 meissa GmbH
Copyright © 2023 meissa GmbH
Licensed under the [Apache License, Version 2.0](LICENSE) (the "License")
Pls. find licenses of our subcomponents [here](doc/SUBCOMPONENT_LICENSE)

View file

@ -10,32 +10,32 @@
## Manual init the restic repository for the first time
1. apply backup-and-restore pod:
`kubectl -n forgejo scale deployment backup-restore --replicas=1`
`kubectl scale deployment backup-restore --replicas=1`
2. exec into pod and execute restore pod (press tab to get your exact pod name)
`kubectl -n forgejo exec -it backup-restore-... -- /usr/local/bin/init.bb`
`kubectl exec -it backup-restore-... -- /usr/local/bin/init.sh`
3. remove backup-and-restore pod:
`kubectl -n forgejo scale deployment backup-restore --replicas=0`
`kubectl scale deployment backup-restore --replicas=0`
## Manual backup the restic repository for the first time
1. apply backup-and-restore pod:
`kubectl -n forgejo scale deployment backup-restore --replicas=1`
`kubectl scale deployment backup-restore --replicas=1`
2. exec into pod and execute backup pod (press tab to get your exact pod name)
`kubectl -n forgejo exec -it backup-restore-... -- /usr/local/bin/backup.bb`
`kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh`
3. remove backup-and-restore pod:
`kubectl -n forgejo scale deployment backup-restore --replicas=0`
`kubectl scale deployment backup-restore --replicas=0`
## Manual restore
1. apply backup-and-restore pod:
`kubectl -n forgejo scale deployment backup-restore --replicas=1`
`kubectl scale deployment backup-restore --replicas=1`
2. Scale down forgejo deployment:
`kubectl -n forgejo scale deployment forgejo --replicas=0`
`kubectl scale deployment forgejo --replicas=0`
3. exec into pod and execute restore pod (press tab to get your exact pod name)
`kubectl -n forgejo exec -it backup-restore-... -- /usr/local/bin/restore.bb`
`kubectl exec -it backup-restore-... -- /usr/local/bin/restore.sh`
4. Start forgejo again:
`kubectl -n forgejo scale deployment forgejo --replicas=1`
`kubectl scale deployment forgejo --replicas=1`
5. remove backup-and-restore pod:
`kubectl -n forgejo scale deployment backup-restore --replicas=0`
`kubectl scale deployment backup-restore --replicas=0`

41
doc/Releasing.md Normal file
View file

@ -0,0 +1,41 @@
# Release process
## ... for testing (snapshots)
Make sure your clojars.org credentials are correctly set in your ~/.lein/profiles.clj file.
``` bash
git add .
git commit
```
``` bash
lein deploy # or lein deploy clojars
```
## ... for stable release
Make sure tags are protected in gitlab:
Repository Settings -> Protected Tags -> set \*.\*.\* as tag and save.
``` bash
git checkout main # for old projects replace main with master
git add .
git commit
```
Execute tests
``` bash
shadow-cljs compile test
node target/node-tests.js
lein test
```
Release with type (NONE, PATCH, MINOR, MAJOR):
``` bash
RELEASE_TYPE=[TYPE] pyb prepare_release after_publish
```
Done.

View file

@ -1,135 +0,0 @@
# Playbook Upgrade from 1.19 to 7.0.5
## Info: Relevant Breaking Changes:
* 1.19.3: First version under consideration
* 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0
* 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0
* 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0
* 8.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#8-0-0
## Preparations
1. Stop Forgejo Prod: `k scale deployment forgejo --replicas=0`
1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'`
1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1`
1. Execute Manual Backup: `kubectl exec -n forgejo -it backup-restore-... -- /usr/local/bin/backup.sh`
### Create 2nd Repo Prod Server
1. Terraform Preparations for 2nd Server: TODO
1. Install c4k-forgejo Version `3.5.0`!
with config `"forgejo-image-version-overwrite": "1.19.3-0"` (in server-setup)
1. Stop Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Disable Backup Cron: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'`
1. Scale up Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1`
1. Restore Forgejo Backup: See [BackupAndRestore.md](BackupAndRestore.md)
1. Check for `..._INSTALL_LOCK: true` in ConfigMap `forgejo-env`
1. Scale up Forgejo Deployment and check for (startup) problems: `k scale -n forgejo deployment forgejo --replicas=1`
## Upgrade to 1.20.1-0
1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Adjust configmap: `k edit -n forgejo cm forgejo-env`
1. Remove `FORGEJO__database__CHARSET: utf8` (This was a misconfiguration, since this option only had effect for mysql dbs)
1. Change `FORGEJO__mailer__MAILER_TYPE: smtp+startls` TO `FORGEJO__mailer__PROTOCOL: smtp+starttls` (Missed deprecation from 1.19)
1. Change `FORGEJO__service__EMAIL_DOMAIN_WHITELIST: repo.test.meissa.de` TO `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: repo.test.meissa.de` (Fallback deprecation in 1.21)
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `1.20.1-0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors: `k logs -n forgejo forgejo-...`
## Upgrade to 1.21.1-0
1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `1.21.1-0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors: `k logs -n forgejo forgejo-...`
1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`.
## Upgrade to 7.0.0
1. Check DB Version.
1. MariaDB or MySQL needs to be 8.0 or higher.
2. Postgres needs to be 12 or higher
1. API Endpoints
1. Check if the [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) API endpoint is used
1. as the per_page param is not used for [limit](https://codeberg.org/forgejo/forgejo/commit/0aab2d38a7d91bc8caff332e452364468ce52d9a) anymore
2. Check if [/repos/{owner}/{repo}/push_mirrors](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) and [/repos/{owner}/{repo}/push_mirrors](https://code.forgejo.org/api/swagger/#/repository/repoAddPushMirror) API endpoints are used
1. The date format of created and last_update fields are now [timestamps](https://codeberg.org/forgejo/forgejo/commit/0ee7cbf725f45650136be45f8e0f74d395f73b5c)
3. [pprof](https://forgejo.org/docs/v7.0/admin/config-cheat-sheet/#server-server) endpoint changed labels
1. graceful-lifecycle to gracefulLifecycle
2. process-type to processType
3. process-description to processDescription This allows for those endpoints to be scraped by services requiring prometheus style labels such as grafana-agent.
1. The Gitea themes were renamed and the \[ui\].THEMES setting must be changed as follows:
1. gitea is replaced by gitea-light
2. arc-green is replaced by gitea-dark
3. auto is replaced by gitea-auto
1. Migration warning
2. If the logs show a line like the following, run `doctor convert` to fix it.
3. Current database is using a case-insensitive collation "utf8mb4_general_ci"
4. Large instances may experience slow migrations when the database is upgraded to support SHA-256 git repositories.
1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Adjust configmap: `k edit -n forgejo cm forgejo-env`
1. Change `FORGEJO__oauth2__ENABLE: "true"` TO `FORGEJO__oauth2__ENABLED: "true"`
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `7.0.0` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors: `k logs -n forgejo forgejo-...`
## Upgrade to 8.0.3 (no relevant breaking changes)
1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `8.0.3` with `k edit -n forgejo deployment forgejo`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors: `k logs -n forgejo forgejo-...`
## Enable Federation
1. Scale down Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=0`
1. Adjust configmap: `k edit -n forgejo cm forgejo-env`
1. Change `FORGEJO__federation__ENABLED: "false"` TO `FORGEJO__federation__ENABLED: "true"`
1. Delete app.ini: `k exec -n forgejo -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Scale up Forgejo Deployment: `k scale -n forgejo deployment forgejo --replicas=1`
1. Check for errors: `k logs -n forgejo forgejo-...`
## Post Work
1. Switch DNS to new server
1. Reenable Backup Cron on new server: `k patch -n forgejo cronjobs forgejo-backup -p '{"spec" : {"suspend" : false }}'`
1. Execute manual Backup on new server: `kubectl exec -n forgejo -it backup-restore-... -- /usr/local/bin/backup.sh`
1. Scale down Backup-Restore Deployment: `kubectl scale -n forgejo deployment backup-restore --replicas=1`
1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens!
1. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work.
## Known Errors
### Error in v1.20.1-0
In the logs the following error can be found. This will be resolved automatically with the next upgrade (v1.21).
```
2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21
```
# Add Shynet Analytics
1. Log into shynet & create new Service
1. Copy the generated html snippet and save it somewhere you remember
1. SSH into prod server
1. Make the necessary folders and files in forgejo data dir:
1. `kubectl exec -n forgejo -it forgejo-... -- bash`
1. `mkdir -p /data/gitea/templates/custom`
1. `touch /data/gitea/templates/custom/footer.tmpl`
1. Open the `footer.tmpl` and paste the saved snippet
1. Restart the pod
1. `k scale -n forgejo deployment forgejo --replicas=0`
1. `k scale -n forgejo deployment forgejo --replicas=1`
1. Add Information about analytics: Clone Datenschutz Repo
1. `git clone ssh://git@repo.prod.meissa.de:2222/meissa/Datenschutz.git`
1. Merge forgejo-upgrade into main
1. `git merge forgejo-upgrade`
1. Push to origin
1. `git push`

View file

@ -0,0 +1,87 @@
# Playbook Upgrade from 1.19 to 7.0.5
## Info: Relevant Breaking Changes:
* 1.19.3:Current version
* 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0
* 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0
* 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0
## Preparations
1. Stop Forgejo Prod: `k scale deployment forgejo --replicas=0`
1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'`
1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1`
1. Execute Manual Backup: `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh`
### Create 2nd Repo Prod Server
1. Terraform Preparations for 2nd Server: TODO
1. Install c4k-forgejo Version TODO
with config `"forgejo-image-version-overwrite": "1.19.3-0"`
1. Stop Forgejo Deployment: `k scale deployment forgejo --replicas=0`
1. Disable Backup Cron: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : true }}'`
1. Scale up Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1`
1. Restore Forgejo Backup: See [BackupAndRestore.md](BackupAndRestore.md)
1. Check for `..._INSTALL_LOCK: true` in ConfigMap `forgejo-env`
1. Scale up Forgejo Deployment and check for (startup) problems: `k scale deployment forgejo --replicas=1`
## Upgrade to 1.20.1-0
1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0`
1. Adjust configmap: `k edit cm forgejo-env`
1. Remove `FORGEJO__database__CHARSET: utf8` (This was a misconfiguration, since this option only had effect for mysql dbs)
1. Change `FORGEJO__mailer__MAILER_TYPE: smtp+startls` TO `FORGEJO__mailer__PROTOCOL: smtp+starttls` (Missed deprecation from 1.19)
1. Change `FORGEJO__service__EMAIL_DOMAIN_WHITELIST: repo.test.meissa.de` TO `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: repo.test.meissa.de` (Fallback deprecation in 1.21)
1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `1.20.1-0` with `k edit deployment forgejo`
1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1`
1. Check for errors
## Upgrade to 1.21.1-0
1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0`
1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `1.21.1-0` with `k edit deployment forgejo`
1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1`
1. Check for errors
1. After upgrading, login as an admin, go to the `/admin` page and click run `Sync missed branches from git data to databases` (`Fehlende Branches aus den Git-Daten in die Datenbank synchronisieren`). If this is not done there will be messages such as `LoadBranches: branch does not exist in the logs`.
## Upgrade to 7.0.0
1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0`
1. Adjust configmap: `k edit cm forgejo-env`
1. Change `FORGEJO__oauth2__ENABLE: "true"` TO `FORGEJO__oauth2__ENABLED: "true"`
1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `7.0.0` with `k edit deployment forgejo`
1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1`
1. Check for errors
## Upgrade to 7.0.5 (no breaking changes)
TODO: Upgrade to 8.0.0 instead after Release!
1. Scale down Forgejo Deployment: `k scale deployment forgejo --replicas=0`
1. Delete app.ini: `k exec -it backup-restore-... -- rm /var/backups/gitea/conf/app.ini`
1. Set version to `7.0.5` with `k edit deployment forgejo`
1. Scale up Forgejo Deployment: `k scale deployment forgejo --replicas=1`
1. Check for errors
## Post Work
1. Switch DNS to new server
1. Reenable Backup Cron on new server: `k patch cronjobs forgejo-backup -p '{"spec" : {"suspend" : false }}'`
1. Execute manual Backup on new server: `kubectl exec -it backup-restore-... -- /usr/local/bin/backup.sh`
1. Scale down Backup-Restore Deployment: `kubectl scale deployment backup-restore --replicas=1`
1. The scope of all access tokens might (invisibly) have changed (in v1.20). Thus, rotate all tokens!
1. Users should check their ssh keys: if they use rsa keys the minimum length should be 3072 bits! However, shorter keys should still work.
## Known Errors
### Error in v1.20.1-0
In the logs the following error can be found. This will be resolved automatically with the next upgrade (v1.21).
```
2024/07/08 08:31:30 ...g/config_provider.go:321:deprecatedSetting() [E] Deprecated fallback `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead. This fallback will be/has been removed in 1.21
```

195
doc/Upgrading.md Normal file
View file

@ -0,0 +1,195 @@
# Upgrading process
## adhoc (on kubernetes cluster)
Ssh into your kubernetes cluster running the forgejo instance.
``` bash
kubectl edit configmap forgejo-env
# make sure INSTALL_LOCK under security is set to true to disable the installation screen
# save and exit
kubectl edit deployments forgejo
# search for your current forgejo version, e.g. 1.19
# replace with new version
# save and exit
kubectl scale deployment forgejo --replicas=0
kubectl scale deployment forgejo --replicas=1
```
Logging into the admin account should now show the new version.
You may want to update your c4k-forgejo resources to reflect the changes made on the cluster.
## Upgrading from 1.19
### Config related issues with c4k-forgejo v3.2.2
These errors show in the log, when just upgrading to forgejo v7.0.4 from 1.19 without changing the config.
The related config options are listed below the errors.
- Oauth2: ENABLED instead of ENABLE
- `FORGEJO__oauth2__ENABLED: "true"`
- [E] Deprecated config option `[log]` `ROUTER` present. Use `[log]` `logger.router.MODE` instead.
- `FORGEJO__log_0x2E_logger_0x2E_router__MODE: console, file`
- [E] Deprecated config option `[service]` `EMAIL_DOMAIN_WHITELIST` present. Use `[service]
` `EMAIL_DOMAIN_ALLOWLIST` instead.
- `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: YOUR_ALLOW_LIST`
- [E] Deprecated config option `[mailer]` `MAILER_TYPE` present. Use `[mailer]` `PROTOCOL`
instead.
- [E] Deprecated fallback `[mailer]` `PROTOCOL = smtp+startls` present. Use `[mailer]` `PROTOCOL = smtp+starttls`` instead.
- `FORGEJO__mailer__PROTOCOL: smtp+starttls`
### Breaking Changes since 1.19
#### 1.19.3 & 1.19.4: Version installed by c4k-forgejo v3.2.2
#### 1.20.1-0: Breaking https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-20-1-0
##### app.ini
- Check [queue] section - n/e
- Check [repository.editor] - n/e
- Check [storage] - n/e
- Check ssh_keygen_path in app.ini - n/e
- Is WORK_PATH set? Or app.ini writeable by forgejo server user?
- 1. No
- 2. Yes
- If not, it shows in the logs starting with: `Unable to update WORK_PATH`
- Also ssh pushing will likely fail
- *test ssh*
- Set logger.router.mode as described in environment-to-ini
- See: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/contrib/environment-to-ini
- Check [git.reflog] and maybe move to [git.config] - n/e
- Check [indexer], [mailer], [repository] - n/e
##### tokens
- Scoped and personal access tokens were refactored
- Scope may change, if we have tokens they should be rotated
#### 1.21.1-0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#1-21-1-0
##### custom themes
- Move to `custom/public/assets/`
##### git branches
- `/admin` page and click run Sync missed branches from git data to databases.
##### db - mysql
- c4k uses postgres
##### ssh server
- We don't use host cert used for auth
##### ssh keys
- All users need to check their key length, now 3072
##### tokens
- Finer restrictions might now return 404 errors on users tokens in certain teams with certain restrictions
#### 7.0.0: https://codeberg.org/forgejo/forgejo/src/branch/forgejo/RELEASE-NOTES.md#7-0-0
##### webhooks
- Do we use webhooks?
##### db
- Psql min ver is 12, c4k-common uses 14+
##### api
- [/repos/{owner}/{repo}/releases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases)
- [/repos/{owner}/{repo}/push_mirrors](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors)
- Application profiling
##### repos
- Do we have repo descriptions?
- https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d
##### app.ini
- Check [ui] - n/e
### Vor dem Upgrade
- Host cert used for auth? - no
- Do we use webhooks? - no
- Do we use:
- [/repos/{owner}/{repo}/releases - repoListReleases](https://code.forgejo.org/api/swagger/#/repository/repoListReleases) - no
- In the ListReleases, the `per_page` parameter has been decoupled from the `limit` parameter, we do not use the repoListReleases endpoint
- In the `ArtifactDeploymentApi` in dda-devops-build we only use the `POST` method
- The respective endpoint is [repoCreateRelease](https://code.forgejo.org/api/swagger/#/repository/repoCreateRelease)
- [`/repos/{owner}/{repo}/push_mirrors`](https://code.forgejo.org/api/swagger/#/repository/repoListPushMirrors) - no
- Application profiling - no
- Do we have repo descriptions? - yes
- There is now a sanitizer that only allows links, emphasis, code and emojis
- See: https://codeberg.org/forgejo/forgejo/commit/1075ff74b5050f671c5f9824ae39390230b3c85d
- Our repository descriptions are mostly plaintext and links
### Upgrade plan
TEST indicates actions that only apply to the test server and are ignored in PROD.
PROD indicates actions that only apply to the prod server and are ignored in TEST.
See also the overview for upgrading: https://forgejo.org/docs/latest/admin/upgrade/
- Set up Forgejo server with c4k-forgejo v3.2.2
- Has Forgejo v1.19
- TEST
- Delete old remote ids
- `ssh-keygen -f "/home/${USER}/.ssh/known_hosts" -R "repo.test.meissa.de"`
- Ssh to server
- Forgejo pod downscale
- `k scale deployment forgejo --replicas=0`
- Install lock off
- `k edit cm forgejo-env`
- Set to `FORGEJO__security__INSTALL_LOCK: "false"`
- Forgejo pod upscale
- `k scale deployment forgejo --replicas=1`
- Create admin test or prod admin and install forgejo
- `gopass show server/meissa/forgejo-test` bzw `-prod`
- Forgejo pod downscale
- Install lock on
- Set to `FORGEJO__security__INSTALL_LOCK: "true"`
- TEST
- Forgejo pod upscale
- Log in
- Make Ssh keys
- ed_xyz
- rsa mit 2048
- rsa mit 4096
- Create repos
- Forgejo pod downscale
- PROD
- Backup pod upscale
- `k scale deployment backup-restore --replicas=1`
- Restore backups
- Delete or rename app.ini's in the pod
- Backup pod downscale
- `k scale deployment backup-restore --replicas=0`
- Set image version to 7.0.4 in forgejo deployment
- `k edit deployment.apps forgejo`
- Update configmap:
- Double check install lock enabled
- `FORGEJO__oauth2__ENABLED: "true"`
- `FORGEJO__log_0x2E_logger_0x2E_router__MODE: console, file`
- `FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST:`
- `FORGEJO__mailer__PROTOCOL: smtp+starttls`
- `FORGEJO__federation__ENABLED: true`
- TEST
- Backup pod upscale
- Delete or rename app.ini's in the pod
- Backup pod downscale
- Forgejo pod upscale
- Migrations happen automatically
- `/admin` page and click run Sync missed branches from git data to databases
- and **Sync missed tags ...*
- Rsa keys with size 2048 can not be added anymore. However, it seems they still can be used if they are on the server
- Team members having app tokens need to recreate them with proper scopes
- Add analytics: https://forgejo.org/docs/latest/admin/customization/

View file

@ -6,7 +6,7 @@ from ddadevops import *
name = "c4k-forgejo"
MODULE = "backup"
PROJECT_ROOT_PATH = "../.."
version = "4.0.1-dev"
version = "3.4.1"
@init

View file

@ -1,4 +1,5 @@
FROM domaindrivenarchitecture/dda-backup:latest
# Prepare Entrypoint Script
ADD resources /tmp
RUN /tmp/install.bb
RUN /tmp/install.sh

View file

@ -1,46 +0,0 @@
#!/usr/bin/env bb
(require
'[dda.backup.core :as bc]
'[dda.backup.restic :as rc]
'[dda.backup.postgresql :as pg]
'[dda.backup.backup :as bak])
(def restic-repo {:password-file (bc/env-or-file "RESTIC_PASSWORD_FILE")
:restic-repository (bc/env-or-file "RESTIC_REPOSITORY")})
(def file-config (merge restic-repo {:backup-path "files"
:execution-directory "/var/backups/"
:files ["gitea/" "git/repositories/"]}))
(def db-config (merge restic-repo {:backup-path "pg-database"
:pg-host (bc/env-or-file "POSTGRES_SERVICE")
:pg-port (bc/env-or-file "POSTGRES_PORT")
:pg-db (bc/env-or-file "POSTGRES_DB")
:pg-user (bc/env-or-file "POSTGRES_USER")
:pg-password (bc/env-or-file "POSTGRES_PASSWORD")}))
(def aws-config {:aws-access-key-id (bc/env-or-file "AWS_ACCESS_KEY_ID")
:aws-secret-access-key (bc/env-or-file "AWS_SECRET_ACCESS_KEY")})
(def dry-run {:dry-run true :debug true})
(defn prepare!
[]
(bc/create-aws-credentials! aws-config)
(pg/create-pg-pass! db-config))
(defn restic-repo-init!
[]
(rc/init! file-config)
(rc/init! db-config))
(defn restic-backup!
[]
(bak/backup-file! file-config)
(bak/backup-db! db-config))
(prepare!)
(restic-repo-init!)
(restic-backup!)

View file

@ -0,0 +1,19 @@
#!/bin/bash
set -Eexo pipefail
function main() {
file_env AWS_ACCESS_KEY_ID
file_env AWS_SECRET_ACCESS_KEY
file_env RESTIC_DAYS_TO_KEEP 30
file_env RESTIC_MONTHS_TO_KEEP 12
backup-db-dump
backup-fs-from-directory '/var/backups/' 'gitea/' 'git/repositories/'
}
source /usr/local/lib/functions.sh
source /usr/local/lib/pg-functions.sh
source /usr/local/lib/file-functions.sh
main

View file

@ -1,3 +0,0 @@
{:deps {org.clojure/spec.alpha {:mvn/version "0.4.233"}
orchestra/orchestra {:mvn/version "2021.01.01-1"}
org.domaindrivenarchitecture/dda-backup {:local/root "/usr/local/lib/dda-backup"}}}

View file

@ -1,3 +0,0 @@
{:deps {org.clojure/spec.alpha {:mvn/version "0.4.233"}
orchestra/orchestra {:mvn/version "2021.01.01-1"}
org.domaindrivenarchitecture/dda-build {:mvn/version "0.1.1-SNAPSHOT"}}}

View file

@ -0,0 +1,15 @@
#!/bin/bash
set -Eexo pipefail
function main() {
create-pg-pass
while true; do
sleep 1m
done
}
source /usr/local/lib/functions.sh
source /usr/local/lib/pg-functions.sh
main

View file

@ -0,0 +1,13 @@
#!/bin/bash
set -Eexo pipefail
function main() {
create-pg-pass
/usr/local/bin/backup.sh
}
source /usr/local/lib/functions.sh
source /usr/local/lib/pg-functions.sh
main

View file

@ -1,3 +0,0 @@
#!/usr/bin/env bb
(println "initialized")

View file

@ -0,0 +1,16 @@
#!/bin/bash
set -Eexo pipefail
function main() {
file_env AWS_ACCESS_KEY_ID
file_env AWS_SECRET_ACCESS_KEY
init-database-repo
init-file-repo
}
source /usr/local/lib/functions.sh
source /usr/local/lib/pg-functions.sh
source /usr/local/lib/file-functions.sh
main

View file

@ -1,14 +0,0 @@
#!/usr/bin/env bb
(require
'[dda.image.ubuntu :as ub]
'[dda.image.install :as in])
(ub/upgrade-system!)
(in/install! "bb-backup.edn" :target-name "bb.edn" :mod "0400")
(in/install! "backup.bb")
(in/install! "restore.bb")
(in/install! "list-snapshots.bb")
(in/install! "wait.bb")
(ub/cleanup-container!)

View file

@ -0,0 +1,21 @@
#!/bin/bash
set -exo pipefail
function main()
{
upgradeSystem
install -m 0700 /tmp/entrypoint.sh /
install -m 0700 /tmp/entrypoint-start-and-wait.sh /
install -m 0700 /tmp/init.sh /usr/local/bin/
install -m 0700 /tmp/backup.sh /usr/local/bin/
install -m 0700 /tmp/restore.sh /usr/local/bin/
install -m 0700 /tmp/restic-snapshots.sh /usr/local/bin/
cleanupDocker
} > /dev/null
source /tmp/install_functions_debian.sh
DEBIAN_FRONTEND=noninteractive DEBCONF_NOWARNINGS=yes main

View file

@ -1,28 +0,0 @@
#!/usr/bin/env bb
(require
'[dda.backup.core :as bc]
'[dda.backup.restic :as rc])
(def restic-repo {:password-file (bc/env-or-file "RESTIC_PASSWORD_FILE")
:restic-repository (bc/env-or-file "RESTIC_REPOSITORY")})
(def file-config (merge restic-repo {:backup-path "files"}))
(def db-config (merge restic-repo {:backup-path "pg-database"}))
(def aws-config {:aws-access-key-id (bc/env-or-file "AWS_ACCESS_KEY_ID")
:aws-secret-access-key (bc/env-or-file "AWS_SECRET_ACCESS_KEY")})
(defn prepare!
[]
(bc/create-aws-credentials! aws-config))
(defn list-snapshots!
[]
(rc/list-snapshots! file-config)
(rc/list-snapshots! db-config))
(prepare!)
(list-snapshots!)

View file

@ -0,0 +1,16 @@
#!/bin/bash
set -exo pipefail
function main() {
file_env AWS_ACCESS_KEY_ID
file_env AWS_SECRET_ACCESS_KEY
restic -r ${RESTIC_REPOSITORY}/files snapshots
restic -r ${RESTIC_REPOSITORY}/pg-database snapshots
}
source /usr/local/lib/functions.sh
source /usr/local/lib/file-functions.sh
main

View file

@ -1,46 +0,0 @@
#!/usr/bin/env bb
(require '[babashka.tasks :as tasks]
'[dda.backup.core :as bc]
'[dda.backup.postgresql :as pg]
'[dda.backup.restore :as rs])
(def restic-repo {:password-file (bc/env-or-file "RESTIC_PASSWORD_FILE")
:restic-repository (bc/env-or-file "RESTIC_REPOSITORY")})
(def file-config (merge restic-repo {:backup-path "files"
:restore-target-directory "/var/backups/restore"
:snapshot-id "latest"}))
(def db-config (merge restic-repo {:backup-path "pg-database"
:pg-host (bc/env-or-file "POSTGRES_SERVICE")
:pg-port (bc/env-or-file "POSTGRES_PORT")
:pg-db (bc/env-or-file "POSTGRES_DB")
:pg-user (bc/env-or-file "POSTGRES_USER")
:pg-password (bc/env-or-file "POSTGRES_PASSWORD")
:snapshot-id "latest"}))
(def aws-config {:aws-access-key-id (bc/env-or-file "AWS_ACCESS_KEY_ID")
:aws-secret-access-key (bc/env-or-file "AWS_SECRET_ACCESS_KEY")})
(def dry-run {:dry-run true :debug true})
(defn prepare!
[]
(pg/create-pg-pass! db-config)
(bc/create-aws-credentials! aws-config))
(defn restic-restore!
[]
(rs/restore-file! file-config)
(tasks/shell ["bash" "-c" "rm -rf /var/backups/gitea/*"])
(tasks/shell ["bash" "-c" "rm -rf /var/backups/git/repositories/*"])
(tasks/shell ["mv" "/var/backups/restore/gitea" "/var/backups/"])
(tasks/shell ["mv" "/var/backups/restore/git/repositories" "/var/backups/git/"])
(tasks/shell ["chown" "-R" "1000:1000" "/var/backups"])
(pg/drop-create-db! (merge db-config {:debug true}))
(rs/restore-db! (merge db-config {:debug true})))
(prepare!)
(restic-restore!)

View file

@ -0,0 +1,37 @@
#!/bin/bash
set -Eexo pipefail
function main() {
file_env AWS_ACCESS_KEY_ID
file_env AWS_SECRET_ACCESS_KEY
file_env POSTGRES_DB
file_env POSTGRES_PASSWORD
file_env POSTGRES_USER
# Restore latest snapshot into /var/backups/restore
restore-directory '/var/backups/restore'
rm -rf /var/backups/gitea/*
rm -rf /var/backups/git/repositories/*
cp -r /var/backups/restore/gitea /var/backups/ #ToDo: mv instead of cp or rm -rf after
cp -r /var/backups/restore/git/repositories /var/backups/git/ #ToDo: mv instead of cp or rm -rf after
# adjust file permissions for the git user
chown -R 1000:1000 /var/backups
# TODO: Regenerate Git Hooks? Do we need this?
#/usr/local/bin/gitea -c '/data/gitea/conf/app.ini' admin regenerate hooks
# Restore db
drop-create-db
restore-db
}
source /usr/local/lib/functions.sh
source /usr/local/lib/pg-functions.sh
source /usr/local/lib/file-functions.sh
main

View file

@ -1,27 +0,0 @@
#!/usr/bin/env bb
(require
'[dda.backup.core :as bc]
'[dda.backup.postgresql :as pg])
(def restic-repo {:password-file (bc/env-or-file "RESTIC_PASSWORD_FILE")
:restic-repository (bc/env-or-file "RESTIC_REPOSITORY")})
(def db-config (merge restic-repo {:backup-path "pg-database"
:pg-host (bc/env-or-file "POSTGRES_SERVICE")
:pg-port (bc/env-or-file "POSTGRES_PORT")
:pg-db (bc/env-or-file "POSTGRES_DB")
:pg-user (bc/env-or-file "POSTGRES_USER")
:pg-password (bc/env-or-file "POSTGRES_PASSWORD")}))
(defn prepare!
[]
(pg/create-pg-pass! db-config))
(defn wait! []
(while true
(Thread/sleep 1000)))
(prepare!)
(wait!)

View file

@ -1,4 +0,0 @@
FROM c4k-forgejo-backup:latest
ADD resources /tmp/
RUN ENV_PASSWORD=env-password FILE_PASSWORD_FILE=/tmp/file_password /tmp/test.bb

View file

@ -1,3 +0,0 @@
{:deps {org.clojure/spec.alpha {:mvn/version "0.4.233"}
orchestra/orchestra {:mvn/version "2021.01.01-1"}
org.domaindrivenarchitecture/dda-backup {:local/root "/usr/local/lib/dda-backup"}}}

View file

@ -1,62 +0,0 @@
#!/usr/bin/env bb
(require '[babashka.tasks :as tasks]
'[dda.backup.core :as bc]
'[dda.backup.restic :as rc]
'[dda.backup.postgresql :as pg]
'[dda.backup.backup :as bak]
'[dda.backup.restore :as rs])
(def restic-repo {:password-file "restic-pwd"
:restic-repository "restic-repo"})
(def file-config (merge restic-repo {:backup-path "files"
:files ["test-backup"]
:restore-target-directory "test-restore"}))
(def db-config (merge restic-repo {:backup-path "db"
:pg-db "mydb"
:pg-user "user"
:pg-password "password"}))
(def dry-run {:dry-run true :debug true})
(defn prepare!
[]
(spit "/tmp/file_password" "file-password")
(println (bc/env-or-file "FILE_PASSWORD"))
(println (bc/env-or-file "ENV_PASSWORD"))
(spit "restic-pwd" "ThePassword")
(tasks/shell "mkdir" "-p" "test-backup")
(spit "test-backup/file" "I was here")
(tasks/shell "mkdir" "-p" "test-restore")
(pg/create-pg-pass! db-config))
(defn restic-repo-init!
[]
(rc/init! file-config)
(rc/init! (merge db-config dry-run)))
(defn restic-backup!
[]
(bak/backup-file! file-config)
(bak/backup-db! (merge db-config dry-run)))
(defn list-snapshots!
[]
(rc/list-snapshots! file-config)
(rc/list-snapshots! (merge db-config dry-run)))
(defn restic-restore!
[]
(rs/restore-file! file-config)
(pg/drop-create-db! (merge db-config dry-run))
(rs/restore-db! (merge db-config dry-run)))
(prepare!)
(restic-repo-init!)
(restic-backup!)
(list-snapshots!)
(restic-restore!)

View file

@ -6,7 +6,7 @@ from ddadevops import *
name = 'c4k-forgejo'
MODULE = 'federated'
PROJECT_ROOT_PATH = '../..'
version = "4.0.1-dev"
version = "3.4.1"
@init
def initialize(project):

View file

@ -2,7 +2,7 @@
"name": "c4k-forgejo",
"description": "Generate c4k yaml for a forgejo deployment.",
"author": "meissa GmbH",
"version": "4.0.1-SNAPSHOT",
"version": "3.4.1",
"homepage": "https://gitlab.com/domaindrivenarchitecture/c4k-forgejo#readme",
"repository": "https://www.npmjs.com/package/c4k-forgejo",
"license": "APACHE2",

View file

@ -1,17 +1,16 @@
(defproject org.domaindrivenarchitecture/c4k-forgejo "4.0.1-SNAPSHOT"
(defproject org.domaindrivenarchitecture/c4k-forgejo "3.4.1"
:description "forgejo c4k-installation package"
:url "https://domaindrivenarchitecture.org"
:license {:name "Apache License, Version 2.0"
:url "https://www.apache.org/licenses/LICENSE-2.0.html"}
:dependencies [[org.clojure/clojure "1.11.4" :scope "provided"]
[org.clojure/tools.reader "1.5.0"]
[org.domaindrivenarchitecture/c4k-common-clj "8.0.0"]
:dependencies [[org.clojure/clojure "1.11.3" :scope "provided"]
[org.clojure/tools.reader "1.4.2"]
[org.domaindrivenarchitecture/c4k-common-clj "6.4.1"]
[hickory "0.7.1" :exclusions [viebel/codox-klipse-theme]]]
:target-path "target/%s/"
:source-paths ["src/main/cljc"
"src/main/clj"]
:resource-paths ["src/main/resources"
"project.clj"]
:resource-paths ["src/main/resources"]
:repositories [["snapshots" :clojars]
["releases" :clojars]]
:deploy-repositories [["snapshots" {:sign-releases false :url "https://clojars.org/repo"}]
@ -24,9 +23,9 @@
:main dda.c4k-forgejo.uberjar
:uberjar-name "c4k-forgejo-standalone.jar"
:dependencies [[org.clojure/tools.cli "1.1.230"]
[ch.qos.logback/logback-classic "1.5.7"
[ch.qos.logback/logback-classic "1.5.6"
:exclusions [com.sun.mail/javax.mail]]
[org.slf4j/jcl-over-slf4j "2.0.16"]
[org.slf4j/jcl-over-slf4j "2.0.13"]
[com.github.clj-easy/graal-build-time "1.0.5"]]}}
:release-tasks [["test"]
["vcs" "assert-committed"]

View file

@ -4,7 +4,7 @@
"src/test/cljc"
"src/test/cljs"
"src/test/resources"]
:dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "8.0.0"]
:dependencies [[org.domaindrivenarchitecture/c4k-common-cljs "6.4.1"]
[hickory "0.7.1"]]
:builds {:frontend {:target :browser
:modules {:main {:init-fn dda.c4k-forgejo.browser/init}}

View file

@ -7,11 +7,10 @@
(set! *warn-on-reflection* true)
(defn -main [& cmd-args]
(uberjar/main-cm
(uberjar/main-common
"c4k-forgejo"
core/config?
core/auth?
core/config-defaults
core/config-objects
core/auth-objects
core/k8s-objects
cmd-args))

View file

@ -9,15 +9,7 @@
[dda.c4k-common.postgres :as postgres]
[dda.c4k-common.namespace :as ns]))
(def config-defaults {:namespace "forgejo"
:issuer "staging"
:deploy-federated "false"
:federation-enabled "false"
:db-name "forgejo"
:pv-storage-size-gb 5
:pvc-storage-class-name ""
:postgres-image "postgres:14"
:postgres-size :2gb})
(def config-defaults {:issuer "staging", :deploy-federated "false"})
(def rate-limit-defaults {:max-rate 10, :max-concurrent-requests 5})
(def config? (s/keys :req-un [::forgejo/fqdn
@ -27,7 +19,6 @@
::forgejo/service-noreply-address]
:opt-un [::forgejo/issuer
::forgejo/deploy-federated
::forgejo/federation-enabled
::forgejo/default-app-name
::forgejo/service-domain-whitelist
::forgejo/forgejo-image-version-overwrite
@ -42,39 +33,33 @@
(def vol? (s/keys :req-un [::forgejo/volume-total-storage-size]))
(defn config-objects [config] ; ToDo: ADR for generate functions - vector or no vector?
(defn k8s-objects [config auth] ; ToDo: ADR for generate functions - vector or no vector?
(let [storage-class (if (contains? config :postgres-data-volume-path) :manual :local-path)]
(map yaml/to-string
(filter #(not (nil? %))
(cm/concat-vec
(ns/generate config)
[(postgres/generate-configmap config)
(ns/generate (merge {:namespace "forgejo"} config))
[(postgres/generate-config {:postgres-size :2gb :db-name "forgejo"})
(postgres/generate-secret auth)
(when (contains? config :postgres-data-volume-path)
(postgres/generate-persistent-volume (select-keys config [:postgres-data-volume-path :pv-storage-size-gb])))
(postgres/generate-pvc (merge config {:pvc-storage-class-name storage-class}))
(postgres/generate-deployment config)
(postgres/generate-pvc {:pv-storage-size-gb 5
:pvc-storage-class-name storage-class})
(postgres/generate-deployment {:postgres-image "postgres:14"
:postgres-size :2gb})
(postgres/generate-service config)
(forgejo/generate-deployment config)
(forgejo/generate-service)
(forgejo/generate-service-ssh)
(forgejo/generate-data-volume config)
(forgejo/generate-appini-env config)]
(forgejo/generate-ratelimit-ingress-and-cert config) ; this function has a vector as output
(forgejo/generate-appini-env config)
(forgejo/generate-secrets auth)
(forgejo/generate-rate-limit-middleware rate-limit-defaults)] ; this does not have a vector as output
(forgejo/generate-rate-limit-ingress-and-cert (merge {:namespace "forgejo"} config)) ; this function has a vector as output
(when (contains? config :restic-repository)
[(backup/generate-config config)
(backup/generate-secret auth)
(backup/generate-cron)
(backup/generate-backup-restore-deployment config)])
(when (contains? config :mon-cfg)
(mon/generate-config)))))))
(defn auth-objects [config auth]
(map yaml/to-string
(filter #(not (nil? %))
(cm/concat-vec
(ns/generate config)
[(postgres/generate-secret config auth)
(forgejo/generate-secrets auth)]
(when (contains? config :restic-repository)
[(backup/generate-secret auth)])
(when (contains? config :mon-cfg)
(mon/generate-auth (:mon-cfg config) (:mon-auth auth)))))))
(when (:contains? config :mon-cfg)
(mon/generate (:mon-cfg config) (:mon-auth auth))))))))

View file

@ -33,7 +33,6 @@
(s/def ::default-app-name string?)
(s/def ::fqdn pred/fqdn-string?)
(s/def ::deploy-federated boolean-string?)
(s/def ::federation-enabled boolean-string?)
(s/def ::mailer-from pred/bash-env-string?)
(s/def ::mailer-host pred/bash-env-string?)
(s/def ::mailer-port pred/bash-env-string?)
@ -54,7 +53,6 @@
::service-noreply-address]
:opt-un [::issuer
::deploy-federated
::federation-enabled
::default-app-name
::service-domain-whitelist
::forgejo-image-version-overwrite]))
@ -69,11 +67,11 @@
(defn data-storage-by-volume-size
[total]
total)
;;TODO: remove unneccessaries, fedaration is merged
(def federated-image-name "domaindrivenarchitecture/c4k-forgejo-federated")
(def federated-image-version "latest")
(def non-federated-image-name "codeberg.org/forgejo/forgejo")
(def non-federated-image-version "8.0.3")
(def non-federated-image-version "1.19")
(defn-spec generate-image-str string?
[config config?]
@ -90,7 +88,7 @@
(defn generate-appini-env
[config]
(let [{:keys [default-app-name
federation-enabled
deploy-federated
fqdn
mailer-from
mailer-host
@ -99,7 +97,7 @@
service-noreply-address]
:or {default-app-name "forgejo instance"
service-domain-whitelist fqdn}} config
federation-enabled-bool (boolean-from-string federation-enabled)]
deploy-federated-bool (boolean-from-string deploy-federated)]
(->
(yaml/load-as-edn "forgejo/appini-env-configmap.yaml")
(cm/replace-all-matching-values-by-new-value "APPNAME" default-app-name)
@ -111,7 +109,7 @@
(cm/replace-all-matching-values-by-new-value "WHITELISTDOMAINS" service-domain-whitelist)
(cm/replace-all-matching-values-by-new-value "NOREPLY" service-noreply-address)
(cm/replace-all-matching-values-by-new-value "IS_FEDERATED"
(if federation-enabled-bool
(if deploy-federated-bool
"true"
"false")))))
@ -123,36 +121,56 @@
mailer-pw]} auth]
(->
(yaml/load-as-edn "forgejo/secrets.yaml")
(cm/replace-all-matching "DBUSER" (b64/encode postgres-db-user))
(cm/replace-all-matching "DBPW" (b64/encode postgres-db-password))
(cm/replace-all-matching "MAILERUSER" (b64/encode mailer-user))
(cm/replace-all-matching "MAILERPW" (b64/encode mailer-pw)))))
(cm/replace-all-matching-values-by-new-value "DBUSER" (b64/encode postgres-db-user))
(cm/replace-all-matching-values-by-new-value "DBPW" (b64/encode postgres-db-password))
(cm/replace-all-matching-values-by-new-value "MAILERUSER" (b64/encode mailer-user))
(cm/replace-all-matching-values-by-new-value "MAILERPW" (b64/encode mailer-pw)))))
(defn-spec generate-ratelimit-ingress-and-cert seq?
[config config?]
(let [{:keys [fqdn max-rate max-concurrent-requests namespace]} config]
(ing/generate-simple-ingress (merge
(defn generate-ingress-and-cert
[config]
(let [{:keys [fqdn]} config]
(ing/generate-ingress-and-cert
(merge
{:service-name "forgejo-service"
:service-port 3000
:fqdns [fqdn]
:average-rate max-rate
:burst-rate max-concurrent-requests
:namespace namespace}
:fqdns [fqdn]}
config))))
(defn-spec generate-rate-limit-ingress-and-cert pred/map-or-seq?
[config config?]
(->
(generate-ingress-and-cert config) ; returns a vector
(#(assoc-in % ; Attention: heavily relying on the output order of ing/generate-ingress-and-cert
[1 :metadata :annotations :traefik.ingress.kubernetes.io/router.middlewares]
(str
(-> (second %) :metadata :annotations :traefik.ingress.kubernetes.io/router.middlewares)
", default-ratelimit@kubernetescrd")))))
; using :average and :burst seems sensible, :period may be interesting for fine tuning later on
(defn-spec generate-rate-limit-middleware pred/map-or-seq?
[config rate-limit-config?]
(let [{:keys [max-rate max-concurrent-requests]} config]
(->
(yaml/load-as-edn "forgejo/middleware-ratelimit.yaml")
(cm/replace-key-value :average max-rate)
(cm/replace-key-value :burst max-concurrent-requests))))
(defn-spec generate-data-volume pred/map-or-seq?
[config vol?]
(let [{:keys [volume-total-storage-size]} config
data-storage-size (data-storage-by-volume-size volume-total-storage-size)]
(->
(yaml/load-as-edn "forgejo/datavolume.yaml")
(cm/replace-all-matching "DATASTORAGESIZE" (str (str data-storage-size) "Gi")))))
(cm/replace-all-matching-values-by-new-value "DATASTORAGESIZE" (str (str data-storage-size) "Gi")))))
(defn-spec generate-deployment pred/map-or-seq?
[config config?]
(let [{:keys [deploy-federated]} config
deploy-federated-bool (boolean-from-string deploy-federated)]
(->
(yaml/load-as-edn "forgejo/deployment.yaml")
(cm/replace-all-matching "IMAGE_NAME" (generate-image-str config))))
(cm/replace-all-matching-values-by-new-value "IMAGE_NAME" (generate-image-str config)))))
(defn generate-service
[]

View file

@ -79,7 +79,8 @@
(when (not (st/blank? app-name))
{:default-app-name app-name})
(when (not (st/blank? domain-whitelist))
{:service-domain-whitelist domain-whitelist}))))
{:service-domain-whitelist domain-whitelist})
)))
(defn validate-all! []
(br/validate! "fqdn" ::forgejo/fqdn)
@ -102,21 +103,16 @@
(defn init []
(br/append-hickory (generate-content-div))
(let [config-only false
auth-only false]
(-> js/document
(.getElementById "generate-button")
(.addEventListener "click"
#(do (validate-all!)
(-> (cm/generate-cm
(-> (cm/generate-common
(config-from-document)
(br/get-content-from-element "auth" :deserializer edn/read-string)
core/config-defaults
core/config-objects
core/auth-objects
config-only
auth-only)
(br/set-output!))))))
core/k8s-objects)
(br/set-output!)))))
(add-validate-listener "fqdn")
(add-validate-listener "deploy-federated")
(add-validate-listener "mailer-from")

View file

@ -21,7 +21,7 @@ spec:
- image: domaindrivenarchitecture/c4k-forgejo-backup
name: backup-app
imagePullPolicy: IfNotPresent
command: ["wait.bb"]
command: ["/entrypoint-start-and-wait.sh"]
env:
- name: POSTGRES_USER
valueFrom:

View file

@ -17,7 +17,7 @@ spec:
- name: backup-app
image: domaindrivenarchitecture/c4k-forgejo-backup
imagePullPolicy: IfNotPresent
command: ["backup.bb"]
command: ["/entrypoint.sh"]
env:
- name: POSTGRES_USER
valueFrom:

View file

@ -16,6 +16,7 @@ data:
FORGEJO__database__NAME: forgejo
FORGEJO__database__LOG_SQL: "false"
FORGEJO__database__SSL_MODE: disable
FORGEJO__database__CHARSET: utf8
#[DEFAULT]
APP_NAME: APPNAME
@ -36,12 +37,12 @@ data:
#[mailer]
FORGEJO__mailer__ENABLED: "true"
FORGEJO__mailer__FROM: FROM
FORGEJO__mailer__PROTOCOL: smtp+starttls
FORGEJO__mailer__MAILER_TYPE: smtp+startls
FORGEJO__mailer__SMTP_ADDR: MAILERHOST
FORGEJO__mailer__SMTP_PORT: MAILERPORT
#[oauth2]
FORGEJO__oauth2__ENABLED: "true"
FORGEJO__oauth2__ENABLE: "true"
#[openid]
FORGEJO__openid__ENABLE_OPENID: "true"
@ -75,7 +76,7 @@ data:
FORGEJO__service__REQUIRE_SIGNIN_VIEW: "false"
FORGEJO__service__REGISTER_EMAIL_CONFIRM: "true"
FORGEJO__service__ENABLE_NOTIFY_MAIL: "true"
FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST: WHITELISTDOMAINS
FORGEJO__service__EMAIL_DOMAIN_WHITELIST: WHITELISTDOMAINS
FORGEJO__service__ALLOW_ONLY_EXTERNAL_REGISTRATION: "false"
FORGEJO__service__ENABLE_BASIC_AUTHENTICATION: "true"
FORGEJO__service__ENABLE_CAPTCHA: "false"

View file

@ -0,0 +1,9 @@
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: ratelimit
namespace: forgejo
spec:
rateLimit: # Config options for rate limiting: https://doc.traefik.io/traefik/middlewares/http/ratelimit/
average: AVG
burst: BRS

View file

@ -14,7 +14,7 @@
(deftest should-generate-image-str
(testing "non-federated-image"
(is (= "codeberg.org/forgejo/forgejo:8.0.3"
(is (= "codeberg.org/forgejo/forgejo:1.19"
(cut/generate-image-str {:fqdn "test.de"
:mailer-from ""
:mailer-host "m.t.de"
@ -63,12 +63,12 @@
:FORGEJO__server__ROOT_URL-c2 "https://test.com",
:FORGEJO__server__SSH_DOMAIN-c1 "test.de",
:FORGEJO__server__SSH_DOMAIN-c2 "test.com",
:FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST-c1 "adb.de",
:FORGEJO__service__EMAIL_DOMAIN_ALLOWLIST-c2 "test.com,test.net",
:FORGEJO__service__EMAIL_DOMAIN_WHITELIST-c1 "adb.de",
:FORGEJO__service__EMAIL_DOMAIN_WHITELIST-c2 "test.com,test.net",
:FORGEJO__service__NO_REPLY_ADDRESS-c1 "",
:FORGEJO__service__NO_REPLY_ADDRESS-c2 "noreply@test.com"}
(th/map-diff (cut/generate-appini-env {:default-app-name ""
:federation-enabled "false"
:deploy-federated "false"
:fqdn "test.de"
:mailer-from ""
:mailer-host "m.t.de"
@ -76,7 +76,7 @@
:service-domain-whitelist "adb.de"
:service-noreply-address ""})
(cut/generate-appini-env {:default-app-name "test forgejo"
:federation-enabled "true"
:deploy-federated "true"
:fqdn "test.com"
:mailer-from "test@test.com"
:mailer-host "mail.test.com"
@ -97,7 +97,7 @@
:spec
{:containers
[{:name "forgejo",
:image "codeberg.org/forgejo/forgejo:8.0.3",
:image "codeberg.org/forgejo/forgejo:1.19",
:imagePullPolicy "IfNotPresent",
:envFrom [{:configMapRef {:name "forgejo-env"}} {:secretRef {:name "forgejo-secrets"}}],
:volumeMounts [{:name "forgejo-data-volume", :mountPath "/data"}],
@ -163,3 +163,26 @@
:storage-c2 "15Gi"}
(th/map-diff (cut/generate-data-volume {:volume-total-storage-size 1})
(cut/generate-data-volume {:volume-total-storage-size 15})))))
(deftest should-generate-middleware-ratelimit
(is (= {:apiVersion "traefik.containo.us/v1alpha1",
:kind "Middleware",
:metadata {:name "ratelimit", :namespace "forgejo"},
:spec {:rateLimit {:average 10, :burst 5}}}
(cut/generate-rate-limit-middleware {:max-rate 10, :max-concurrent-requests 5}))))
(deftest should-generate-middleware-ratelimit-ingress-and-cert
(is (= {:traefik.ingress.kubernetes.io/router.entrypoints "web, websecure",
:traefik.ingress.kubernetes.io/router.middlewares
"default-redirect-https@kubernetescrd, default-ratelimit@kubernetescrd",
:metallb.universe.tf/address-pool "public"}
(-> (second
(cut/generate-rate-limit-ingress-and-cert
{:fqdn "test.de"
:mailer-from ""
:mailer-host "m.t.de"
:mailer-port "123"
:service-noreply-address ""
:average 10
:burst 5}))
:metadata :annotations))))