Compare commits

..

No commits in common. "main" and "4.10.5" have entirely different histories.
main ... 4.10.5

44 changed files with 89 additions and 717 deletions

3
.gitignore vendored
View file

@ -109,6 +109,3 @@ venv.bak/
.clj-kondo/ .clj-kondo/
.lsp/ .lsp/
.calva/
.cpcache/
infrastructure/backup/image/resources/backup-repository-state.edn

View file

@ -4,14 +4,14 @@ stages:
- image - image
.py: &py .py: &py
image: "domaindrivenarchitecture/ddadevops-python:4.10.7" image: "domaindrivenarchitecture/ddadevops-python:4.10.5"
before_script: before_script:
- export RELEASE_ARTIFACT_TOKEN=$MEISSA_REPO_BUERO_RW - export RELEASE_ARTIFACT_TOKEN=$MEISSA_REPO_BUERO_RW
- python --version - python --version
- pip install -r requirements.txt - pip install -r requirements.txt
.img: &img .img: &img
image: "domaindrivenarchitecture/ddadevops-dind:4.10.7" image: "domaindrivenarchitecture/ddadevops-dind:4.10.5"
services: services:
- docker:dind - docker:dind
before_script: before_script:

View file

@ -1,7 +1,8 @@
# dda-devops-build # dda-devops-build
[![Slack](https://img.shields.io/badge/chat-clojurians-green.svg?style=flat)](https://clojurians.slack.com/messages/#dda-pallet/) | [<img src="https://domaindrivenarchitecture.org/img/delta-chat.svg" width=20 alt="DeltaChat"> chat over e-mail](mailto:buero@meissa-gmbh.de?subject=community-chat) | [<img src="https://meissa.de/images/parts/contact/mastodon36_hue9b2464f10b18e134322af482b9c915e_5501_filter_14705073121015236177.png" width=20 alt="M"> meissa@social.meissa-gmbh.de](https://social.meissa-gmbh.de/@meissa) | [Blog](https://domaindrivenarchitecture.org) | [Website](https://meissa.de) [![Slack](https://img.shields.io/badge/chat-clojurians-green.svg?style=flat)](https://clojurians.slack.com/messages/#dda-pallet/) | [<img src="https://meissa-gmbh.de/img/community/Mastodon_Logotype.svg" width=20 alt="team@social.meissa-gmbh.de"> team@social.meissa-gmbh.de](https://social.meissa-gmbh.de/@team) | [Website & Blog](https://domaindrivenarchitecture.org)
![release prod](https://github.com/DomainDrivenArchitecture/dda-devops-build/workflows/release%20prod/badge.svg)
dda-devops-build integrates all the tools we use to work with clouds & provide some nice functions around. dda-devops-build integrates all the tools we use to work with clouds & provide some nice functions around.
@ -83,10 +84,6 @@ Principles we follow are:
* Seperate build artefacts from version controlled code * Seperate build artefacts from version controlled code
* Domain Driven Design - in order to stay sustainable * Domain Driven Design - in order to stay sustainable
## Example Project
An example project which is using dda-devops-build can be found at: https://repo.prod.meissa.de/meissa/buildtest
## Installation ## Installation
Ensure that yout python3 version is at least Python 3.10 Ensure that yout python3 version is at least Python 3.10
@ -97,9 +94,17 @@ pip3 install -r requirements.txt
export PATH=$PATH:~/.local/bin export PATH=$PATH:~/.local/bin
``` ```
## Example Project ## Reference
An example project which is using dda-devops-build can be found at: https://repo.prod.meissa.de/meissa/buildtest * [DevopsBuild](./doc/DevopsBuild.md)
* [DevopsImageBuild](./doc/DevopsImageBuild.md)
* [DevopsTerraformBuild](./doc/DevopsTerraformBuild.md)
* [AwsProvider](doc/DevopsTerraformBuildWithAwsProvider.md)
* [DigitaloceanProvider](doc/DevopsTerraformBuildWithDigitaloceanProvider.md)
* [HetznerProvider](doc/DevopsTerraformBuildWithHetznerProvider.md)
* [ReleaseMixin](./doc/ReleaseMixin.md)
* [ProvsK3sBuild](doc/ProvsK3sBuild.md)
* [C4kBuild](doc/C4kBuild.md)
## Example Build ## Example Build
@ -185,19 +190,6 @@ pyb [patch|minor|major]
pip3 install --upgrade ddadevops pip3 install --upgrade ddadevops
``` ```
## Reference
* [DevopsBuild](./doc/DevopsBuild.md)
* [DevopsImageBuild](./doc/DevopsImageBuild.md)
* [DevopsTerraformBuild](./doc/DevopsTerraformBuild.md)
* [AwsProvider](doc/DevopsTerraformBuildWithAwsProvider.md)
* [DigitaloceanProvider](doc/DevopsTerraformBuildWithDigitaloceanProvider.md)
* [HetznerProvider](doc/DevopsTerraformBuildWithHetznerProvider.md)
* [ReleaseMixin](./doc/ReleaseMixin.md)
* [ProvsK3sBuild](doc/ProvsK3sBuild.md)
* [C4kBuild](doc/C4kBuild.md)
## Development & mirrors ## Development & mirrors
Development happens at: https://repo.prod.meissa.de/meissa/dda-devops-build Development happens at: https://repo.prod.meissa.de/meissa/dda-devops-build

View file

@ -33,7 +33,7 @@ default_task = "dev"
name = "ddadevops" name = "ddadevops"
MODULE = "not-used" MODULE = "not-used"
PROJECT_ROOT_PATH = "." PROJECT_ROOT_PATH = "."
version = "4.13.2-dev" version = "4.10.5"
summary = "tools to support builds combining gopass, terraform, dda-pallet, aws & hetzner-cloud" summary = "tools to support builds combining gopass, terraform, dda-pallet, aws & hetzner-cloud"
description = __doc__ description = __doc__
authors = [Author("meissa GmbH", "buero@meissa-gmbh.de")] authors = [Author("meissa GmbH", "buero@meissa-gmbh.de")]
@ -140,6 +140,12 @@ def lint(project):
shell=True, shell=True,
check=True, check=True,
) )
run(
"pylint -d W0511,R0903,C0301,W0614,C0114,C0115,C0116,similarities,W1203,W0702,W0702,"
+ "R0913,R0902,R0914,R1732,R1705,W0707,C0123,W0703,C0103 src/main/python/ddadevops/",
shell=True,
check=True,
)
@task @task

View file

@ -35,12 +35,7 @@ classDiagram
| name | name in context of build & ENV | - | | | name | name in context of build & ENV | - | |
## Example Usage ## Example Usage
### build.py
### Example project
A complete example project you can find on: https://repo.prod.meissa.de/meissa/buildtest
### Example of a build.py
```python ```python
from os import environ from os import environ

View file

@ -14,16 +14,13 @@ classDiagram
## Input ## Input
| name | description | default | | name | description | default |
| --------------------------------- | ----------------------------------------------------------------- | --------- | | ----------------------------- | ----------------------------------------------------------------- | --------- |
| k3s_provision_user | the user used to provision k3s | "root" | | k3s_provision_user | the user used to provision k3s | "root" |
| k3s_letsencrypt_email | email address used for letsencrypt | | | k3s_letsencrypt_email | email address used for letsencrypt | |
| k3s_letsencrypt_endpoint | letsencrypt endpoint. Valid values are staging, prod | "staging" | | k3s_letsencrypt_endpoint | letsencrypt endpoint. Valid values are staging, prod | "staging" |
| k3s_app_filename_to_provision | an k8s manifest to apply imediately after k3s setup was sucessful | | | k3s_app_filename_to_provision | an k8s manifest to apply imediately after k3s setup was sucessful | |
| k3s_enable_echo | provision the echo app on k3s. Valid values are true, false | "false" | | k3s_enable_echo | provision the echo app on k3s. Valid values are true, false | "false" |
| k3s_provs_template | use a individual template for provs config | None | | k3s_provs_template | use a individual template for provs config | None |
| k3s_enable_hetzner_csi | enable hetzner csi | False |
| k3s_hetzner_api_token | hetzner_api_token | None |
| k3s_hetzner_encryption_passphrase | encryption passphrase for volumes | None |
### Credentials Mapping defaults ### Credentials Mapping defaults

View file

@ -26,23 +26,22 @@ classDiagram
## Input ## Input
| name | description | default | | name | description | default |
| ----------------------------- |-----------------------------------------------------------------------------------------------------------------------| --------------- | | ----------------------------- | --------------------------------------------------------------------------------------------------------------------- | --------------- |
| release_type | one of MAJOR, MINOR, PATCH, NONE | "NONE" | | release_type | one of MAJOR, MINOR, PATCH, NONE | "NONE" |
| release_main_branch | the name of your trunk | "main" | | release_main_branch | the name of your trank | "main" |
| release_primary_build_file | path to the build file having the leading version info (read & write). Valid extensions are .clj, .json, .gradle, .py | "./project.clj" | | release_primary_build_file | path to the build file having the leading version info (read & write). Valid extensions are .clj, .json, .gradle, .py | "./project.clj" |
| release_secondary_build_files | list of secondary build files, version is written in. | [] | | release_secondary_build_files | list of secondary build files, version is written in. | [] |
| release_artifact_server_url | Optional: The base url of your forgejo/gitea instance to publish a release tode | | | release_artifact_server_url | Optional: The base url of your forgejo/gitea instance to publish a release tode | |
| release_organisation | Optional: The repository organisation name | | | release_organisation | Optional: The repository organisation name | |
| release_repository_name | Optional: The repository name name | | | release_repository_name | Optional: The repository name name | |
| release_artifacts | Optional: The list of artifacts to publish to the release generated name | [] | | release_artifacts | Optional: The list of artifacts to publish to the release generated name | [] |
| release_tag_prefix | Optional: Prefix of tag | "" |
## Example Usage just for creating releases ## Example Usage just for creating releases
### build.py ### build.py
```python ```python
from os import environ rom os import environ
from pybuilder.core import task, init from pybuilder.core import task, init
from ddadevops import * from ddadevops import *

View file

@ -1,56 +0,0 @@
from os import environ
from datetime import datetime
from pybuilder.core import task, init
from ddadevops import *
import logging
name = 'dda-backup'
MODULE = 'NOT_SET'
PROJECT_ROOT_PATH = '../..'
version = "4.12.2-dev"
@init
def initialize(project):
image_tag = version
if "dev" in image_tag:
image_tag += datetime.now().strftime("%Y-%m-%d-%H-%M-%S")
input = {
"name": name,
"module": MODULE,
"stage": "notused",
"project_root_path": PROJECT_ROOT_PATH,
"build_types": ["IMAGE"],
"mixin_types": [],
"image_naming": "NAME_ONLY",
"image_tag": f"{image_tag}",
}
project.build_depends_on("ddadevops>=4.7.0")
build = DevopsImageBuild(project, input)
build.initialize_build_dir()
@task
def image(project):
build = get_devops_build(project)
build.image()
@task
def test(project):
build = get_devops_build(project)
build.test()
@task
def drun(project):
build = get_devops_build(project)
build.drun()
@task
def publish(project):
build = get_devops_build(project)
build.dockerhub_login()
build.dockerhub_publish()

View file

@ -1,79 +0,0 @@
## Init Statemachine
### Inputs
1. `restic-password: ""`
2. `restic-password-to-rotate: ""`
### Manual init the restic repository for the first time
1. apply backup-and-restore pod:
`kubectl scale deployment backup-restore --replicas=1`
2. exec into pod and execute restore pod (press tab to get your exact pod name)
`kubectl exec -it backup-restore-... -- /usr/local/bin/init.sh`
3. remove backup-and-restore pod:
`kubectl scale deployment backup-restore --replicas=0`
### Password Rotation
1. apply backup-and-restore pod:
`kubectl scale deployment backup-restore --replicas=1`
2. add new password to restic repository
`restic key add ....`
=> Trigger ::
field (1) credential current
filed (2) credential new
3. replace field (1) with (2) & clear (2)
4. remove old key - ???
`restic remove ....`
```mermaid
stateDiagram-v2
[*] --> init
init --> backup_ready: trigger, restic-password !empty
backup_ready --> new_password_added: restic-password !empty && restic-password-to-rotate !empty
new_password_added --> backup_ready: restic-password !empty && restic-password-to-rotate empty
```
### First Steps
1. Cloud Testserver hochfahren
2. Dort backup-restore deployment (leeres Secret mgl.?), neues Secret "rotation-credential-secret" als Daten
3. mounten von angelegtem Secret in Pod backup-restore
4. ba*bash*ka Skript in pod starten -> liest Secret ?leer
5. Micha cons.
```mermaid
sequenceDiagram
participant k8s
participant e as entrypoint.sh
participant rm as restic-management.clj
k8s ->> e: cronjob calls
e ->> rm: start-file
rm ->> rm: rotate
activate rm
rm ->> rm: read-backup-repository-state (state)
rm ->> rm: read-secret (backup-secret/restic-password, rotation-credential-secret/rotation-credential)
rm ->> rm: switch
activate rm
rm ->> rm: if init && restic-password != null
activate rm
rm ->> rm: init.sh
rm ->> rm: state init -> backup-ready
deactivate rm
rm ->> rm: if backup-ready && rotation-credential != null
activate rm
rm ->> rm: add-new-password-to-restic-repository.sh
rm ->> rm: state backup-ready -> new-password-added
deactivate rm
rm ->> rm: if new-password-added && rotation-credential == null
activate rm
rm ->> rm: remove-old-password-from-restic-repository.sh
rm ->> rm: state new-password-added -> backup-ready
deactivate rm
deactivate rm
rm ->> rm: store-repository-state (state)
deactivate rm
```

View file

@ -1,5 +0,0 @@
FROM ubuntu:jammy
# install it
ADD resources /tmp/
RUN /tmp/install.sh

View file

@ -1,70 +0,0 @@
backup_file_path='files'
function init-file-repo() {
if [ -z ${CERTIFICATE_FILE} ];
then
restic -r ${RESTIC_REPOSITORY}/${backup_file_path} -v init
else
restic -r ${RESTIC_REPOSITORY}/${backup_file_path} -v init --cacert ${CERTIFICATE_FILE}
fi
}
# First arg is the directory, second is optional for the path to a certificate file
function backup-directory() {
local directory="$1"; shift
if [ -z ${CERTIFICATE_FILE} ];
then
restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} unlock --cleanup-cache
cd ${directory} && restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} backup .
restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} forget --group-by '' --keep-last 1 --keep-daily ${RESTIC_DAYS_TO_KEEP} --keep-monthly ${RESTIC_MONTHS_TO_KEEP} --prune
else
restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} unlock --cleanup-cache --cacert ${CERTIFICATE_FILE}
cd ${directory} && restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} backup . --cacert ${CERTIFICATE_FILE}
restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} forget --group-by '' --keep-last 1 --keep-daily ${RESTIC_DAYS_TO_KEEP} --keep-monthly ${RESTIC_MONTHS_TO_KEEP} --prune --cacert ${CERTIFICATE_FILE}
fi
}
# First arg is the directory, the remaining args are the sub-directories (relative to the first directory) to backup.
function backup-fs-from-directory() {
local directory="$1"; shift
if [ -z ${CERTIFICATE_FILE} ];
then
restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} unlock --cleanup-cache
cd ${directory} && restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} backup $@
restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} forget --group-by '' --keep-last 1 --keep-daily ${RESTIC_DAYS_TO_KEEP} --keep-monthly ${RESTIC_MONTHS_TO_KEEP} --prune
else
restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} unlock --cleanup-cache --cacert ${CERTIFICATE_FILE}
cd ${directory} && restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} backup $@ --cacert ${CERTIFICATE_FILE}
restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} forget --group-by '' --keep-last 1 --keep-daily ${RESTIC_DAYS_TO_KEEP} --keep-monthly ${RESTIC_MONTHS_TO_KEEP} --prune --cacert ${CERTIFICATE_FILE}
fi
}
# Das tut so nicht!
function restore-directory() {
local directory="$1"; shift
local snapshot_id="${1:-latest}"; shift
if [ -z ${CERTIFICATE_FILE} ];
then
restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} unlock --cleanup-cache
rm -rf ${directory}*
restic -v -r $RESTIC_REPOSITORY/${backup_file_path} restore ${snapshot_id} --target ${directory}
else
restic -v -r ${RESTIC_REPOSITORY}/${backup_file_path} unlock --cleanup-cache --cacert ${CERTIFICATE_FILE}
rm -rf ${directory}*
restic -v -r $RESTIC_REPOSITORY/${backup_file_path} restore ${snapshot_id} --target ${directory} --cacert ${CERTIFICATE_FILE}
fi
}
function list-snapshot-files() {
if [ -z ${CERTIFICATE_FILE} ];
then
restic -r ${RESTIC_REPOSITORY}/${backup_file_path} snapshots
else
restic -r ${RESTIC_REPOSITORY}/${backup_file_path} snapshots --cacert ${CERTIFICATE_FILE}
fi
}

View file

@ -1,21 +0,0 @@
# usage: file_env VAR [DEFAULT]
# ie: file_env 'XYZ_DB_PASSWORD' 'example'
# (will allow for "$XYZ_DB_PASSWORD_FILE" to fill in the value of
# "$XYZ_DB_PASSWORD" from a file, especially for Docker's secrets feature)
function file_env() {
local var="$1"
local fileVar="${var}_FILE"
local def="${2:-}"
if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
exit 1
fi
local val="$def"
if [ "${!var:-}" ]; then
val="${!var}"
elif [ "${!fileVar:-}" ]; then
val="$(< "${!fileVar}")"
fi
export "$var"="$val"
unset "$fileVar"
}

View file

@ -1,36 +0,0 @@
#!/bin/bash
set -exo pipefail
function babashka_install() {
babashka_version="1.3.189"
curl -SsLo /tmp/babashka-${babashka_version}-linux-amd64.tar.gz https://github.com/babashka/babashka/releases/download/v${babashka_version}/babashka-${babashka_version}-linux-amd64.tar.gz
curl -SsLo /tmp/checksum https://github.com/babashka/babashka/releases/download/v${babashka_version}/babashka-${babashka_version}-linux-amd64.tar.gz.sha256
echo " /tmp/babashka-$babashka_version-linux-amd64.tar.gz"|tee -a /tmp/checksum
sha256sum -c --status /tmp/checksum
tar -C /tmp -xzf /tmp/babashka-${babashka_version}-linux-amd64.tar.gz
install -m 0700 -o root -g root /tmp/bb /usr/local/bin/
}
function main() {
{
upgradeSystem
apt-get install -qqy ca-certificates curl gnupg postgresql-client-14 restic
curl -Ss --fail https://www.postgresql.org/media/keys/ACCC4CF8.asc | gpg --dearmor | tee /etc/apt/trusted.gpg.d/postgresql-common_pgdg_archive_keyring.gpg
sh -c 'echo "deb [signed-by=/etc/apt/trusted.gpg.d/postgresql-common_pgdg_archive_keyring.gpg] https://apt.postgresql.org/pub/repos/apt jammy-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
upgradeSystem
babashka_install
} > /dev/null
update-ca-certificates
install -m 0400 /tmp/functions.sh /usr/local/lib/
install -m 0400 /tmp/pg-functions.sh /usr/local/lib/
install -m 0400 /tmp/file-functions.sh /usr/local/lib/
install -m 0740 /tmp/restic_management.clj /usr/local/bin/
cleanupDocker
}
source /tmp/install_functions_debian.sh
DEBIAN_FRONTEND=noninteractive DEBCONF_NOWARNINGS=yes main

View file

@ -1,149 +0,0 @@
backup_pg_role_path='pg-role'
backup_pg_database_path='pg-database'
function init-command() {
restic -r ${RESTIC_REPOSITORY}/${backup_pg_role_path} -v init $@
}
function init-role-repo() {
if [ -z ${CERTIFICATE_FILE} ];
then
init-command
else
init-command --cacert ${CERTIFICATE_FILE}
fi
}
function init-database-command() {
restic -r ${RESTIC_REPOSITORY}/${backup_pg_database_path} -v init $@
}
function init-database-repo() {
if [ -z ${CERTIFICATE_FILE} ];
then
init-database-command
else
init-database-command --cacert ${CERTIFICATE_FILE}
fi
}
function drop-create-db() {
psql -d template1 -h ${POSTGRES_SERVICE} -p ${POSTGRES_PORT} -U ${POSTGRES_USER} \
--no-password -c "DROP DATABASE \"${POSTGRES_DB}\";"
psql -d template1 -h ${POSTGRES_SERVICE} -p ${POSTGRES_PORT} -U ${POSTGRES_USER} \
--no-password -c "CREATE DATABASE \"${POSTGRES_DB}\";"
}
function create-pg-pass() {
local pg_host=${POSTGRES_HOST:-localhost}
echo "${pg_host}:${POSTGRES_DB}:${POSTGRES_USER}:${POSTGRES_PASSWORD}" > /root/.pgpass
echo "${POSTGRES_HOST}:template1:${POSTGRES_USER}:${POSTGRES_PASSWORD}" >> /root/.pgpass
chmod 0600 /root/.pgpass
}
function roles-unlock-command() {
restic -v -r ${RESTIC_REPOSITORY}/${backup_pg_role_path} unlock --cleanup-cache $@
}
function roles-forget-command() {
restic -v -r ${RESTIC_REPOSITORY}/${backup_pg_role_path} forget --group-by '' --keep-last 1 --keep-daily ${RESTIC_DAYS_TO_KEEP} --keep-monthly ${RESTIC_MONTHS_TO_KEEP} --prune $@
}
function backup-roles() {
local role_prefix="$1"; shift
if [ -z ${CERTIFICATE_FILE} ];
then
roles-unlock-command
pg_dumpall -h ${POSTGRES_SERVICE} -p ${POSTGRES_PORT} -U${POSTGRES_USER} --no-password --roles-only | \
grep ${role_prefix} | restic -r ${RESTIC_REPOSITORY}/${backup_pg_role_path} backup --stdin
roles-forget-command
else
roles-unlock-command --cacert ${CERTIFICATE_FILE}
pg_dumpall -h ${POSTGRES_SERVICE} -p ${POSTGRES_PORT} -U${POSTGRES_USER} --no-password --roles-only | \
grep ${role_prefix} | restic -r ${RESTIC_REPOSITORY}/${backup_pg_role_path} backup --stdin --cacert ${CERTIFICATE_FILE}
roles-forget-command --cacert ${CERTIFICATE_FILE}
fi
}
function db-unlock-command() {
restic -v -r ${RESTIC_REPOSITORY}/${backup_pg_database_path} unlock --cleanup-cache $@
}
function db-forget-command() {
restic -v -r ${RESTIC_REPOSITORY}/${backup_pg_database_path} forget --group-by '' --keep-last 1 --keep-daily ${RESTIC_DAYS_TO_KEEP} --keep-monthly ${RESTIC_MONTHS_TO_KEEP} --prune $@
}
function backup-db-dump() {
if [ -z ${CERTIFICATE_FILE} ];
then
db-unlock-command
pg_dump -d ${POSTGRES_DB} -h ${POSTGRES_SERVICE} -p ${POSTGRES_PORT} \
-U ${POSTGRES_USER} --no-password --serializable-deferrable | \
restic -r ${RESTIC_REPOSITORY}/${backup_pg_database_path} backup --stdin
db-forget-command
else
db-unlock-command --cacert ${CERTIFICATE_FILE}
pg_dump -d ${POSTGRES_DB} -h ${POSTGRES_SERVICE} -p ${POSTGRES_PORT} \
-U ${POSTGRES_USER} --no-password --serializable-deferrable | \
restic -r ${RESTIC_REPOSITORY}/${backup_pg_database_path} backup --stdin --cacert ${CERTIFICATE_FILE}
db-forget-command --cacert ${CERTIFICATE_FILE}
fi
}
function restore-roles() {
local snapshot_id="${1:-latest}"; shift
if [ -z ${CERTIFICATE_FILE} ];
then
roles-unlock-command
restic -r ${RESTIC_REPOSITORY}/${backup_pg_role_path} dump ${snapshot_id} stdin | \
psql -d template1 -h ${POSTGRES_SERVICE} -p ${POSTGRES_PORT} -U ${POSTGRES_USER} \
--no-password
else
roles-unlock-command --cacert ${CERTIFICATE_FILE}
restic -r ${RESTIC_REPOSITORY}/${backup_pg_role_path} dump ${snapshot_id} stdin --cacert ${CERTIFICATE_FILE} | \
psql -d template1 -h ${POSTGRES_SERVICE} -p ${POSTGRES_PORT} -U ${POSTGRES_USER} \
--no-password
fi
}
function restore-db() {
local snapshot_id="${1:-latest}"; shift
if [ -z ${CERTIFICATE_FILE} ];
then
db-unlock-command
restic -r ${RESTIC_REPOSITORY}/${backup_pg_database_path} dump ${snapshot_id} stdin | \
psql -d ${POSTGRES_DB} -h ${POSTGRES_SERVICE} -p ${POSTGRES_PORT} -U ${POSTGRES_USER} \
--no-password
else
db-unlock-command --cacert ${CERTIFICATE_FILE}
restic -r ${RESTIC_REPOSITORY}/${backup_pg_database_path} dump ${snapshot_id} stdin --cacert ${CERTIFICATE_FILE} | \
psql -d ${POSTGRES_DB} -h ${POSTGRES_SERVICE} -p ${POSTGRES_PORT} -U ${POSTGRES_USER} \
--no-password
fi
}
function list-snapshot-roles() {
if [ -z ${CERTIFICATE_FILE} ];
then
restic -r ${RESTIC_REPOSITORY}/${backup_pg_role_path} snapshots
else
restic -r ${RESTIC_REPOSITORY}/${backup_pg_database_path} snapshots --cacert ${CERTIFICATE_FILE}
fi
}
function list-snapshot-db() {
if [ -z ${CERTIFICATE_FILE} ];
then
restic -r ${RESTIC_REPOSITORY}/${backup_pg_database_path} snapshots
else
restic -r ${RESTIC_REPOSITORY}/${backup_pg_database_path} snapshots --cacert ${CERTIFICATE_FILE}
fi
}

View file

@ -1,51 +0,0 @@
#! /usr/bin/env bb
(ns restic-management
(:require
[clojure.spec.alpha :as s]
[clojure.java.io :as io]
[clojure.edn :as edn]))
(s/def ::state string?)
(s/def ::backup-repository-state
(s/keys :req-un [::state]))
(def state {:state ""})
(defn store-backup-repository-state [s]
(spit "backup-repository-state.edn" s))
(defn read-backup-repository-state []
(try
(with-open [r (io/reader "backup-repository-state.edn")]
(edn/read (java.io.PushbackReader. r)))
(catch java.io.IOException e
(printf "Couldn't open '%s': %s\n" "backup-repository-state.edn" (.getMessage e)))
(catch RuntimeException e
(printf "Error parsing edn file '%s': %s\n" "backup-repository-state.edn" (.getMessage e)))))
(defn read-secret [s]
(slurp (str "/var/run/secrets/" s)))
;"/var/run/secrets/rotation-credential-secret/rotation-credential"))
;(println (read-backup-repository-state))
;(println (:state (read-backup-repository-state)))
;(println (s/valid? ::backup-repository-state (read-backup-repository-state)))
(println (read-secret "rotation-credential-secret/rotation-credential"))
(println (read-secret "backup-secrets/restic-password"))
(s/def ::new-password string?)
(s/def ::old-password string?)
(s/def ::password-state
(s/keys :req-un [::new-password ::old-password]))
(defn rotate []
(let [state {:new-password (read-secret "rotation-credential-secret/rotation-credential")
:old-password (read-secret "backup-secrets/restic-password")}]
(store-backup-repository-state (prn-str state))))
(rotate)

View file

@ -1,7 +0,0 @@
FROM dda-backup:latest
# install it
RUN apt update && apt install -qqy openjdk-17-jre-headless
ADD resources /tmp/
RUN rm -rf /root/.m2
RUN /tmp/install-test.bb

View file

@ -1,4 +0,0 @@
{:deps {org.clojure/spec.alpha {:mvn/version "0.4.233"}
orchestra/orchestra {:mvn/version "2021.01.01-1"}
org.domaindrivenarchitecture/dda-backup {:mvn/version "0.1.1-SNAPSHOT"}}}

View file

@ -1,32 +0,0 @@
#!/usr/bin/env bb
(require '[babashka.tasks :as tasks])
(defn curl-and-check!
[filename artifact-url sha256-url]
(let [filepath (str "/tmp/" filename)]
(tasks/shell "curl" "-SsLo" filepath artifact-url)
(tasks/shell "curl" "-SsLo" "/tmp/checksum" sha256-url)
(tasks/shell "bash" "-c" (str "echo \" " filepath "\"|tee -a /tmp/checksum"))
;(tasks/shell "sha256sum" "-c" "--status" "/tmp/checksum")
))
(defn tar-install!
[filename binname]
(let [filepath (str "/tmp/" filename)]
(tasks/shell "tar" "-C" "/tmp" "-xzf" filepath)
(tasks/shell "install" "-m" "0700" "-o" "root" "-g" "root" (str "/tmp/" binname) "/usr/local/bin/")))
(defn install!
[filename]
(tasks/shell "install" "-m" "0700" "-o" "root" "-g" "root" (str "/tmp/" filename) "/usr/local/bin/"))
(tasks/shell "bb" "/tmp/test.bb")
(curl-and-check!
"provs-syspec.jar"
"https://repo.prod.meissa.de/attachments/0a1da41e-aa5b-4a3e-a3b1-215cf2d5b021"
"https://repo.prod.meissa.de/attachments/f227cf65-cb0f-46a7-a6cd-28f46917412a")
(install! "provs-syspec.jar")
(tasks/shell "apt" "update")
(tasks/shell "apt" "install" "-qqy" "openjdk-17-jre-headless")
(tasks/shell "java" "-jar" "/usr/local/bin/provs-syspec.jar" "local" "-c" "/tmp/spec.yml" )

View file

@ -1,7 +0,0 @@
package:
- name: "restic"
command:
- command: "bb -h"
- command: "/tmp/test.bb"

View file

@ -1,27 +0,0 @@
#!/usr/bin/env bb
(require '[babashka.tasks :as tasks]
'[dda.backup.management :as mgm])
(defn restic-repo-init!
[]
(spit "restic-pwd" "ThePassword")
(mgm/init! {:password-file "restic-pwd"
:restic-repository "restic-repo"}))
(defn restic-backup!
[]
(tasks/shell "mkdir" "test-backup")
(spit "test-backup/file" "I was here")
(tasks/shell "restic" "backup" "--password-file" "restic-pwd" "--repo" "restic-repo" "test-backup"))
(defn restic-restore!
[]
(tasks/shell "mkdir" "test-restore")
(tasks/shell "restic" "restore" "--password-file" "restic-pwd" "--repo" "restic-repo" "--target" "test-restore" "latest")
)
(restic-repo-init!)
(restic-backup!)
(restic-restore!)

View file

@ -6,7 +6,7 @@ from ddadevops import *
name = "ddadevops" name = "ddadevops"
MODULE = "clj-cljs" MODULE = "clj-cljs"
PROJECT_ROOT_PATH = "../.." PROJECT_ROOT_PATH = "../.."
version = "4.13.2-dev" version = "4.10.5"
@init @init
def initialize(project): def initialize(project):

View file

@ -6,7 +6,7 @@ function main() {
upgradeSystem upgradeSystem
mkdir -p /usr/share/man/man1 mkdir -p /usr/share/man/man1
apt-get -qqy install curl openjdk-17-jre-headless leiningen apt-get -qqy install openjdk-17-jre-headless leiningen curl
# shadow-cljs # shadow-cljs
npm install -g npm npm install -g npm
@ -15,30 +15,33 @@ function main() {
# download kubeconform & graalvm # download kubeconform & graalvm
kubeconform_version="0.6.4" kubeconform_version="0.6.4"
curl -SsLo /tmp/kubeconform-linux-amd64.tar.gz https://github.com/yannh/kubeconform/releases/download/v${kubeconform_version}/kubeconform-linux-amd64.tar.gz curl -SsLo /tmp/kubeconform.tar.gz https://github.com/yannh/kubeconform/releases/download/v${kubeconform_version}/kubeconform-linux-amd64.tar.gz
curl -SsLo /tmp/CHECKSUMS https://github.com/yannh/kubeconform/releases/download/v${kubeconform_version}/CHECKSUMS curl -SsLo /tmp/CHECKSUMS https://github.com/yannh/kubeconform/releases/download/v${kubeconform_version}/CHECKSUMS
# checksum kubeconform # checksum kubeconform
checksum checksum
# install kubeconform # install kubeconform
tar -C /usr/local/bin -xf /tmp/kubeconform-linux-amd64.tar.gz --exclude=LICENSE tar -C /usr/local/bin -xf /tmp/kubeconform.tar.gz --exclude=LICENSE
#install pyb #install pyb
apt-get -qqy install python3 python3-pip git apt-get -qqy install python3 python3-pip git
pip3 install pybuilder 'ddadevops>=4.7.0' deprecation dda-python-terraform boto3 pyyaml inflection --break-system-packages pip3 install pybuilder 'ddadevops>=4.7.0' deprecation dda-python-terraform boto3 pyyaml inflection --break-system-packages
#check
lein --help
cleanupDocker cleanupDocker
} > /dev/null } > /dev/null
} }
function checksum() { function checksum() {
awk '{print $1 " /tmp/" $2}' /tmp/CHECKSUMS|sed -n '2p' > /tmp/kubeconform-checksum checksum_var=$(awk '{print $1}' /tmp/CHECKSUMS|sed -n '2p')
cat /tmp/kubeconform-checksum sha256sum_var=$(sha256sum /tmp/kubeconform.tar.gz|awk '{print $1}')
sha256sum -c --status /tmp/kubeconform-checksum
if [ $checksum_var == $sha256sum_var ]; then
echo "Kubeconform checksum verification succesful"
else
echo "Failure in kubeconform checksum verification"
exit 1
fi
} }
source /tmp/install_functions_debian.sh source /tmp/install_functions_debian.sh

View file

View file

@ -6,7 +6,7 @@ from ddadevops import *
name = "ddadevops" name = "ddadevops"
MODULE = "clj" MODULE = "clj"
PROJECT_ROOT_PATH = "../.." PROJECT_ROOT_PATH = "../.."
version = "4.13.2-dev" version = "4.10.5"
@init @init
def initialize(project): def initialize(project):

View file

@ -3,4 +3,4 @@ FROM debian:stable-slim
ADD resources /tmp ADD resources /tmp
RUN /tmp/install.sh RUN /tmp/install.sh
ENV LANG=en_US.UTF-8 \ ENV LANG=en_US.UTF-8 \
JAVA_HOME=/usr/lib/jvm/graalvm JAVA_HOME=/usr/lib64/graalvm/graalvm-community-java17

View file

@ -10,9 +10,9 @@ function main() {
# download kubeconform & graalvm # download kubeconform & graalvm
kubeconform_version="0.6.4" kubeconform_version="0.6.4"
graalvm_jdk_version="21.0.2" graalvm_jdk_version="17.0.9"
curl -SsLo /tmp/kubeconform-linux-amd64.tar.gz https://github.com/yannh/kubeconform/releases/download/v${kubeconform_version}/kubeconform-linux-amd64.tar.gz curl -SsLo /tmp/kubeconform.tar.gz https://github.com/yannh/kubeconform/releases/download/v${kubeconform_version}/kubeconform-linux-amd64.tar.gz
curl -SsLo /tmp/CHECKSUMS https://github.com/yannh/kubeconform/releases/download/v${kubeconform_version}/CHECKSUMS curl -SsLo /tmp/CHECKSUMS https://github.com/yannh/kubeconform/releases/download/v${kubeconform_version}/CHECKSUMS
curl -SsLo /tmp/graalvm-community-jdk.tar.gz https://github.com/graalvm/graalvm-ce-builds/releases/download/jdk-${graalvm_jdk_version}/graalvm-community-jdk-${graalvm_jdk_version}_linux-x64_bin.tar.gz curl -SsLo /tmp/graalvm-community-jdk.tar.gz https://github.com/graalvm/graalvm-ce-builds/releases/download/jdk-${graalvm_jdk_version}/graalvm-community-jdk-${graalvm_jdk_version}_linux-x64_bin.tar.gz
curl -SsLo /tmp/graalvm-checksum https://github.com/graalvm/graalvm-ce-builds/releases/download/jdk-${graalvm_jdk_version}/graalvm-community-jdk-${graalvm_jdk_version}_linux-x64_bin.tar.gz.sha256 curl -SsLo /tmp/graalvm-checksum https://github.com/graalvm/graalvm-ce-builds/releases/download/jdk-${graalvm_jdk_version}/graalvm-community-jdk-${graalvm_jdk_version}_linux-x64_bin.tar.gz.sha256
@ -21,7 +21,7 @@ function main() {
checksum checksum
# install kubeconform # install kubeconform
tar -C /usr/local/bin -xf /tmp/kubeconform-linux-amd64.tar.gz --exclude=LICENSE tar -C /usr/local/bin -xf /tmp/kubeconform.tar.gz --exclude=LICENSE
# install graalvm # install graalvm
tar -C /usr/lib/jvm/ -xf /tmp/graalvm-community-jdk.tar.gz tar -C /usr/lib/jvm/ -xf /tmp/graalvm-community-jdk.tar.gz
@ -29,28 +29,29 @@ function main() {
ln -s /usr/lib/jvm/$dirname_graalvm /usr/lib/jvm/graalvm ln -s /usr/lib/jvm/$dirname_graalvm /usr/lib/jvm/graalvm
ln -s /usr/lib/jvm/graalvm/bin/gu /usr/local/bin ln -s /usr/lib/jvm/graalvm/bin/gu /usr/local/bin
update-alternatives --install /usr/bin/java java /usr/lib/jvm/graalvm/bin/java 2 update-alternatives --install /usr/bin/java java /usr/lib/jvm/graalvm/bin/java 2
gu install native-image
ln -s /usr/lib/jvm/graalvm/bin/native-image /usr/local/bin ln -s /usr/lib/jvm/graalvm/bin/native-image /usr/local/bin
#install pyb #install pyb
apt-get -qqy install python3 python3-pip apt-get -qqy install python3 python3-pip
pip3 install pybuilder 'ddadevops>=4.7.0' deprecation dda-python-terraform boto3 pyyaml inflection --break-system-packages pip3 install pybuilder 'ddadevops>=4.7.0' deprecation dda-python-terraform boto3 pyyaml inflection --break-system-packages
#check
native-image --version
lein -v
cleanupDocker cleanupDocker
} > /dev/null } > /dev/null
} }
function checksum() { function checksum() {
#kubeconform checksum_kubeconform=$(awk '{print $1}' /tmp/CHECKSUMS|sed -n '2p')
awk '{print $1 " /tmp/" $2}' /tmp/CHECKSUMS|sed -n '2p' > /tmp/kubeconform-checksum sha256sum_kubeconform=$(sha256sum /tmp/kubeconform.tar.gz|awk '{print $1}')
sha256sum -c --status /tmp/kubeconform-checksum checksum_graalvm_jdk=$(awk '{print $1}' /tmp/graalvm-checksum)
sha256sum_graalvm_jdk=$(sha256sum /tmp/graalvm-community-jdk.tar.gz|awk '{print $1}')
#graalvm if [ $checksum_kubeconform == $sha256sum_kubeconform -a $checksum_graalvm_jdk == $sha256sum_graalvm_jdk ]; then
echo " /tmp/graalvm-community-jdk.tar.gz"|tee -a /tmp/graalvm-checksum echo "Kubeconform & graalvm_jdk checksum verification succesful"
sha256sum -c --status /tmp/graalvm-checksum else
echo "Failure in kubeconform|graalvm_jdk checksum verification"
exit 1
fi
} }
source /tmp/install_functions_debian.sh source /tmp/install_functions_debian.sh

View file

View file

@ -6,7 +6,7 @@ from ddadevops import *
name = "ddadevops" name = "ddadevops"
MODULE = "ddadevops" MODULE = "ddadevops"
PROJECT_ROOT_PATH = "../.." PROJECT_ROOT_PATH = "../.."
version = "4.13.2-dev" version = "4.10.5"
@init @init

View file

View file

@ -6,7 +6,7 @@ from ddadevops import *
name = "ddadevops" name = "ddadevops"
MODULE = "dind" MODULE = "dind"
PROJECT_ROOT_PATH = "../.." PROJECT_ROOT_PATH = "../.."
version = "4.13.2-dev" version = "4.10.5"
@init @init

View file

View file

@ -6,7 +6,7 @@ from ddadevops import *
name = "ddadevops" name = "ddadevops"
MODULE = "kotlin" MODULE = "kotlin"
PROJECT_ROOT_PATH = "../.." PROJECT_ROOT_PATH = "../.."
version = "4.13.2-dev" version = "4.10.5"
@init @init

View file

View file

@ -6,7 +6,7 @@ from ddadevops import *
name = "ddadevops" name = "ddadevops"
MODULE = "python" MODULE = "python"
PROJECT_ROOT_PATH = "../.." PROJECT_ROOT_PATH = "../.."
version = "4.13.2-dev" version = "4.10.5"
@init @init

View file

View file

@ -51,10 +51,7 @@ class ImageBuildService:
else: else:
self.__copy_build_resources_from_dir__(devops) self.__copy_build_resources_from_dir__(devops)
self.file_api.cp_recursive("image", build_path) self.file_api.cp_recursive("image", build_path)
try:
self.file_api.cp_recursive("test", build_path) self.file_api.cp_recursive("test", build_path)
except:
print("Folder 'test' not found")
def image(self, devops: Devops): def image(self, devops: Devops):
image = devops.specialized_builds[BuildType.IMAGE] image = devops.specialized_builds[BuildType.IMAGE]

View file

@ -53,8 +53,7 @@ class ReleaseService:
bump_version = release_version.create_bump() bump_version = release_version.create_bump()
release_message = f"release: {release_version.to_string()}" release_message = f"release: {release_version.to_string()}"
bump_message = f"bump version to: {bump_version.to_string()}" bump_message = f"bump version to: {bump_version.to_string()}"
release_tag = f"{release.release_tag_prefix}{release_version.to_string()}" self.git_api.tag_annotated(release_version.to_string(), release_message, 0)
self.git_api.tag_annotated(release_tag, release_message, 0)
self.__set_version_and_commit__( self.__set_version_and_commit__(
bump_version, bump_version,
release.build_files(), release.build_files(),

View file

@ -11,7 +11,6 @@ class BuildFileType(Enum):
JS = ".json" JS = ".json"
JAVA_GRADLE = ".gradle" JAVA_GRADLE = ".gradle"
JAVA_CLOJURE = ".clj" JAVA_CLOJURE = ".clj"
JAVA_CLOJURE_EDN = ".edn"
PYTHON = ".py" PYTHON = ".py"
@ -42,8 +41,6 @@ class BuildFile(Validateable):
result = BuildFileType.JAVA_CLOJURE result = BuildFileType.JAVA_CLOJURE
case ".py": case ".py":
result = BuildFileType.PYTHON result = BuildFileType.PYTHON
case ".edn":
result = BuildFileType.JAVA_CLOJURE_EDN
case _: case _:
result = None result = None
return result return result
@ -51,13 +48,11 @@ class BuildFile(Validateable):
def __get_file_type_regex_str(self, file_type: BuildFileType): def __get_file_type_regex_str(self, file_type: BuildFileType):
match file_type: match file_type:
case BuildFileType.JAVA_GRADLE: case BuildFileType.JAVA_GRADLE:
return r"(?P<pre_version>\bversion\s?=\s?)\"(?P<version>\d*\.\d*\.\d*(-SNAPSHOT)?)\"" return r'(?P<pre_version>\bversion\s?=\s?)\"(?P<version>\d*\.\d*\.\d*(-SNAPSHOT)?)\"'
case BuildFileType.PYTHON: case BuildFileType.PYTHON:
return r"(?P<pre_version>\bversion\s?=\s?)\"(?P<version>\d*\.\d*\.\d*(-SNAPSHOT|-dev\d*)?)\"" return r'(?P<pre_version>\bversion\s?=\s?)\"(?P<version>\d*\.\d*\.\d*(-SNAPSHOT|-dev\d*)?)\"'
case BuildFileType.JAVA_CLOJURE: case BuildFileType.JAVA_CLOJURE:
return r"(?P<pre_version>\(defproject\s(\S)*\s)\"(?P<version>\d*\.\d*\.\d*(-SNAPSHOT)?)\"" return r'(?P<pre_version>\(defproject\s(\S)*\s)\"(?P<version>\d*\.\d*\.\d*(-SNAPSHOT)?)\"'
case BuildFileType.JAVA_CLOJURE_EDN:
return r"(?P<pre_version>\:version\s+)\"(?P<version>\d*\.\d*\.\d*(-SNAPSHOT)?)\""
case _: case _:
return "" return ""
@ -67,15 +62,8 @@ class BuildFile(Validateable):
match build_file_type: match build_file_type:
case BuildFileType.JS: case BuildFileType.JS:
version_str = json.loads(self.content)["version"] version_str = json.loads(self.content)["version"]
case ( case BuildFileType.JAVA_GRADLE | BuildFileType.PYTHON | BuildFileType.JAVA_CLOJURE:
BuildFileType.JAVA_GRADLE version_str = re.search(self.__get_file_type_regex_str(build_file_type), self.content).group("version")
| BuildFileType.PYTHON
| BuildFileType.JAVA_CLOJURE
| BuildFileType.JAVA_CLOJURE_EDN
):
version_str = re.search(
self.__get_file_type_regex_str(build_file_type), self.content
).group("version")
except: except:
raise RuntimeError(f"Version not found in file {self.file_path}") raise RuntimeError(f"Version not found in file {self.file_path}")
@ -96,15 +84,10 @@ class BuildFile(Validateable):
json_data = json.loads(self.content) json_data = json.loads(self.content)
json_data["version"] = new_version.to_string() json_data["version"] = new_version.to_string()
self.content = json.dumps(json_data, indent=4) self.content = json.dumps(json_data, indent=4)
case ( case BuildFileType.JAVA_GRADLE | BuildFileType.PYTHON | BuildFileType.JAVA_CLOJURE:
BuildFileType.JAVA_GRADLE
| BuildFileType.PYTHON
| BuildFileType.JAVA_CLOJURE
| BuildFileType.JAVA_CLOJURE_EDN
):
substitute = re.sub( substitute = re.sub(
self.__get_file_type_regex_str(build_file_type), self.__get_file_type_regex_str(build_file_type),
rf'\g<pre_version>"{new_version.to_string()}"', fr'\g<pre_version>"{new_version.to_string()}"',
self.content, self.content,
1, 1,
) )

View file

@ -78,12 +78,6 @@ class DnsRecord(Validateable):
result.append("ipv4 & ipv6 may not both be empty.") result.append("ipv4 & ipv6 may not both be empty.")
return result return result
def ip(self) -> str:
if (self.ipv4):
return self.ipv4
else:
return self.ipv6
class Devops(Validateable): class Devops(Validateable):
def __init__( def __init__(

View file

@ -20,14 +20,6 @@ CONFIG_CERTMANAGER = """certmanager:
""" """
CONFIG_ECHO = """echo: $echo CONFIG_ECHO = """echo: $echo
""" """
CONFIG_HETZNER_CSI = """hetzner:
hcloudApiToken:
source: "PLAIN" # PLAIN, GOPASS or PROMPT
parameter: $hcloud_api # the api key for the hetzner cloud
encryptionPassphrase:
source: "PLAIN" # PLAIN, GOPASS or PROMPT
parameter: $encryption # the encryption passphrase for created volumes
"""
class K3s(Validateable): class K3s(Validateable):
@ -36,11 +28,8 @@ class K3s(Validateable):
self.k3s_letsencrypt_email = inp.get("k3s_letsencrypt_email") self.k3s_letsencrypt_email = inp.get("k3s_letsencrypt_email")
self.k3s_letsencrypt_endpoint = inp.get("k3s_letsencrypt_endpoint", "staging") self.k3s_letsencrypt_endpoint = inp.get("k3s_letsencrypt_endpoint", "staging")
self.k3s_app_filename_to_provision = inp.get("k3s_app_filename_to_provision") self.k3s_app_filename_to_provision = inp.get("k3s_app_filename_to_provision")
self.k3s_enable_echo = inp.get("k3s_enable_echo", None) self.k3s_enable_echo = inp.get("k3s_enable_echo", "false")
self.k3s_provs_template = inp.get("k3s_provs_template", None) self.k3s_provs_template = inp.get("k3s_provs_template", None)
self.k3s_enable_hetzner_csi = inp.get("k3s_enable_hetzner_csi", False)
self.k3s_hetzner_api_token = inp.get("k3s_hetzner_api_token", None)
self.k3s_hetzner_encryption_passphrase = inp.get("k3s_hetzner_encryption_passphrase", None)
self.provision_dns: Optional[DnsRecord] = None self.provision_dns: Optional[DnsRecord] = None
def validate(self) -> List[str]: def validate(self) -> List[str]:
@ -48,9 +37,6 @@ class K3s(Validateable):
result += self.__validate_is_not_empty__("k3s_letsencrypt_email") result += self.__validate_is_not_empty__("k3s_letsencrypt_email")
result += self.__validate_is_not_empty__("k3s_letsencrypt_endpoint") result += self.__validate_is_not_empty__("k3s_letsencrypt_endpoint")
result += self.__validate_is_not_empty__("k3s_app_filename_to_provision") result += self.__validate_is_not_empty__("k3s_app_filename_to_provision")
if self.k3s_enable_hetzner_csi:
result += self.__validate_is_not_empty__("k3s_hetzner_api_token")
result += self.__validate_is_not_empty__("k3s_hetzner_encryption_passphrase")
if self.provision_dns: if self.provision_dns:
result += self.provision_dns.validate() result += self.provision_dns.validate()
return result return result
@ -75,9 +61,6 @@ class K3s(Validateable):
substitutes["letsencrypt_endpoint"] = self.k3s_letsencrypt_endpoint substitutes["letsencrypt_endpoint"] = self.k3s_letsencrypt_endpoint
if self.k3s_enable_echo is not None: if self.k3s_enable_echo is not None:
substitutes["echo"] = self.k3s_enable_echo substitutes["echo"] = self.k3s_enable_echo
if self.k3s_enable_hetzner_csi:
substitutes["hcloud_api"] = self.k3s_hetzner_api_token
substitutes["encryption"] = self.k3s_hetzner_encryption_passphrase
return self.__config_template__().substitute(substitutes) return self.__config_template__().substitute(substitutes)
def command(self, devops: Devops): def command(self, devops: Devops):
@ -86,7 +69,7 @@ class K3s(Validateable):
cmd = [ cmd = [
"provs-server.jar", "provs-server.jar",
"k3s", "k3s",
f"{self.k3s_provision_user}@{self.provision_dns.ip()}", f"{self.k3s_provision_user}@{self.provision_dns.fqdn}",
"-c", "-c",
f"{devops.build_path()}/out_k3sServerConfig.yaml", f"{devops.build_path()}/out_k3sServerConfig.yaml",
"-a", "-a",
@ -106,6 +89,4 @@ class K3s(Validateable):
template_text += CONFIG_IPV4 template_text += CONFIG_IPV4
if self.provision_dns.ipv6 is not None: if self.provision_dns.ipv6 is not None:
template_text += CONFIG_IPV6 template_text += CONFIG_IPV6
if self.k3s_enable_hetzner_csi:
template_text += CONFIG_HETZNER_CSI
return Template(template_text) return Template(template_text)

View file

@ -24,7 +24,6 @@ class Release(Validateable):
"release_secondary_build_files", [] "release_secondary_build_files", []
) )
self.version = version self.version = version
self.release_tag_prefix = inp.get("release_tag_prefix", "")
self.release_artifact_server_url = inp.get("release_artifact_server_url") self.release_artifact_server_url = inp.get("release_artifact_server_url")
self.release_organisation = inp.get("release_organisation") self.release_organisation = inp.get("release_organisation")
self.release_repository_name = inp.get("release_repository_name") self.release_repository_name = inp.get("release_repository_name")

View file

@ -58,8 +58,8 @@ class ImageApi:
) )
def drun(self, name: str): def drun(self, name: str):
self.execution_api.execute_live( self.execution_api.execute(
f'docker run -it {name} /bin/bash' f'docker run -it --entrypoint="" {name} /bin/bash'
) )
def dockerhub_login(self, username: str, password: str): def dockerhub_login(self, username: str, password: str):
@ -121,12 +121,13 @@ class ExecutionApi:
if dry_run: if dry_run:
print(command) print(command)
else: else:
process = Popen(command, shell=shell) process = Popen(command, stdout=PIPE, shell=shell)
outs, errs = process.communicate() for line in iter(process.stdout.readline, b""):
while outs is not None: print(line.decode("utf-8"), end="")
stdout.buffer.write(outs) process.stdout.close()
if process.returncode != 0: return_code = process.wait()
raise RuntimeError(f"Execute live '{command}' failed with code {process.returncode}\nerrs: {errs}") if return_code != 0:
raise RuntimeError(f"Execute live failed with code: {return_code}")
class EnvironmentApi: class EnvironmentApi:

View file

@ -183,34 +183,6 @@ def test_should_parse_and_set_version_for_clj():
== sut.content == sut.content
) )
def test_should_parse_and_set_version_for_clj_edn():
sut = BuildFile(
Path("./deps.edn"),
"""
{:project {:name org.domaindrivenarchitecture/dda-backup
:version "1.1.5-SNAPSHOT"}
}
""",
)
assert sut.get_version() == Version.from_str("1.1.5-SNAPSHOT", "SNAPSHOT")
sut = BuildFile(
Path("./deps.edn"),
"""
{:project {:name org.domaindrivenarchitecture/dda-backup
:version "1.1.5-SNAPSHOT"}
}
""",
)
sut.set_version(Version.from_str("1.1.5-SNAPSHOT", "SNAPSHOT").create_major())
assert (
'\n{:project {:name org.domaindrivenarchitecture/dda-backup\n :version "2.0.0"}\n\n}\n'
== sut.content
)
def test_should_throw_for_clj_wrong_version(): def test_should_throw_for_clj_wrong_version():
sut = BuildFile( sut = BuildFile(
Path("./project.clj"), Path("./project.clj"),

View file

@ -24,7 +24,7 @@ def test_should_calculate_command():
assert ( assert (
"provs-server.jar " "provs-server.jar "
+ "k3s " + "k3s "
+ "k3s_provision_user@::1 " + "k3s_provision_user@example.org "
+ "-c " + "-c "
+ "root_path/target/name/module/out_k3sServerConfig.yaml " + "root_path/target/name/module/out_k3sServerConfig.yaml "
+ "-a " + "-a "