Add nightly CI base image and migrate workflows to python-repositories-ci.
Code Quality Pipeline / code-quality (pull_request) Failing after 1s
Test Python Package / unit-tests (pull_request) Failing after 0s
Test Python Package / integration-tests (pull_request) Failing after 0s
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / coverage-report (pull_request) Has been skipped

Pre-build Python, uv, and locked deps in a Gitea container image so daily jobs skip repeated bootstrap; document runner registry auth and ci-bot credentials.

Co-authored-by: Cursor <[email protected]>
This commit is contained in:
Brian Bjarke Jensen
2026-07-09 22:21:22 +02:00
co-authored by Cursor
parent 369295b8ae
commit 3b05260323
11 changed files with 281 additions and 86 deletions
+16
View File
@@ -0,0 +1,16 @@
.git
.github
.gitea
.mypy_cache
.pytest_cache
.ruff_cache
.venv
__pycache__
*.pyc
.coverage
coverage-unit
coverage-integration
coverage.txt
dist
build
*.egg-info
+52
View File
@@ -0,0 +1,52 @@
# Build and publish the per-repo CI base image to the Gitea container registry.
name: Build CI Image
on:
schedule:
- cron: "0 2 * * *"
workflow_dispatch:
push:
branches:
- main
paths:
- pyproject.toml
- uv.lock
- docker/ci/**
env:
REGISTRY: gitea.lille-vemmelund.dk
REGISTRY_USER: ci-bot
IMAGE: gitea.lille-vemmelund.dk/brian/python-repositories-ci
jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Log in to Gitea container registry
env:
CI_RUNNER_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
run: |
echo "$CI_RUNNER_TOKEN" | docker login "$REGISTRY" -u "$REGISTRY_USER" --password-stdin
- name: Build CI image
env:
CI_RUNNER_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
run: |
echo "$CI_RUNNER_TOKEN" > /tmp/uv_token
docker build -f docker/ci/Dockerfile \
--secret id=uv_token,src=/tmp/uv_token \
-t "${IMAGE}:latest" \
.
rm -f /tmp/uv_token
- name: Tag and push CI image
run: |
STAMP="$(date -u +%Y%m%d%H%M)"
docker tag "${IMAGE}:latest" "${IMAGE}:${STAMP}"
docker push "${IMAGE}:latest"
docker push "${IMAGE}:${STAMP}"
echo "Pushed ${IMAGE}:latest and ${IMAGE}:${STAMP}"
+5 -11
View File
@@ -8,23 +8,17 @@ on:
jobs: jobs:
code-quality: code-quality:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Install dependencies
env: env:
UV_LINK_MODE: copy UV_LINK_MODE: copy
run: uv sync --all-extras UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
- name: Type check with mypy - name: Type check with mypy
run: uv run mypy . run: uv run mypy .
+4 -9
View File
@@ -7,22 +7,17 @@ on:
jobs: jobs:
update-check: update-check:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
with: with:
token: ${{ secrets.CI_RUNNER_TOKEN }} token: ${{ secrets.CI_RUNNER_TOKEN }}
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Upgrade dependencies - name: Upgrade dependencies
env:
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv lock --upgrade run: uv lock --upgrade
- name: Commit and push changes - name: Commit and push changes
+9 -10
View File
@@ -9,25 +9,24 @@ on:
jobs: jobs:
build-and-publish: build-and-publish:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Update version in pyproject.toml to match tag - name: Update version in pyproject.toml to match tag
env:
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: scripts/ci/bump-version.sh --from-tag "${GITHUB_REF##*/}" run: scripts/ci/bump-version.sh --from-tag "${GITHUB_REF##*/}"
- name: Build package - name: Build package
env:
UV_LINK_MODE: copy
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: | run: |
uv sync --no-dev uv sync --no-dev --frozen
uv build uv build
- name: Publish to Gitea Package Registry - name: Publish to Gitea Package Registry
+4 -11
View File
@@ -8,7 +8,7 @@ on:
jobs: jobs:
release: release:
if: ${{ !contains(github.event.head_commit.message, '[skip ci]') }} if: ${{ !contains(github.event.head_commit.message, '[skip ci]') }}
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
@@ -22,19 +22,12 @@ jobs:
COMMIT_MSG: ${{ github.event.head_commit.message }} COMMIT_MSG: ${{ github.event.head_commit.message }}
run: scripts/ci/parse-merge-commit.sh "$COMMIT_MSG" run: scripts/ci/parse-merge-commit.sh "$COMMIT_MSG"
- name: Set up Python
if: steps.meta.outputs.bump != 'skip'
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
if: steps.meta.outputs.bump != 'skip'
run: pip install uv
- name: Bump version - name: Bump version
if: steps.meta.outputs.bump != 'skip' if: steps.meta.outputs.bump != 'skip'
id: bump id: bump
env:
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: scripts/ci/bump-version.sh "${{ steps.meta.outputs.bump }}" run: scripts/ci/bump-version.sh "${{ steps.meta.outputs.bump }}"
- name: Generate release notes - name: Generate release notes
+5 -11
View File
@@ -7,23 +7,17 @@ on:
jobs: jobs:
safety: safety:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Install dependencies
env: env:
UV_LINK_MODE: copy UV_LINK_MODE: copy
run: uv sync UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --frozen
- name: Run safety check - name: Run safety check
run: uv run safety check run: uv run safety check
+15 -33
View File
@@ -8,23 +8,17 @@ on:
jobs: jobs:
unit-tests: unit-tests:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Install dependencies
env: env:
UV_LINK_MODE: copy UV_LINK_MODE: copy
run: uv sync --all-extras UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
- name: Run unit tests - name: Run unit tests
run: | run: |
@@ -43,23 +37,17 @@ jobs:
compression-level: 0 compression-level: 0
integration-tests: integration-tests:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Install dependencies
env: env:
UV_LINK_MODE: copy UV_LINK_MODE: copy
run: uv sync --all-extras UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
- name: Verify Docker - name: Verify Docker
run: docker info run: docker info
@@ -83,24 +71,18 @@ jobs:
coverage-report: coverage-report:
# Merges unit + integration coverage and enforces fail_under from pyproject.toml. # Merges unit + integration coverage and enforces fail_under from pyproject.toml.
needs: [unit-tests, integration-tests] needs: [unit-tests, integration-tests]
runs-on: ubuntu-latest runs-on: python-repositories-ci
if: github.event_name == 'pull_request' || github.event_name == 'push' if: github.event_name == 'pull_request' || github.event_name == 'push'
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Install dependencies
env: env:
UV_LINK_MODE: copy UV_LINK_MODE: copy
run: uv sync --all-extras UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
- name: Download unit coverage - name: Download unit coverage
uses: https://github.com/christopherHX/gitea-download-artifact@v4 uses: https://github.com/christopherHX/gitea-download-artifact@v4
+8 -1
View File
@@ -166,7 +166,14 @@ uv run pytest -v # full suite (requires Doc
Integration tests are marked with `@pytest.mark.integration` and require Docker (testcontainers). Run unit tests alone for quick local feedback. Integration tests are marked with `@pytest.mark.integration` and require Docker (testcontainers). Run unit tests alone for quick local feedback.
CI runs unit and integration tests in parallel with coverage, then merges `.coverage` artifacts in a follow-up job (via [christopherhx/gitea-\*-artifact@v4](https://github.com/christopherHX/gitea-upload-artifact) for Gitea 1.26 compatibility). Combined coverage must be at least **90%**; the floor is set by [`fail_under` in `pyproject.toml`](pyproject.toml#L45-L48) and enforced after merging unit and integration coverage, not on unit-only runs. ### CI base image
Gitea Actions jobs use a pre-built image (`python-repositories-ci`) with Python 3.12,
`uv`, and locked dependencies baked in. The image is rebuilt nightly and when
`uv.lock` changes; see [`docs/ci-image.md`](docs/ci-image.md) for runner setup and
bootstrap order.
CI runs unit and integration tests in parallel with coverage, then merges `.coverage` artifacts in a follow-up job (via [christopherhx/gitea-\*-artifact@v4](https://github.com/christopherHX/gitea-upload-artifact) for Gitea 1.26 compatibility). Combined coverage must meet the floor in [`fail_under` in `pyproject.toml`](pyproject.toml#L45-L48); enforcement happens after merging unit and integration coverage, not on unit-only runs.
To check coverage locally (requires Docker for the full suite): To check coverage locally (requires Docker for the full suite):
+28
View File
@@ -0,0 +1,28 @@
# CI job image for Gitea Actions (act_runner requires Node.js in job containers).
FROM node:20-bookworm
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates \
curl \
docker.io \
git \
&& rm -rf /var/lib/apt/lists/*
COPY --from=ghcr.io/astral-sh/uv:0.7.0 /uv /usr/local/bin/uv
ENV UV_PYTHON_INSTALL_DIR=/opt/uv-python \
UV_PYTHON=3.12 \
UV_LINK_MODE=copy \
PATH="/app/.venv/bin:${PATH}"
RUN uv python install 3.12
WORKDIR /app
COPY pyproject.toml uv.lock .python-version ./
RUN --mount=type=secret,id=uv_token \
UV_INDEX_GITEA_USERNAME=ci-bot \
UV_INDEX_GITEA_PASSWORD="$(cat /run/secrets/uv_token)" \
uv sync --all-extras --frozen --no-install-project
+135
View File
@@ -0,0 +1,135 @@
# CI base image
This repository uses a **per-repo Docker image** for Gitea Actions jobs instead of
installing Python and `uv` on every run. Infrastructure images (redis, minio, etc.)
are cached cluster-wide via Harbor (see homelab-platform
[`docs/harbor-registry-mirror.md`](https://gitea.lille-vemmelund.dk/LilleVemmelund/homelab-platform/src/branch/main/docs/harbor-registry-mirror.md)).
## Image contents
Built from [`docker/ci/Dockerfile`](../docker/ci/Dockerfile):
- Node.js 20 (required by act_runner job containers)
- Python 3.12 (installed via `uv python install`) and pinned `uv` 0.7.0
- Docker CLI (integration tests via testcontainers)
- Dev dependencies from `uv.lock` (`uv sync --all-extras --no-install-project`)
Published to the Gitea container registry:
- `gitea.lille-vemmelund.dk/brian/python-repositories-ci:latest`
- `gitea.lille-vemmelund.dk/brian/python-repositories-ci:YYYYMMDDHHmm` (timestamped rollback tag)
## Rebuild triggers
[`ci-image.yml`](../.gitea/workflows/ci-image.yml) runs on:
- Nightly cron (`0 2 * * *` UTC)
- Manual `workflow_dispatch`
- Push to `main` when `pyproject.toml`, `uv.lock`, or `docker/ci/**` change
## Workflow usage
Python jobs use `runs-on: python-repositories-ci` and a fast incremental sync:
```yaml
runs-on: python-repositories-ci
steps:
- uses: actions/checkout@v4
- name: Sync dependencies
env:
UV_LINK_MODE: copy
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
```
`ci-image.yml` uses `runs-on: ubuntu-latest` so it can bootstrap before the custom
image exists.
## Registry authentication
act_runner pulls the job image **before any workflow step runs**, so a `docker
login` step inside a job cannot authenticate that pull. Authentication must be
configured on the runner (or via a job-level `container.credentials` block — not
used here).
### k8s Gitea runners (automatic)
The homelab-platform runners mount Gitea registry credentials automatically:
- Secret: `gitea-runners/gitea-registry-dockerconfig` (created by
[`scripts/create-gitea-registry-secret.sh`](https://gitea.lille-vemmelund.dk/LilleVemmelund/homelab-platform/src/branch/main/scripts/create-gitea-registry-secret.sh))
- Mounted at `/root/.docker/config.json` on the **runner** container (not DinD)
- Configured in
[`platform/gitea-runners/values.yaml`](https://gitea.lille-vemmelund.dk/LilleVemmelund/homelab-platform/src/branch/main/platform/gitea-runners/values.yaml)
No manual `docker login` is required on the in-cluster runners once the secret
exists. To create or rotate credentials:
```bash
export KUBECONFIG=/path/to/homelab-cluster/talos/_out/kubeconfig
export GITEA_REGISTRY_USERNAME='ci-bot'
export GITEA_REGISTRY_PASSWORD='personal-access-token-with-read-package'
bash scripts/create-gitea-registry-secret.sh
```
Then restart runner pods so they pick up the updated secret. See homelab-platform
[`docs/gitea-actions-runners.md`](https://gitea.lille-vemmelund.dk/LilleVemmelund/homelab-platform/src/branch/main/docs/gitea-actions-runners.md)
for full runner setup.
Verify on a running pod:
```bash
kubectl -n gitea-runners exec homelab-cluster-gitea-runner-0 -c runner -- \
test -f /root/.docker/config.json && echo "registry auth mounted"
```
### Standalone runners (e.g. Unraid `homelab`)
The 4th runner is outside the k8s cluster and does **not** get the automatic
mount. Configure registry auth manually on that host:
```bash
docker login gitea.lille-vemmelund.dk -u ci-bot -p <token>
```
Also add the `python-repositories-ci` label to that runner's act_runner config.
### `ci-image.yml` push login
The build workflow still runs `docker login` before `docker push`. That step
authenticates **DinD inside the job** for pushing the image to Gitea — a different
code path from act_runner pulling the job container.
## Runner label
Jobs use the `python-repositories-ci` act_runner label, configured in
homelab-platform
[`platform/gitea-runners/values.yaml`](https://gitea.lille-vemmelund.dk/LilleVemmelund/homelab-platform/src/branch/main/platform/gitea-runners/values.yaml):
```yaml
python-repositories-ci:docker://gitea.lille-vemmelund.dk/brian/python-repositories-ci:latest
```
After label changes, roll runner pods so they re-register with Gitea.
## Bootstrap order
1. Ensure homelab-platform runners have `gitea-registry-dockerconfig` and the
`python-repositories-ci` label (see homelab-platform docs).
2. Merge `ci-image.yml`, `docker/ci/Dockerfile`, and workflow migrations to `main`.
3. Run **Build CI Image** manually once (Actions → Build CI Image → Run workflow).
4. Confirm a test workflow job starts on `python-repositories-ci`.
Until step 3 completes, jobs targeting `python-repositories-ci` will fail because
the image does not exist in the Gitea registry yet.
## Local build
```bash
echo "$CI_RUNNER_TOKEN" > /tmp/uv_token
docker build -f docker/ci/Dockerfile \
--secret id=uv_token,src=/tmp/uv_token \
-t python-repositories-ci:local .
rm -f /tmp/uv_token
```