Compare commits

..
44 Commits
Author SHA1 Message Date
CI Bot fef9552cbf chore: release v0.3.2 [skip ci] 2026-06-30 14:12:18 +00:00
brian e9b21895f4 Merge pull request '[patch] Fix optional dependency handling for Redis and MinIO adapters' (#19) from cursor/fix-optional-dependencies into main
Code Quality Pipeline / code-quality (push) Successful in 31s
Release on merge to main / release (push) Successful in 8s
Test Python Package / test (push) Successful in 37s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/19
2026-06-30 16:08:01 +02:00
Brian Bjarke JensenandCursor f5953906c1 Fix optional-deps import test for full-suite execution.
Code Quality Pipeline / code-quality (pull_request) Successful in 31s
PR Title Check / check-title (pull_request) Successful in 5s
Test Python Package / test (pull_request) Successful in 41s
Run the base-import assertion in a subprocess so integration tests do not pollute sys.modules.

Co-authored-by: Cursor <[email protected]>
2026-06-30 16:05:41 +02:00
Brian Bjarke JensenandCursor 1f02195c27 Delegate top-level adapter imports to adapters package.
Code Quality Pipeline / code-quality (pull_request) Successful in 47s
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / test (pull_request) Failing after 1m1s
Keep lazy loading in adapters/__init__.py as the single place to register new adapters.

Co-authored-by: Cursor <[email protected]>
2026-06-30 15:57:52 +02:00
Brian Bjarke JensenandCursor f092ca2022 Fix trailing whitespace in LICENSE and .gitignore.
Code Quality Pipeline / code-quality (pull_request) Successful in 37s
PR Title Check / check-title (pull_request) Successful in 5s
Test Python Package / test (pull_request) Failing after 47s
Co-authored-by: Cursor <[email protected]>
2026-06-30 15:48:01 +02:00
Brian Bjarke JensenandCursor c6eed43d70 Fix optional dependency handling for Redis and MinIO adapters.
Lazy-load adapters at package boundaries and fail fast with install hints when extras are missing.

Co-authored-by: Cursor <[email protected]>
2026-06-30 15:47:04 +02:00
brian 9e4b1e2c5e Merge pull request 'Avoid testcontainers Redis deprecation warning in integration tests.' (#18) from fix/redis-testcontainer-deprecation into main
Code Quality Pipeline / code-quality (push) Successful in 31s
Release on merge to main / release (push) Successful in 6s
Test Python Package / test (push) Successful in 43s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/18
2026-06-29 15:04:39 +02:00
Brian Bjarke JensenandCursor 0096df78a5 Avoid testcontainers Redis deprecation warning in integration tests.
Code Quality Pipeline / code-quality (pull_request) Successful in 47s
PR Title Check / check-title (pull_request) Successful in 5s
Test Python Package / test (pull_request) Successful in 52s
Use a local RedisTestContainer with wait strategies instead of the deprecated RedisContainer, and keep conftest focused on fixtures.

Co-authored-by: Cursor <[email protected]>
2026-06-29 15:00:45 +02:00
brian 3073bfbf50 Merge pull request 'Fix PR coverage comment posting in CI.' (#17) from fix/coverage-pr-comment into main
Code Quality Pipeline / code-quality (push) Successful in 36s
Release on merge to main / release (push) Successful in 5s
Test Python Package / test (push) Successful in 47s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/17
2026-06-29 11:37:34 +02:00
Brian Bjarke JensenandCursor 30a1de32e8 Fix PR coverage comment posting in CI.
Test Python Package / test (pull_request) Successful in 48s
Code Quality Pipeline / code-quality (pull_request) Successful in 31s
PR Title Check / check-title (pull_request) Successful in 5s
Build the Gitea API payload with json.dumps so multiline coverage output is valid JSON, and fail the step when the API rejects the request.

Co-authored-by: Cursor <[email protected]>
2026-06-28 23:46:49 +02:00
brian 8465957976 Merge pull request '[patch] update dependencies' (#16) from renovate/auto-deps-update into main
Test Python Package / test (push) Successful in 47s
Code Quality Pipeline / code-quality (push) Successful in 30s
Release on merge to main / release (push) Failing after 7s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/16
2026-06-28 23:46:13 +02:00
Brian Bjarke Jensen 3ecb9c3136 chore: retrigger CI
Code Quality Pipeline / code-quality (pull_request) Successful in 46s
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / test (pull_request) Successful in 47s
2026-06-28 23:41:02 +02:00
Brian Bjarke JensenandCursor 0f31170209 Fix mypy errors from minio 7.2.20 type stubs.
Code Quality Pipeline / code-quality (pull_request) Successful in 36s
PR Title Check / check-title (pull_request) Failing after 6s
Test Python Package / test (pull_request) Successful in 43s
Guard bucket_name before Minio API calls, update S3Error test mocks for the new constructor signature, and add types-redis to the pre-commit mypy hook.

Co-authored-by: Cursor <[email protected]>
2026-06-28 23:32:31 +02:00
CI Bot e3cab5e0b6 chore(deps): update dependencies [automated]
Code Quality Pipeline / code-quality (pull_request) Failing after 22s
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / test (pull_request) Successful in 1m3s
2026-06-28 21:22:26 +00:00
brian f428a974f6 Merge pull request 'Fix dependency bot PR creation for Gitea.' (#15) from cursor/fix-dependency-bot-gitea-pr into main
Code Quality Pipeline / code-quality (push) Successful in 31s
Release on merge to main / release (push) Successful in 6s
Test Python Package / test (push) Successful in 57s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/15
2026-06-28 22:56:14 +02:00
Brian Bjarke JensenandCursor a3b9912107 Fix dependency bot PR creation for Gitea.
Code Quality Pipeline / code-quality (pull_request) Successful in 27s
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / test (pull_request) Successful in 1m3s
Replace peter-evans/create-pull-request with a Gitea API script that commits, pushes, and opens the deps update PR.

Co-authored-by: Cursor <[email protected]>
2026-06-28 22:41:59 +02:00
brian abaa078471 Merge pull request 'Fix dependency update bot to use uv lock --upgrade.' (#14) from cursor/fix-dependency-update-bot-uv-lock into main
Code Quality Pipeline / code-quality (push) Successful in 47s
Release on merge to main / release (push) Successful in 6s
Test Python Package / test (push) Successful in 1m2s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/14
2026-06-28 22:34:48 +02:00
Brian Bjarke JensenandCursor 9864f0cc9d Fix dependency update bot to use uv lock --upgrade.
Code Quality Pipeline / code-quality (pull_request) Successful in 45s
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / test (pull_request) Successful in 53s
Replace invalid uv pip update/compile commands with the native lockfile upgrade workflow.

Co-authored-by: Cursor <[email protected]>
2026-06-28 22:30:28 +02:00
brian e39b467b6b Merge pull request 'Automate releases from PR titles and clean up CI workflows.' (#13) from cursor/pr-title-release-automation into main
Code Quality Pipeline / code-quality (push) Successful in 30s
Release on merge to main / release (push) Successful in 6s
Test Python Package / test (push) Successful in 1m4s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/13
2026-06-28 22:25:01 +02:00
Brian Bjarke Jensen 85c1a64e14 Sync uv.lock with pyproject.toml version 0.3.1.
Code Quality Pipeline / code-quality (pull_request) Successful in 30s
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / test (pull_request) Successful in 1m7s
2026-06-28 22:21:59 +02:00
Brian Bjarke JensenandCursor a8491c005c Automate releases from PR titles and clean up CI workflows.
Code Quality Pipeline / code-quality (pull_request) Failing after 39s
PR Title Check / check-title (pull_request) Successful in 5s
Test Python Package / test (pull_request) Successful in 1m3s
Add release and PR title checks, shared CI scripts, Gitea PR template, and align pyproject.toml to v0.3.1.

Co-authored-by: Cursor <[email protected]>
2026-06-28 22:18:40 +02:00
brian 2bb361e3a3 Merge pull request 'Add public repository interfaces and subclassable adapter CRUD API.' (#11) from cursor/public-repository-api into main
Code Quality Pipeline / code-quality (push) Successful in 30s
Test Python Package / test (push) Successful in 52s
Publish Python Package / build-and-publish (push) Successful in 22s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/11
2026-06-28 20:05:14 +02:00
Brian Bjarke JensenandCursor c5169df10d Use pre-commit for Prettier in CI and document local hook setup.
Test Python Package / test (pull_request) Successful in 50s
Code Quality Pipeline / code-quality (pull_request) Successful in 31s
Run prettier via pre-commit in code-quality workflow and add README instructions for installing and running hooks during development.

Co-authored-by: Cursor <[email protected]>
2026-06-28 19:56:25 +02:00
Brian Bjarke JensenandCursor 497533f0e3 Fix mypy context manager typing for adapter subclasses.
Code Quality Pipeline / code-quality (pull_request) Failing after 26s
Test Python Package / test (pull_request) Successful in 1m7s
Return Self from __enter__ so with-blocks preserve subclass types, and align mypy python_version with the 3.12 runtime target.

Co-authored-by: Cursor <[email protected]>
2026-06-28 19:42:15 +02:00
Brian Bjarke JensenandCursor e98fd3b90d Add public repository interfaces and subclassable adapter CRUD API.
Code Quality Pipeline / code-quality (pull_request) Failing after 19s
Test Python Package / test (pull_request) Successful in 50s
Define split JSON and object repository ABCs, promote adapter methods to public CRUD, add example domain repositories, modernize interface stubs, and make the package installable for tests.

Co-authored-by: Cursor <[email protected]>
2026-06-28 10:58:33 +02:00
brian f4280f15c1 Merge pull request 'Remove non-existent Mongo adapter from README.' (#10) from cleanup/remove-mongo-adapter-mentions into main
Test Python Package / test (push) Successful in 1m12s
Code Quality Pipeline / code-quality (push) Successful in 24s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/10
2026-06-21 23:05:47 +02:00
Brian Bjarke Jensen d9b9fa49a2 updated package settings
Code Quality Pipeline / code-quality (pull_request) Successful in 41s
Test Python Package / test (pull_request) Successful in 1m18s
2026-06-21 22:45:29 +02:00
Brian Bjarke Jensen 9d25f7c722 updated package settings
Code Quality Pipeline / code-quality (pull_request) Failing after 39s
Test Python Package / test (pull_request) Successful in 1m23s
2026-06-21 22:43:14 +02:00
Brian Bjarke Jensen 163581526d updated package settings
Code Quality Pipeline / code-quality (pull_request) Failing after 21s
Test Python Package / test (pull_request) Failing after 12s
2026-06-21 22:39:50 +02:00
Brian Bjarke Jensen 933c4ee7c5 updated package settings
Test Python Package / test (pull_request) Successful in 1m27s
Code Quality Pipeline / code-quality (pull_request) Successful in 41s
2026-06-21 22:34:50 +02:00
Brian Bjarke Jensen 2b0d98f84c updated package settings
Code Quality Pipeline / code-quality (pull_request) Failing after 51s
Test Python Package / test (pull_request) Failing after 2m1s
2026-06-21 22:26:41 +02:00
Brian Bjarke JensenandCursor 729fa48505 Remove non-existent Mongo adapter from README.
Code Quality Pipeline / code-quality (pull_request) Failing after 2m9s
Test Python Package / test (pull_request) Failing after 2m10s
The package only ships Redis and MinIO adapters, so the docs should not reference a mongo extra.

Co-authored-by: Cursor <[email protected]>
2026-06-21 22:17:44 +02:00
Brian Bjarke Jensen fa011be862 Merge pull request 'added support for content type when putting object in minio' (#9) from add-support-for-content-type-in-minio into main
Code Quality Pipeline / code-quality (push) Successful in 26s
Test Python Package / test (push) Successful in 33s
Publish Python Package / build-and-publish (push) Successful in 15s
Dependency Update Bot / update-check (push) Failing after 10s
Security Audit / safety (push) Failing after 2m9s
Reviewed-on: brian/python-repositories#9
2025-09-18 19:40:27 +02:00
Brian Bjarke Jensen 2e9330949c ruff format fix
Code Quality Pipeline / code-quality (pull_request) Successful in 29s
Test Python Package / test (pull_request) Successful in 35s
2025-09-18 19:38:22 +02:00
Brian Bjarke Jensen 3ab8da92fb added support for content type when putting object in minio
Code Quality Pipeline / code-quality (pull_request) Failing after 24s
Test Python Package / test (pull_request) Successful in 1m9s
2025-09-18 19:34:05 +02:00
Brian Bjarke Jensen 1dd604dc2a Merge pull request 'moved import of optional packages into classes' (#6) from make-optional-parts-independent into main
Test Python Package / test (push) Successful in 34s
Publish Python Package / build-and-publish (push) Successful in 15s
Code Quality Pipeline / code-quality (push) Successful in 27s
Reviewed-on: brian/python-repositories#6
2025-09-17 12:38:58 +02:00
Brian Bjarke Jensen e5c4025aeb added graceful handling of missing optional packages
Code Quality Pipeline / code-quality (pull_request) Successful in 26s
Test Python Package / test (pull_request) Successful in 34s
2025-09-17 12:36:22 +02:00
Brian Bjarke Jensen 396cf83a05 code quality fixes
Code Quality Pipeline / code-quality (pull_request) Successful in 30s
Test Python Package / test (pull_request) Successful in 1m12s
2025-09-17 12:14:37 +02:00
Brian Bjarke Jensen 538b26b62f mypy fixed 2025-09-17 12:04:45 +02:00
Brian Bjarke Jensen ee6acc6791 moved import of optional packages into classes
Code Quality Pipeline / code-quality (pull_request) Failing after 1m53s
Test Python Package / test (pull_request) Failing after 22s
2025-09-17 11:44:47 +02:00
Brian Bjarke Jensen 0ca07c02ac Merge pull request 'exposed minio adapter' (#5) from expose-minio-adapter into main
Code Quality Pipeline / code-quality (push) Successful in 36s
Test Python Package / test (push) Successful in 35s
Publish Python Package / build-and-publish (push) Successful in 15s
Reviewed-on: brian/python-repositories#5
2025-09-15 14:44:00 +02:00
Brian Bjarke Jensen 7446e88850 added py.typed mypy config file
Code Quality Pipeline / code-quality (pull_request) Successful in 34s
Test Python Package / test (pull_request) Successful in 40s
2025-09-15 14:15:39 +02:00
Brian Bjarke Jensen d5f437f77c ruff format fix
Code Quality Pipeline / code-quality (pull_request) Successful in 1m21s
Test Python Package / test (pull_request) Successful in 1m30s
2025-09-15 13:59:03 +02:00
Brian Bjarke Jensen 7beca7c398 exposed minio adapter
Code Quality Pipeline / code-quality (pull_request) Failing after 1m9s
Test Python Package / test (pull_request) Successful in 1m29s
2025-09-15 13:38:06 +02:00
45 changed files with 2412 additions and 1087 deletions
+19
View File
@@ -0,0 +1,19 @@
## Version bump
If this PR changes files under `python_repositories/`, the **title must** start with one of:
- `[patch]` or `[fix]` — bug fix (0.3.1 → 0.3.2)
- `[minor]` or `[feat]` — new feature (0.3.1 → 0.4.0)
- `[major]` or `[breaking]` — breaking change (0.3.1 → 1.0.0)
Docs-, CI-, or test-only PRs do not need a prefix.
**Example title:** `[minor] Add streaming support to RedisAdapter`
## Summary
<!-- What changed and why -->
## Test plan
- [ ] Integration tests pass locally
+3 -5
View File
@@ -16,7 +16,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
python-version-file: .python-version
- name: Install uv
run: pip install uv
@@ -36,9 +36,7 @@ jobs:
run: uv run ruff format --check .
- name: Pyupgrade check
run: uv run pyupgrade --py313-plus $(git ls-files '*.py') && git diff --exit-code
run: uv run pyupgrade --py312-plus $(git ls-files '*.py') && git diff --exit-code
- name: Prettier format
run: |
npm install --save-dev --save-exact prettier
npx prettier --check .
run: uv run pre-commit run prettier --all-files
+13 -15
View File
@@ -11,26 +11,24 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
token: ${{ secrets.CI_RUNNER_TOKEN }}
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
python-version-file: .python-version
- name: Install uv and pip-review
run: |
pip install uv pip-review
- name: Install uv
run: pip install uv
- name: Update dependencies in pyproject.toml
run: |
uv pip update --all --group dev
uv pip compile --all
- name: Upgrade dependencies
run: uv lock --upgrade
- name: Commit and push changes
uses: peter-evans/create-pull-request@v6
with:
commit-message: "chore(deps): update dependencies [automated]"
branch: "renovate/auto-deps-update"
title: "chore(deps): update dependencies"
body: "This PR was created automatically to update dependencies."
token: ${{ secrets.CI_RUNNER_TOKEN }}
env:
API_URL: ${{ vars.API_URL }}
REPO_OWNER: ${{ github.repository_owner }}
REPO_NAME: ${{ github.event.repository.name }}
CI_RUNNER_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
run: scripts/ci/create-deps-update-pr.sh
+23
View File
@@ -0,0 +1,23 @@
name: PR Title Check
on:
pull_request:
branches:
- main
jobs:
check-title:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check PR title when source files change
env:
PR_TITLE: ${{ github.event.pull_request.title }}
run: |
git fetch origin "${{ github.base_ref }}"
git diff --name-only "origin/${{ github.base_ref }}...HEAD" \
| scripts/ci/check-pr-title.sh "$PR_TITLE"
+2 -6
View File
@@ -17,17 +17,13 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Update version in pyproject.toml to match tag
run: |
TAG_VERSION="${GITHUB_REF##*/}"
VERSION="${TAG_VERSION#v}"
sed -i.bak -E "s/^version = \".*\"/version = \"${VERSION}\"/" pyproject.toml
rm pyproject.toml.bak
run: scripts/ci/bump-version.sh --from-tag "${GITHUB_REF##*/}"
- name: Build package
run: |
+73
View File
@@ -0,0 +1,73 @@
name: Release on merge to main
on:
push:
branches:
- main
jobs:
release:
if: ${{ !contains(github.event.head_commit.message, '[skip ci]') }}
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.CI_RUNNER_TOKEN }}
- name: Parse merge commit
id: meta
env:
COMMIT_MSG: ${{ github.event.head_commit.message }}
run: scripts/ci/parse-merge-commit.sh "$COMMIT_MSG"
- name: Bump version
if: steps.meta.outputs.bump != 'skip'
id: bump
run: scripts/ci/bump-version.sh "${{ steps.meta.outputs.bump }}"
- name: Generate release notes
if: steps.meta.outputs.bump != 'skip'
id: notes
env:
PR_TITLE: ${{ steps.meta.outputs.pr_title }}
run: |
PREV_TAG=$(git describe --tags --abbrev=0)
NEW_TAG="v${{ steps.bump.outputs.version }}"
scripts/ci/generate-release-notes.sh "$NEW_TAG" "$PR_TITLE" "$PREV_TAG"
- name: Commit, tag, and push
if: steps.meta.outputs.bump != 'skip'
env:
VERSION: ${{ steps.bump.outputs.version }}
CI_RUNNER_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
run: |
git config user.name "CI Bot"
git config user.email "[email protected]"
git add pyproject.toml
git commit -m "chore: release v${VERSION} [skip ci]"
git tag "v${VERSION}"
git remote set-url origin "https://x-access-token:${CI_RUNNER_TOKEN}@${GITHUB_SERVER_URL#https://}/${GITHUB_REPOSITORY}.git"
git push origin main
git push origin "v${VERSION}"
- name: Create Gitea release
if: steps.meta.outputs.bump != 'skip'
env:
API_URL: ${{ vars.API_URL }}
REPO_OWNER: ${{ github.repository_owner }}
REPO_NAME: ${{ github.event.repository.name }}
CI_RUNNER_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
VERSION: ${{ steps.bump.outputs.version }}
PR_TITLE: ${{ steps.meta.outputs.pr_title }}
RELEASE_BODY: ${{ steps.notes.outputs.body }}
run: |
SUMMARY=$(echo "$PR_TITLE" | sed -E 's/^\[(patch|fix|minor|feat|major|breaking)\][[:space:]]*//i')
export RELEASE_NAME="v${VERSION} — ${SUMMARY}"
PAYLOAD=$(python3 -c 'import json, os; print(json.dumps({"tag_name": "v" + os.environ["VERSION"], "name": os.environ["RELEASE_NAME"], "body": os.environ["RELEASE_BODY"]}))')
curl -s -X POST \
"${API_URL}/repos/${REPO_OWNER}/${REPO_NAME}/releases" \
-H "Authorization: token ${CI_RUNNER_TOKEN}" \
-H "Content-Type: application/json" \
-d "$PAYLOAD"
-29
View File
@@ -1,29 +0,0 @@
name: Security Audit
on:
schedule:
- cron: "0 0 * * 0" # Weekly
workflow_dispatch:
jobs:
safety:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
- name: Install uv
run: pip install uv
- name: Install dependencies
env:
UV_LINK_MODE: copy
run: uv sync
- name: Run safety check
run: uv run safety check
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
python-version-file: .python-version
- name: Install uv
run: pip install uv
@@ -1,38 +0,0 @@
name: Sync pyproject.toml Version
on:
workflow_run:
workflows: ["Publish Python Package"]
types:
- completed
jobs:
commit-version-update:
if: ${{ github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
- name: Configure git
run: |
git config user.name "CI Bot"
git config user.email "[email protected]"
- name: Update version in pyproject.toml to match tag
run: |
TAG_VERSION="${{ github.event.workflow_run.head_branch }}"
VERSION="${TAG_VERSION#v}"
sed -i.bak -E "s/^version = \".*\"/version = \"${VERSION}\"/" pyproject.toml
rm pyproject.toml.bak
- name: Commit and push changes
run: |
git add pyproject.toml
git commit -m "chore: update version in pyproject.toml to ${VERSION} [skip ci]" || echo "No changes to commit"
git push
+17 -5
View File
@@ -16,7 +16,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
python-version-file: .python-version
- name: Install uv
run: pip install uv
@@ -32,6 +32,7 @@ jobs:
run: uv run pytest --cov=python_repositories --cov-report=term-missing > coverage.txt
- name: Post coverage summary to PR
if: github.event_name == 'pull_request'
env:
API_URL: ${{ vars.API_URL }}
REPO_OWNER: ${{ github.repository_owner }}
@@ -39,9 +40,20 @@ jobs:
PR_NUMBER: ${{ github.event.pull_request.number }}
CI_RUNNER_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
run: |
COVERAGE=$(cat coverage.txt)
COMMENT_BODY="**Test Coverage Report:**\n\`\`\`\n$COVERAGE\n\`\`\`"
curl -s -X POST "$API_URL/repos/$REPO_OWNER/$REPO_NAME/issues/$PR_NUMBER/comments" \
PAYLOAD=$(python3 -c '
import json
import pathlib
coverage = pathlib.Path("coverage.txt").read_text()
print(
json.dumps(
{
"body": f"**Test Coverage Report:**\n```\n{coverage}\n```",
}
)
)
')
curl -sf -X POST "$API_URL/repos/$REPO_OWNER/$REPO_NAME/issues/$PR_NUMBER/comments" \
-H "Authorization: token $CI_RUNNER_TOKEN" \
-H "Content-Type: application/json" \
-d "{\"body\": \"$COMMENT_BODY\"}"
-d "$PAYLOAD"
-1
View File
@@ -173,4 +173,3 @@ cython_debug/
# PyPI configuration file
.pypirc
+3 -1
View File
@@ -24,13 +24,15 @@ repos:
rev: v1.8.0
hooks:
- id: mypy
additional_dependencies:
- types-redis
# Python syntax modernization with pyupgrade
- repo: https://github.com/asottile/pyupgrade
rev: v3.20.0
hooks:
- id: pyupgrade
args: ["--py311-plus"]
args: ["--py312-plus"]
# Formatting for Markdown, JSON, and YAML with Prettier
- repo: https://github.com/pre-commit/mirrors-prettier
+9 -9
View File
@@ -2,17 +2,17 @@ MIT License
Copyright (c) 2025 brian
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
associated documentation files (the "Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
associated documentation files (the "Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the
following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial
The above copyright notice and this permission notice shall be included in all copies or substantial
portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
+142 -8
View File
@@ -1,17 +1,151 @@
# python-repositories
Various python repository interfaces exposed as a python package.
Unified repository interfaces and technology-specific adapters for Python projects.
Subclass an adapter in your own repository to add domain-specific methods while reusing connection management and CRUD operations.
## Architecture
| Layer | Responsibility |
| ---------------- | ----------------------------------------------------------------- |
| **Interfaces** | Abstract contracts for connection, context, and CRUD |
| **Adapters** | Technology-specific base classes (`RedisAdapter`, `MinioAdapter`) |
| **Your project** | Subclass an adapter and add domain methods |
## Optional dependencies
This package supports interacting with multiple different backends:
Repository **interfaces** import with the base package. **Adapters** require the matching extra; importing an adapter without its extra raises `ImportError` with install instructions.
- Redis
- Mongo
- MinIO
To add support for a specific backend install this package with one or more of these optional packages:
Install with the extras you need:
```bash
uv add python-repositories[redis, mongo, minio]
uv add python-repositories[redis]
uv add python-repositories[minio]
uv add python-repositories[redis,minio]
```
### Redis (`JsonRepositoryInterface`)
Requires Redis with the RedisJSON module (e.g. redis-stack).
| Environment variable | Description |
| -------------------- | ---------------------------------------------------- |
| `REDIS_URI` | Redis connection URL (e.g. `redis://localhost:6379`) |
### MinIO (`ObjectRepositoryInterface`)
| Environment variable | Description |
| -------------------- | ------------------------------------------- |
| `MINIO_ENDPOINT` | MinIO server endpoint |
| `MINIO_ACCESS_KEY` | Access key |
| `MINIO_SECRET_KEY` | Secret key |
| `MINIO_BUCKET` | Bucket name (created on connect if missing) |
## Quick start
### JSON documents with Redis
```python
from python_repositories.examples.user_json_repository import UserJsonRepository
with UserJsonRepository() as repo:
repo.save_user("alice", {"name": "Alice", "email": "[email protected]"})
user = repo.get_user("alice")
repo.delete_user("alice")
```
### Binary objects with MinIO
```python
from io import BytesIO
from python_repositories.examples.artifact_object_repository import (
ArtifactObjectRepository,
)
with ArtifactObjectRepository() as repo:
repo.store_artifact("report-1", BytesIO(b"pdf bytes here"))
data = repo.get_artifact("report-1")
```
### Subclassing in your own project
```python
from python_repositories import RedisAdapter
class UserRepository(RedisAdapter):
def _key(self, user_id: str) -> str:
return f"user:{user_id}"
def get_user(self, user_id: str) -> dict | None:
return self.get(self._key(user_id))
def save_user(self, user_id: str, user: dict) -> None:
self.set(self._key(user_id), user)
```
## Public API
```python
from python_repositories import (
ConnectionAwareInterface,
ContextAwareInterface,
JsonRepositoryInterface,
ObjectRepositoryInterface,
RedisAdapter,
MinioAdapter,
)
```
## Development
```bash
uv sync --all-extras
uv run pre-commit install # once per clone — runs hooks on git commit
uv run pytest tests/integration/ -v
```
`pre-commit` is included in the dev dependency group. `uv sync` installs the CLI, but git does not run hooks until you install them with `pre-commit install` (one time per clone). After that, commits run the checks defined in [`.pre-commit-config.yaml`](.pre-commit-config.yaml) (ruff, mypy, pyupgrade, prettier, and general file hygiene).
To run all hooks manually without committing:
```bash
uv run pre-commit run --all-files
```
Integration tests require Docker (testcontainers).
## Releases
Releases are automated when a pull request is merged to `main`. CI reads the **merged PR title** to decide whether and how to bump the version.
### How it works
1. Open a PR targeting `main` (see [`.gitea/PULL_REQUEST_TEMPLATE.md`](.gitea/PULL_REQUEST_TEMPLATE.md)).
2. If the PR changes files under `python_repositories/`, the title **must** start with a version bump prefix (enforced by CI).
3. On merge, [`release.yml`](.gitea/workflows/release.yml) bumps [`pyproject.toml`](pyproject.toml), commits, tags `vX.Y.Z`, creates a Gitea release with auto-generated notes, and pushes the tag.
4. [`publish.yml`](.gitea/workflows/publish.yml) builds and publishes the package to the Gitea Package Registry.
Docs-, CI-, and test-only PRs do not need a prefix and will not trigger a release.
### PR title prefixes
| Prefix | Bump | Example |
| ------------------------- | ---------- | --------------------- |
| `[patch]` or `[fix]` | patch | `0.3.1``0.3.2` |
| `[minor]` or `[feat]` | minor | `0.3.1``0.4.0` |
| `[major]` or `[breaking]` | major | `0.3.1``1.0.0` |
| _(none)_ | no release | docs / CI / deps only |
Example titles:
- `[minor] Add public repository interfaces and subclassable adapter CRUD API`
- `[patch] Fix mypy context manager typing for adapter subclasses`
### Release notes
Release notes are generated from commits since the previous tag (see [`scripts/ci/generate-release-notes.sh`](scripts/ci/generate-release-notes.sh)).
### Manual release
You can still push a `v*.*.*` tag manually; `publish.yml` will build and publish. The current baseline version is **0.3.1**.
+12 -4
View File
@@ -1,6 +1,6 @@
[project]
name = "python-repositories"
version = "0.1.0"
version = "0.3.2"
description = "Various python repository interfaces exposed as a python package."
authors = [
{ name = "Brian Bjarke Jensen", email = "[email protected]" }
@@ -26,21 +26,29 @@ minio = [
"minio>=7.2.16",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["."]
[tool.uv.sources]
python-utils = { index = "gitea" }
[[tool.uv.index]]
name = "threadripper-proxpi-cache"
url = "http://10.0.0.2:5001/index/"
url = "https://proxpi.lille-vemmelund.dk/index/"
default = true
[[tool.uv.index]]
name = "gitea"
url = "https://gitea.gt-proj.com/api/packages/brian/pypi/simple/"
url = "https://gitea.lille-vemmelund.dk/api/packages/brian/pypi/simple/"
explicit = true
[tool.mypy]
python_version = "3.10"
python_version = "3.12"
warn_return_any = true # nudge to use stricter types
warn_unused_configs = true # nudge to remove unused configs
disallow_untyped_defs = true # disallow untyped function definitions
+36 -3
View File
@@ -1,6 +1,39 @@
"""python_repositories: Unified repository interfaces and adapters."""
from . import adapters
from . import interfaces
from __future__ import annotations
__all__ = ["adapters", "interfaces"]
from typing import TYPE_CHECKING
# Interfaces are always available; they have no optional backend dependencies.
from . import adapters
from .interfaces import (
ConnectionAwareInterface,
ContextAwareInterface,
JsonRepositoryInterface,
ObjectRepositoryInterface,
)
# Adapters are imported only for static type checkers; runtime loading is delegated below.
if TYPE_CHECKING:
from .adapters.minio_adapter import MinioAdapter as MinioAdapter
from .adapters.redis_adapter import RedisAdapter as RedisAdapter
__all__ = [
"ConnectionAwareInterface",
"ContextAwareInterface",
"JsonRepositoryInterface",
"ObjectRepositoryInterface",
*adapters.__all__,
]
def __getattr__(name: str) -> object:
"""Delegate adapter lookups to adapters; lazy loading is defined there."""
if name in adapters.__all__:
return getattr(adapters, name)
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
def __dir__() -> list[str]:
"""Expose lazy adapter names in tab completion and dir()."""
return sorted(__all__)
+38 -1
View File
@@ -1,5 +1,42 @@
from .redis_adapter import RedisAdapter as RedisAdapter
"""
Adapters for various backend repositories (e.g., Redis, Minio).
This module exposes concrete implementations for repository interfaces.
"""
from __future__ import annotations
import importlib
from typing import TYPE_CHECKING
# Adapters are imported only for static type checkers; runtime loading is deferred below.
if TYPE_CHECKING:
from .minio_adapter import MinioAdapter
from .redis_adapter import RedisAdapter
# Map public adapter names to their defining module and class.
# Each adapter module fails fast with an install hint if its extra is missing.
# When adding a new adapter, update this dict and __all__ only.
_LAZY_EXPORTS = {
"RedisAdapter": (".redis_adapter", "RedisAdapter"),
"MinioAdapter": (".minio_adapter", "MinioAdapter"),
}
__all__ = [
"RedisAdapter",
"MinioAdapter",
]
def __getattr__(name: str) -> object:
"""Load an adapter on first access so the base package installs without backend clients."""
if name in _LAZY_EXPORTS:
module_path, attr = _LAZY_EXPORTS[name]
module = importlib.import_module(module_path, __package__)
return getattr(module, attr)
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
def __dir__() -> list[str]:
"""Expose lazy adapter names in tab completion and dir()."""
return sorted(__all__)
+33 -17
View File
@@ -3,19 +3,27 @@
from __future__ import annotations
import os
from io import BytesIO
from typing import Self
import structlog
from minio import Minio, S3Error
from python_utils import check_env
from python_repositories.interfaces import (
ContextAwareInterface,
ConnectionAwareInterface,
ContextAwareInterface,
ObjectRepositoryInterface,
)
try:
import minio
except ImportError as exc:
raise ImportError(
"MinIO support requires the minio extra. "
"Install with: pip install python-repositories[minio]"
) from exc
class MinioAdapter(
ObjectRepositoryInterface,
ContextAwareInterface,
ConnectionAwareInterface,
):
@@ -42,10 +50,10 @@ class MinioAdapter(
},
)
# Prepare internal variables
self._client: Minio | None = None
self._client: minio.Minio | None = None
self._bucket_name: str | None = None
def __enter__(self) -> MinioAdapter:
def __enter__(self) -> Self:
"""Enter the context."""
self.connect()
return self
@@ -77,7 +85,7 @@ class MinioAdapter(
secret_key = str(os.getenv(self.secret_key_env_var_name))
bucket = str(os.getenv(self.bucket_env_var_name))
# Connect client
client = Minio(
client = minio.Minio(
endpoint=endpoint,
access_key=access_key,
secret_key=secret_key,
@@ -107,19 +115,26 @@ class MinioAdapter(
@property
def is_connected(self) -> bool:
"""Check if connected to Minio server."""
res = bool(isinstance(self._client, Minio))
res = self._client is not None
self.logger.debug(res)
return res
def _put(self, object_name: str, data: BytesIO) -> None:
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
"""Put an object into the Minio bucket."""
# Check input
if not isinstance(object_name, str) or len(object_name) == 0:
raise ValueError("object_name must be a non-empty string")
if not isinstance(data, BytesIO) or data.getbuffer().nbytes == 0:
raise ValueError("data must be a non-empty BytesIO object")
if not isinstance(content_type, str) or len(content_type) == 0:
raise ValueError("content_type must be a non-empty string")
# Check connection
if self._client is None or not self.is_connected:
if self._client is None or self._bucket_name is None or not self.is_connected:
raise ConnectionError("Not connected to Minio")
# Prepare buffer for reading
num_bytes = data.getbuffer().nbytes
@@ -132,18 +147,19 @@ class MinioAdapter(
data=data,
length=num_bytes,
part_size=self.chunk_size,
content_type=content_type,
)
self.logger.debug(
f"Put object '{object_name}' into bucket '{self._bucket_name}'"
)
def _get(self, object_name: str) -> BytesIO | None:
def get(self, object_name: str) -> BytesIO | None:
"""Get an object from the Minio bucket."""
# Check input
if not isinstance(object_name, str) or len(object_name) == 0:
raise ValueError("object_name must be a non-empty string")
# Check connection
if self._client is None or not self.is_connected:
if self._client is None or self._bucket_name is None or not self.is_connected:
raise ConnectionError("Not connected to Minio")
# Get data from bucket
# N.B. bucket name is set when connecting
@@ -161,7 +177,7 @@ class MinioAdapter(
f"Got object '{object_name}' from bucket '{self._bucket_name}'"
)
return buffer
except S3Error as exc:
except minio.S3Error as exc:
if exc.code == "NoSuchKey":
self.logger.warning(
f"Object '{object_name}' not found in bucket '{self._bucket_name}'"
@@ -172,13 +188,13 @@ class MinioAdapter(
self.logger.error(repr(exc))
return None
def _delete(self, object_name: str) -> None:
def delete(self, object_name: str) -> None:
"""Delete an object from the Minio bucket."""
# Check input
if not isinstance(object_name, str) or len(object_name) == 0:
raise ValueError("object_name must be a non-empty string")
# Check connection
if self._client is None or not self.is_connected:
if self._client is None or self._bucket_name is None or not self.is_connected:
raise ConnectionError("Not connected to Minio")
# Delete object from bucket
# N.B. bucket name is set when connecting
@@ -190,14 +206,14 @@ class MinioAdapter(
f"Deleted object '{object_name}' from bucket '{self._bucket_name}'"
)
def _list_objects(self, prefix: str = "") -> list[str]:
def list_objects(self, prefix: str = "") -> list[str]:
"""List objects in the Minio bucket with an optional prefix."""
# Check input
if not isinstance(prefix, str):
raise ValueError("prefix must be a string")
# Check connection
# N.B. bucket name is set when connecting
if self._client is None or not self.is_connected:
if self._client is None or self._bucket_name is None or not self.is_connected:
raise ConnectionError("Not connected to Minio")
# List objects in bucket
objects = self._client.list_objects(
+21 -13
View File
@@ -1,30 +1,37 @@
"""Definition of RedisAdapter class."""
from __future__ import annotations
from typing import cast
from typing import Self, cast
import os
import structlog
import redis
from redis.commands.json.path import Path as RedisPath
from python_utils import check_env
from python_repositories.interfaces import (
ContextAwareInterface,
ConnectionAwareInterface,
ContextAwareInterface,
JsonRepositoryInterface,
)
try:
import redis
from redis.commands.json.path import Path as RedisPath
except ImportError as exc:
raise ImportError(
"Redis support requires the redis extra. "
"Install with: pip install python-repositories[redis]"
) from exc
class RedisAdapter(
JsonRepositoryInterface,
ContextAwareInterface,
ConnectionAwareInterface,
):
"""Redis adapter exposing basic CRUD functionality."""
uri_env_var_name: str = "REDIS_URI"
path: str = RedisPath.root_path()
path: str = "." # JSON root path, updated in __init__
encoding: str = "UTF-8"
def __init__(self) -> None:
@@ -36,8 +43,9 @@ class RedisAdapter(
check_env(self.uri_env_var_name)
# Prepare internal variables
self._client: redis.Redis | None = None
self.path: str = RedisPath.root_path()
def __enter__(self) -> RedisAdapter:
def __enter__(self) -> Self:
"""Enter the context."""
self.connect()
return self
@@ -85,11 +93,11 @@ class RedisAdapter(
@property
def is_connected(self) -> bool:
"""Check if connected to Redis server."""
res = bool(isinstance(self._client, redis.Redis))
res = self._client is not None
self.logger.debug(res)
return res
def _set(self, key: str, data: dict) -> None:
def set(self, key: str, data: dict) -> None:
"""Set a JSON object in Redis."""
# Check input
if not isinstance(key, str) or len(key) == 0:
@@ -103,7 +111,7 @@ class RedisAdapter(
self._client.json().set(key, self.path, data)
self.logger.debug(f"Set {key} to {data}")
def _get(self, key: str) -> dict | None:
def get(self, key: str) -> dict | None:
"""Get a JSON object from Redis."""
# Check input
if not isinstance(key, str) or len(key) == 0:
@@ -119,7 +127,7 @@ class RedisAdapter(
self.logger.debug(f"Got {data} from {key}")
return data
def _delete(self, key: str) -> None:
def delete(self, key: str) -> None:
"""Delete data from Redis."""
# Check input
if not isinstance(key, str) or len(key) == 0:
@@ -131,7 +139,7 @@ class RedisAdapter(
self._client.json().delete(key)
self.logger.debug(f"Deleted {key}")
def _list_keys(self, pattern: str) -> list[str]:
def list_keys(self, pattern: str) -> list[str]:
"""List keys in Redis matching a pattern."""
# Check input
if not isinstance(pattern, str) or len(pattern) == 0:
+1
View File
@@ -0,0 +1 @@
"""Example domain repositories built on technology adapters."""
@@ -0,0 +1,23 @@
"""Example domain repository backed by MinIO objects."""
from io import BytesIO
from python_repositories.adapters.minio_adapter import MinioAdapter
class ArtifactObjectRepository(MinioAdapter):
"""Example: domain repository backed by MinIO objects."""
def _object_name(self, artifact_id: str) -> str:
return f"artifacts/{artifact_id}"
def get_artifact(self, artifact_id: str) -> BytesIO | None:
return self.get(self._object_name(artifact_id))
def store_artifact(
self,
artifact_id: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
self.put(self._object_name(artifact_id), data, content_type)
@@ -0,0 +1,19 @@
"""Example domain repository backed by Redis JSON."""
from python_repositories.adapters.redis_adapter import RedisAdapter
class UserJsonRepository(RedisAdapter):
"""Example: domain repository backed by Redis JSON."""
def _key(self, user_id: str) -> str:
return f"user:{user_id}"
def get_user(self, user_id: str) -> dict | None:
return self.get(self._key(user_id))
def save_user(self, user_id: str, user: dict) -> None:
self.set(self._key(user_id), user)
def delete_user(self, user_id: str) -> None:
self.delete(self._key(user_id))
@@ -2,8 +2,16 @@ from .connection_aware_interface import (
ConnectionAwareInterface as ConnectionAwareInterface,
)
from .context_aware_interface import ContextAwareInterface as ContextAwareInterface
from .json_repository_interface import (
JsonRepositoryInterface as JsonRepositoryInterface,
)
from .object_repository_interface import (
ObjectRepositoryInterface as ObjectRepositoryInterface,
)
__all__ = [
"ConnectionAwareInterface",
"ContextAwareInterface",
"JsonRepositoryInterface",
"ObjectRepositoryInterface",
]
@@ -9,15 +9,15 @@ class ConnectionAwareInterface(ABC):
@abstractmethod
def connect(self) -> None:
"""Connect to resource."""
raise NotImplementedError()
...
@abstractmethod
def disconnect(self) -> None:
"""Disconnect from resource."""
raise NotImplementedError()
...
@property
@abstractmethod
def is_connected(self) -> bool:
"""Check if connected to resource."""
raise NotImplementedError()
...
@@ -1,21 +1,22 @@
"""Definition of ConnectionAwareInterface abstract base class."""
"""Definition of ContextAwareInterface abstract base class."""
from __future__ import annotations
from abc import ABC, abstractmethod
from typing import Self
class ContextAwareInterface(ABC):
"""Interface that defined context-related methods."""
"""Interface that defines context-related methods."""
@abstractmethod
def __enter__(self) -> ContextAwareInterface:
def __enter__(self) -> Self:
"""Enter the context."""
raise NotImplementedError()
...
@abstractmethod
def __exit__(
self, exc_type: type | None, exc_val: object | None, exc_tb: object | None
) -> None:
"""Exit the context."""
raise NotImplementedError()
...
@@ -0,0 +1,27 @@
"""Definition of JsonRepositoryInterface abstract base class."""
from abc import ABC, abstractmethod
class JsonRepositoryInterface(ABC):
"""Interface that defines JSON document CRUD methods."""
@abstractmethod
def get(self, key: str) -> dict | None:
"""Get a JSON object by key."""
...
@abstractmethod
def set(self, key: str, data: dict) -> None:
"""Set a JSON object by key."""
...
@abstractmethod
def delete(self, key: str) -> None:
"""Delete a JSON object by key."""
...
@abstractmethod
def list_keys(self, pattern: str) -> list[str]:
"""List keys matching a glob pattern."""
...
@@ -0,0 +1,33 @@
"""Definition of ObjectRepositoryInterface abstract base class."""
from abc import ABC, abstractmethod
from io import BytesIO
class ObjectRepositoryInterface(ABC):
"""Interface that defines binary object CRUD methods."""
@abstractmethod
def get(self, object_name: str) -> BytesIO | None:
"""Get an object by name."""
...
@abstractmethod
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
"""Put an object by name."""
...
@abstractmethod
def delete(self, object_name: str) -> None:
"""Delete an object by name."""
...
@abstractmethod
def list_objects(self, prefix: str = "") -> list[str]:
"""List object names with an optional prefix."""
...
View File
+37
View File
@@ -0,0 +1,37 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
usage() {
echo "Usage: bump-version.sh <major|minor|patch>" >&2
echo " bump-version.sh --from-tag <vX.Y.Z>" >&2
exit 1
}
if [[ $# -lt 1 ]]; then
usage
fi
if [[ "$1" == "--from-tag" ]]; then
if [[ $# -ne 2 ]]; then
usage
fi
VERSION="${2#v}"
else
BUMP_TYPE="$1"
CURRENT=$(latest_tag_version)
VERSION=$(bump_semver "$CURRENT" "$BUMP_TYPE")
fi
set_pyproject_version "$VERSION"
if [[ -n "${GITHUB_OUTPUT:-}" ]]; then
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
else
echo "version=${VERSION}"
fi
echo "Updated pyproject.toml to version ${VERSION}" >&2
+41
View File
@@ -0,0 +1,41 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
PR_TITLE="${1:-${PR_TITLE:-}}"
if [[ -z "$PR_TITLE" ]]; then
echo "Usage: check-pr-title.sh <pr-title> (changed files on stdin)" >&2
exit 1
fi
CHANGED_FILES=$(cat)
if ! has_source_changes "$CHANGED_FILES"; then
echo "No changes under ${SOURCE_DIR}/ — PR title prefix not required."
exit 0
fi
if title_has_bump_prefix "$PR_TITLE"; then
echo "PR title has a valid version bump prefix."
exit 0
fi
cat >&2 <<EOF
PR changes files under ${SOURCE_DIR}/ but the title lacks a version bump prefix.
Title: ${PR_TITLE}
When changing package source, start the PR title with one of:
[patch] or [fix] — bug fix
[minor] or [feat] — new feature
[major] or [breaking] — breaking change
Example: [patch] Fix connection retry in RedisAdapter
See .gitea/PULL_REQUEST_TEMPLATE.md for details.
EOF
exit 1
+89
View File
@@ -0,0 +1,89 @@
#!/usr/bin/env bash
# Shared helpers for release and PR title CI scripts.
SOURCE_DIR="python_repositories"
# Returns bump type (major|minor|patch) or empty if no valid prefix.
bump_type_from_title() {
local title="$1"
local lower
lower=$(echo "$title" | tr '[:upper:]' '[:lower:]')
if echo "$lower" | grep -qE '^\[(major|breaking)\]'; then
echo "major"
elif echo "$lower" | grep -qE '^\[(minor|feat)\]'; then
echo "minor"
elif echo "$lower" | grep -qE '^\[(patch|fix)\]'; then
echo "patch"
fi
}
title_has_bump_prefix() {
[[ -n "$(bump_type_from_title "$1")" ]]
}
strip_bump_prefix() {
local title="$1"
echo "$title" | sed -E 's/^\[(patch|fix|minor|feat|major|breaking)\][[:space:]]*//i'
}
extract_pr_title_from_merge_commit() {
local msg="$1"
echo "$msg" | sed -n "s/^Merge pull request '\(.*\)' (#.*/\1/p"
}
changed_source_files() {
grep -E "^${SOURCE_DIR}/" || true
}
has_source_changes() {
local files="$1"
echo "$files" | changed_source_files | grep -q .
}
read_pyproject_version() {
sed -n 's/^version = "\(.*\)"/\1/p' pyproject.toml | head -1
}
latest_tag_version() {
local tag
tag=$(git describe --tags --abbrev=0 2>/dev/null || true)
if [[ -n "$tag" ]]; then
echo "${tag#v}"
else
read_pyproject_version
fi
}
set_pyproject_version() {
local version="$1"
sed -i.bak -E "s/^version = \".*\"/version = \"${version}\"/" pyproject.toml
rm -f pyproject.toml.bak
}
bump_semver() {
local current="$1"
local bump_type="$2"
local major minor patch
IFS=. read -r major minor patch <<< "$current"
case "$bump_type" in
major)
major=$((major + 1))
minor=0
patch=0
;;
minor)
minor=$((minor + 1))
patch=0
;;
patch)
patch=$((patch + 1))
;;
*)
echo "Unknown bump type: $bump_type" >&2
return 1
;;
esac
echo "${major}.${minor}.${patch}"
}
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env bash
set -euo pipefail
: "${API_URL:?API_URL is required}"
: "${REPO_OWNER:?REPO_OWNER is required}"
: "${REPO_NAME:?REPO_NAME is required}"
: "${CI_RUNNER_TOKEN:?CI_RUNNER_TOKEN is required}"
: "${GITHUB_SERVER_URL:?GITHUB_SERVER_URL is required}"
: "${GITHUB_REPOSITORY:?GITHUB_REPOSITORY is required}"
BRANCH="${BRANCH:-renovate/auto-deps-update}"
BASE_BRANCH="${BASE_BRANCH:-main}"
LOCKFILE="${LOCKFILE:-uv.lock}"
if git diff --quiet "$LOCKFILE"; then
echo "No dependency updates available."
exit 0
fi
git config user.name "CI Bot"
git config user.email "[email protected]"
git add "$LOCKFILE"
git commit -m "chore(deps): update dependencies [automated]"
git remote set-url origin "https://x-access-token:${CI_RUNNER_TOKEN}@${GITHUB_SERVER_URL#https://}/${GITHUB_REPOSITORY}.git"
git push origin "HEAD:${BRANCH}"
if curl -s "${API_URL}/repos/${REPO_OWNER}/${REPO_NAME}/pulls?state=open" \
-H "Authorization: token ${CI_RUNNER_TOKEN}" \
| BRANCH="$BRANCH" python3 -c '
import json
import os
import sys
branch = os.environ["BRANCH"]
prs = json.load(sys.stdin)
sys.exit(0 if any(pr.get("head", {}).get("ref") == branch for pr in prs) else 1)
'; then
echo "Pull request already exists; branch push updates it."
exit 0
fi
PAYLOAD=$(BRANCH="$BRANCH" BASE_BRANCH="$BASE_BRANCH" python3 -c '
import json
import os
print(
json.dumps(
{
"title": "chore(deps): update dependencies",
"body": "This PR was created automatically to update dependencies.",
"head": os.environ["BRANCH"],
"base": os.environ["BASE_BRANCH"],
}
)
)
')
curl -sf -X POST "${API_URL}/repos/${REPO_OWNER}/${REPO_NAME}/pulls" \
-H "Authorization: token ${CI_RUNNER_TOKEN}" \
-H "Content-Type: application/json" \
-d "$PAYLOAD"
echo "Created pull request."
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
NEW_TAG="${1:-}"
SUMMARY="${2:-}"
PREV_TAG="${3:-}"
if [[ -z "$NEW_TAG" ]]; then
echo "Usage: generate-release-notes.sh <new-tag> [summary] [prev-tag]" >&2
exit 1
fi
if [[ -z "$PREV_TAG" ]]; then
PREV_TAG=$(git describe --tags --abbrev=0 "${NEW_TAG}^" 2>/dev/null || git describe --tags --abbrev=0 2>/dev/null || true)
fi
NOTES="## Summary"
if [[ -n "$SUMMARY" ]]; then
NOTES="${NOTES}
$(strip_bump_prefix "$SUMMARY")"
else
NOTES="${NOTES}
Automated release ${NEW_TAG}."
fi
if [[ -n "$PREV_TAG" ]]; then
NOTES="${NOTES}
## Changes since ${PREV_TAG}
$(git log "${PREV_TAG}..HEAD" --pretty=format:'- %h %s' || true)"
else
NOTES="${NOTES}
## Changes
$(git log --pretty=format:'- %h %s' -20 || true)"
fi
if [[ -n "${GITHUB_OUTPUT:-}" ]]; then
{
echo 'body<<EOF'
echo "$NOTES"
echo 'EOF'
} >> "$GITHUB_OUTPUT"
else
echo "$NOTES"
fi
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
COMMIT_MSG="${1:-${COMMIT_MSG:-}}"
if [[ -z "$COMMIT_MSG" ]]; then
echo "Usage: parse-merge-commit.sh <commit-message>" >&2
exit 1
fi
PR_TITLE=$(extract_pr_title_from_merge_commit "$COMMIT_MSG")
if [[ -z "$PR_TITLE" ]]; then
bump="skip"
else
bump_type=$(bump_type_from_title "$PR_TITLE")
if [[ -n "$bump_type" ]]; then
bump="$bump_type"
else
bump="skip"
fi
fi
if [[ -n "${GITHUB_OUTPUT:-}" ]]; then
{
echo "pr_title=${PR_TITLE}"
echo "bump=${bump}"
} >> "$GITHUB_OUTPUT"
else
echo "pr_title=${PR_TITLE}"
echo "bump=${bump}"
fi
+13 -10
View File
@@ -1,16 +1,19 @@
"""Integration tests configuration."""
import logging
import os
from collections.abc import Generator
import pytest
import redis
import structlog
import logging
from minio import Minio
from testcontainers.redis import RedisContainer
from testcontainers.minio import MinioContainer
from tests.integration.redis_container_test import REDIS_PORT, RedisTestContainer
collect_ignore = ["redis_container_test.py"]
MINIO_ACCESS_KEY = "minioadmin"
MINIO_SECRET_KEY = "minioadmin"
MINIO_BUCKET = "test-bucket"
@@ -37,16 +40,16 @@ def configure_logging() -> None:
@pytest.fixture(scope="session")
def redis_container() -> Generator[str]:
def redis_container() -> Generator[str, None, None]:
"""Set up a Redis container for testing and yield the Redis URI."""
# Start container
container = RedisContainer(
container = RedisTestContainer(
image="redis/redis-stack:7.2.0-v0",
)
container.start()
# Set environment variable for Redis URI
redis_host = container.get_container_host_ip()
redis_port = container.get_exposed_port(6379)
redis_port = container.get_exposed_port(REDIS_PORT)
redis_uri = f"redis://{redis_host}:{redis_port}"
yield redis_uri
@@ -56,7 +59,7 @@ def redis_container() -> Generator[str]:
@pytest.fixture(scope="session")
def minio_container() -> Generator[dict[str, str]]:
def minio_container() -> Generator[dict[str, str], None, None]:
"""Set up a Minio container for testing and yield the Minio URI."""
# Start container
container = MinioContainer(
@@ -86,7 +89,7 @@ def minio_container() -> Generator[dict[str, str]]:
def set_environment_variables(
redis_container: str,
minio_container: dict[str, str],
) -> Generator[dict[str, str]]:
) -> Generator[dict[str, str], None, None]:
"""Set environment variables needed for tests."""
# Build environment variables dictionary
env_vars = {"REDIS_URI": redis_container}
@@ -103,7 +106,7 @@ def set_environment_variables(
@pytest.fixture(scope="session")
def raw_redis_client(redis_container: str) -> Generator[redis.Redis]:
def raw_redis_client(redis_container: str) -> Generator[redis.Redis, None, None]:
"""Provide a raw Redis client connected to the test Redis container."""
# Connect client
client = redis.Redis.from_url(
@@ -119,7 +122,7 @@ def raw_redis_client(redis_container: str) -> Generator[redis.Redis]:
@pytest.fixture(scope="session")
def raw_minio_client(minio_container: dict[str, str]) -> Generator[Minio]:
def raw_minio_client(minio_container: dict[str, str]) -> Generator[Minio, None, None]:
"""Provide a raw Minio client connected to the test Minio container."""
# Connect client
client = Minio(
@@ -54,66 +54,3 @@ def test_instantiation_fails_when_is_connected_not_implemented() -> None:
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_connect_raises_not_implemented_if_not_overwritten() -> None:
"""Test that connect raises NotImplementedError if not implemented."""
class Incomplete(ConnectionAwareInterface):
"""A class that does not implement connect."""
def connect(self) -> None:
super().connect() # type: ignore
def disconnect(self) -> None:
pass
@property
def is_connected(self) -> bool:
return False
instance = Incomplete()
with pytest.raises(NotImplementedError):
instance.connect()
def test_disconnect_raises_not_implemented_if_not_overwritten() -> None:
"""Test that disconnect raises NotImplementedError if not implemented."""
class Incomplete(ConnectionAwareInterface):
"""A class that does not implement disconnect."""
def connect(self) -> None:
pass
def disconnect(self) -> None:
super().disconnect() # type: ignore
@property
def is_connected(self) -> bool:
return False
instance = Incomplete()
with pytest.raises(NotImplementedError):
instance.disconnect()
def test_is_connected_raises_not_implemented_if_not_overwritten() -> None:
"""Test that is_connected raises NotImplementedError if not implemented."""
class Incomplete(ConnectionAwareInterface):
"""A class that does not implement is_connected."""
def connect(self) -> None:
pass
def disconnect(self) -> None:
pass
@property
def is_connected(self) -> bool:
return super().is_connected # type: ignore
instance = Incomplete()
with pytest.raises(NotImplementedError):
_ = instance.is_connected
@@ -1,6 +1,7 @@
"""Integration tests for ContextAwareInterface."""
from __future__ import annotations
import pytest
from python_repositories.interfaces.context_aware_interface import ContextAwareInterface
@@ -31,42 +32,3 @@ def test_instantiation_fails_when_exit_not_implemented() -> None:
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_enter_raises_not_implemented_if_not_overwritten() -> None:
"""Test that __enter__ raises NotImplementedError if not implemented."""
class Incomplete(ContextAwareInterface):
"""A class that does not implement __enter__."""
def __enter__(self) -> Incomplete:
super().__enter__() # type: ignore
return self
def __exit__(
self, exc_type: type | None, exc_val: object | None, exc_tb: object | None
) -> None:
pass
instance = Incomplete()
with pytest.raises(NotImplementedError):
instance.__enter__()
def test_exit_raises_not_implemented_if_not_overwritten() -> None:
"""Test that __exit__ raises NotImplementedError if not implemented."""
class Incomplete(ContextAwareInterface):
"""A class that does not implement __exit__."""
def __enter__(self) -> ContextAwareInterface:
return self
def __exit__(
self, exc_type: type | None, exc_val: object | None, exc_tb: object | None
) -> None:
super().__exit__(exc_type, exc_val, exc_tb) # type: ignore
instance = Incomplete()
with pytest.raises(NotImplementedError):
instance.__exit__(None, None, None)
+57
View File
@@ -0,0 +1,57 @@
"""Integration tests for example domain repositories."""
from collections.abc import Generator
from io import BytesIO
import random
import pytest
from python_repositories.examples.artifact_object_repository import (
ArtifactObjectRepository,
)
from python_repositories.examples.user_json_repository import UserJsonRepository
@pytest.fixture(scope="module")
def user_data() -> Generator[dict[str, str]]:
"""Provide sample user data for tests."""
yield {"name": "Alice", "email": "[email protected]"}
@pytest.fixture(scope="module")
def artifact_data() -> Generator[BytesIO]:
"""Provide sample artifact data for tests."""
yield BytesIO(random.randbytes(2**20))
def test_user_json_repository_save_and_get(
redis_container: str,
user_data: dict[str, str],
) -> None:
"""Test that UserJsonRepository can save and retrieve a user."""
with UserJsonRepository() as repo:
repo.save_user("alice", user_data)
assert repo.get_user("alice") == user_data
def test_user_json_repository_delete(
redis_container: str,
user_data: dict[str, str],
) -> None:
"""Test that UserJsonRepository can delete a user."""
with UserJsonRepository() as repo:
repo.save_user("alice", user_data)
repo.delete_user("alice")
assert repo.get_user("alice") is None
def test_artifact_object_repository_store_and_get(
minio_container: dict[str, str],
artifact_data: BytesIO,
) -> None:
"""Test that ArtifactObjectRepository can store and retrieve an artifact."""
with ArtifactObjectRepository() as repo:
repo.store_artifact("report-1", artifact_data)
received = repo.get_artifact("report-1")
assert received is not None
artifact_data.seek(0)
assert received.read() == artifact_data.read()
@@ -0,0 +1,82 @@
"""Integration tests for JsonRepositoryInterface."""
import pytest
from python_repositories.interfaces.json_repository_interface import (
JsonRepositoryInterface,
)
def test_instantiation_fails_when_get_not_implemented() -> None:
"""Test that instantiation fails if get is not implemented."""
class Incomplete(JsonRepositoryInterface):
"""A class that does not implement get."""
def set(self, key: str, data: dict) -> None:
pass
def delete(self, key: str) -> None:
pass
def list_keys(self, pattern: str) -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_set_not_implemented() -> None:
"""Test that instantiation fails if set is not implemented."""
class Incomplete(JsonRepositoryInterface):
"""A class that does not implement set."""
def get(self, key: str) -> dict | None:
return None
def delete(self, key: str) -> None:
pass
def list_keys(self, pattern: str) -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_delete_not_implemented() -> None:
"""Test that instantiation fails if delete is not implemented."""
class Incomplete(JsonRepositoryInterface):
"""A class that does not implement delete."""
def get(self, key: str) -> dict | None:
return None
def set(self, key: str, data: dict) -> None:
pass
def list_keys(self, pattern: str) -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_list_keys_not_implemented() -> None:
"""Test that instantiation fails if list_keys is not implemented."""
class Incomplete(JsonRepositoryInterface):
"""A class that does not implement list_keys."""
def get(self, key: str) -> dict | None:
return None
def set(self, key: str, data: dict) -> None:
pass
def delete(self, key: str) -> None:
pass
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
+59 -46
View File
@@ -1,6 +1,3 @@
# pylint: disable=protected-access
# The above line disables pylint's protected member access warnings for this file,
# allowing tests to access MinioAdapter's internal methods as needed for integration testing.
"""Integration tests for the MinioAdapter."""
from collections.abc import Generator
@@ -12,7 +9,9 @@ import random
import os
import logging
from minio import S3Error
from urllib3.response import BaseHTTPResponse
from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.interfaces import ObjectRepositoryInterface
def same_data(
@@ -44,7 +43,7 @@ def same_data(
@pytest.fixture(scope="module")
def data() -> Generator[BytesIO]:
def data() -> Generator[BytesIO, None, None]:
"""Provide a sample data bytes for tests."""
# Generate random bytes
random_bytes = random.randbytes(2**21) # 2 MiB
@@ -55,7 +54,7 @@ def data() -> Generator[BytesIO]:
def data_in_minio(
raw_minio_client: Minio,
data: BytesIO,
) -> Generator[tuple[str, BytesIO]]:
) -> Generator[tuple[str, BytesIO], None, None]:
"""Fixture to set up a known value in Minio before each test."""
object_name = "test_object"
bucket_name = str(os.getenv("MINIO_BUCKET"))
@@ -79,7 +78,7 @@ def data_in_minio(
@pytest.fixture(scope="module")
def minio_adapter() -> Generator[MinioAdapter]:
def minio_adapter() -> Generator[MinioAdapter, None, None]:
"""Fixture to provide a connected MinioAdapter instance."""
adapter = MinioAdapter()
adapter.connect()
@@ -103,7 +102,7 @@ def clear_minio(
def test_should_adhere_to_interface() -> None:
"""Test that the MinioAdapter adheres to the expected interface."""
# Instantiation fails if interface not adhered to
assert issubclass(MinioAdapter, ObjectRepositoryInterface)
_ = MinioAdapter()
@@ -187,7 +186,7 @@ def test_should_get_data(
# Arrange
object_name, expected_data = data_in_minio
# Act
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
# Assert
assert received_data is not None
assert same_data(expected_data, received_data)
@@ -200,7 +199,7 @@ def test_should_get_none_for_nonexistent_object(
# Arrange
object_name = "nonexistent_object"
# Act
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
# Assert
assert received_data is None
@@ -214,7 +213,7 @@ def test_should_raise_value_error_on_invalid_get_object_name(
# Act & Assert
for object_name in invalid_object_names:
with pytest.raises(ValueError):
minio_adapter._get(object_name) # type: ignore
minio_adapter.get(object_name) # type: ignore
def test_should_raise_connection_error_on_get_when_not_connected(
@@ -225,7 +224,7 @@ def test_should_raise_connection_error_on_get_when_not_connected(
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError):
adapter._get("some_object")
adapter.get("some_object")
def test_should_log_warning_when_getting_nonexistent_object(
@@ -236,12 +235,12 @@ def test_should_log_warning_when_getting_nonexistent_object(
adapter = MinioAdapter()
adapter._client = MagicMock(spec=Minio)
adapter._client.get_object.side_effect = S3Error(
code="NoSuchKey",
message="",
resource="",
request_id="",
host_id="",
response="",
MagicMock(spec=BaseHTTPResponse),
"NoSuchKey",
"",
"",
"",
"",
bucket_name="test-bucket",
object_name="missing-object",
)
@@ -249,7 +248,7 @@ def test_should_log_warning_when_getting_nonexistent_object(
object_name = "missing-object"
# Act
with caplog.at_level("WARNING"):
result = adapter._get(object_name)
result = adapter.get(object_name)
# Assert
assert result is None
assert (
@@ -266,12 +265,12 @@ def test_should_log_error_when_getting_with_s3error_other_than_no_such_key(
adapter = MinioAdapter()
adapter._client = MagicMock(spec=Minio)
other_s3error = S3Error(
code="UnhandledError",
message="",
resource="",
request_id="",
host_id="",
response="",
MagicMock(spec=BaseHTTPResponse),
"UnhandledError",
"",
"",
"",
"",
bucket_name="test-bucket",
object_name="missing-object",
)
@@ -280,7 +279,7 @@ def test_should_log_error_when_getting_with_s3error_other_than_no_such_key(
object_name = "missing-object"
# Act
with caplog.at_level("ERROR"):
result = adapter._get(object_name)
result = adapter.get(object_name)
# Assert
assert result is None
assert repr(other_s3error) in caplog.text
@@ -299,7 +298,7 @@ def test_should_log_error_when_getting_with_general_exception(
object_name = "missing-object"
# Act
with caplog.at_level("ERROR"):
result = adapter._get(object_name)
result = adapter.get(object_name)
# Assert
assert result is None
assert repr(general_exception) in caplog.text
@@ -312,12 +311,12 @@ def test_should_put_data(
"""Test that the MinioAdapter can put data into a bucket."""
# Arrange
object_name = "new_test_object"
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is None # ensure object does not exist yet
# Act
minio_adapter._put(object_name, data)
minio_adapter.put(object_name, data)
# Assert
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is not None
assert same_data(data, received_data)
# Cleanup
@@ -333,13 +332,13 @@ def test_should_update_data(
# Arrange
object_name, _ = data_in_minio
new_data = BytesIO(random.randbytes(2**21)) # 2 MiB
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is not None
assert not same_data(received_data, new_data)
# Act
minio_adapter._put(object_name, new_data)
minio_adapter.put(object_name, new_data)
# Assert
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is not None
assert same_data(new_data, received_data)
@@ -354,7 +353,7 @@ def test_should_raise_value_error_on_invalid_put_object_name(
# Act & Assert
for object_name in invalid_object_names:
with pytest.raises(ValueError):
minio_adapter._put(object_name, data) # type: ignore
minio_adapter.put(object_name, data) # type: ignore
def test_should_raise_value_error_on_invalid_put_data(
@@ -367,7 +366,21 @@ def test_should_raise_value_error_on_invalid_put_data(
# Act & Assert
for data in invalid_data:
with pytest.raises(ValueError):
minio_adapter._put(object_name, data) # type: ignore
minio_adapter.put(object_name, data) # type: ignore
def test_should_raise_value_error_on_invalid_put_content_type(
data: BytesIO,
minio_adapter: MinioAdapter,
) -> None:
"""Test that the MinioAdapter raises ValueError when putting with an invalid content type."""
# Arrange
object_name = "valid_object_name"
invalid_content_types = ["", 123, None]
# Act & Assert
for content_type in invalid_content_types:
with pytest.raises(ValueError):
minio_adapter.put(object_name, data, content_type) # type: ignore
def test_should_raise_connection_error_on_put_when_not_connected(
@@ -379,7 +392,7 @@ def test_should_raise_connection_error_on_put_when_not_connected(
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError):
adapter._put("some_object", data)
adapter.put("some_object", data)
def test_should_delete_object(
@@ -389,12 +402,12 @@ def test_should_delete_object(
"""Test that the MinioAdapter can delete an object from a bucket."""
# Arrange
object_name, _ = data_in_minio
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is not None # ensure object exists
# Act
minio_adapter._delete(object_name)
minio_adapter.delete(object_name)
# Assert
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is None
@@ -407,7 +420,7 @@ def test_should_raise_value_error_on_invalid_delete_object_name(
# Act & Assert
for object_name in invalid_object_names:
with pytest.raises(ValueError):
minio_adapter._delete(object_name) # type: ignore
minio_adapter.delete(object_name) # type: ignore
def test_should_raise_connection_error_on_delete_when_not_connected(
@@ -418,7 +431,7 @@ def test_should_raise_connection_error_on_delete_when_not_connected(
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError):
adapter._delete("some_object")
adapter.delete("some_object")
def test_should_list_objects(
@@ -430,9 +443,9 @@ def test_should_list_objects(
object_name, _ = data_in_minio
new_data = BytesIO(random.randbytes(2**21)) # 2 MiB
new_data_name = "another_test_object"
minio_adapter._put(new_data_name, new_data)
minio_adapter.put(new_data_name, new_data)
# Act
objects = minio_adapter._list_objects()
objects = minio_adapter.list_objects()
# Assert
assert isinstance(objects, list)
assert len(objects) == 2
@@ -449,10 +462,10 @@ def test_should_list_objects_with_prefix(
object_name, _ = data_in_minio
new_data = BytesIO(random.randbytes(2**21)) # 2 MiB
new_data_name = "prefix_test_object"
minio_adapter._put(new_data_name, new_data)
minio_adapter.put(new_data_name, new_data)
prefix = "prefix_"
# Act
objects = minio_adapter._list_objects(prefix)
objects = minio_adapter.list_objects(prefix)
# Assert
assert isinstance(objects, list)
assert len(objects) == 1
@@ -469,7 +482,7 @@ def test_should_raise_value_error_on_invalid_list_objects_prefix(
# Act & Assert
for prefix in invalid_prefixes:
with pytest.raises(ValueError):
minio_adapter._list_objects(prefix) # type: ignore
minio_adapter.list_objects(prefix) # type: ignore
def test_should_raise_connection_error_on_list_objects_when_not_connected(
@@ -480,7 +493,7 @@ def test_should_raise_connection_error_on_list_objects_when_not_connected(
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError):
adapter._list_objects()
adapter.list_objects()
# allows local debugging by running file as script
@@ -0,0 +1,99 @@
"""Integration tests for ObjectRepositoryInterface."""
from io import BytesIO
import pytest
from python_repositories.interfaces.object_repository_interface import (
ObjectRepositoryInterface,
)
def test_instantiation_fails_when_get_not_implemented() -> None:
"""Test that instantiation fails if get is not implemented."""
class Incomplete(ObjectRepositoryInterface):
"""A class that does not implement get."""
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
pass
def delete(self, object_name: str) -> None:
pass
def list_objects(self, prefix: str = "") -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_put_not_implemented() -> None:
"""Test that instantiation fails if put is not implemented."""
class Incomplete(ObjectRepositoryInterface):
"""A class that does not implement put."""
def get(self, object_name: str) -> BytesIO | None:
return None
def delete(self, object_name: str) -> None:
pass
def list_objects(self, prefix: str = "") -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_delete_not_implemented() -> None:
"""Test that instantiation fails if delete is not implemented."""
class Incomplete(ObjectRepositoryInterface):
"""A class that does not implement delete."""
def get(self, object_name: str) -> BytesIO | None:
return None
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
pass
def list_objects(self, prefix: str = "") -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_list_objects_not_implemented() -> None:
"""Test that instantiation fails if list_objects is not implemented."""
class Incomplete(ObjectRepositoryInterface):
"""A class that does not implement list_objects."""
def get(self, object_name: str) -> BytesIO | None:
return None
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
pass
def delete(self, object_name: str) -> None:
pass
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
+25 -27
View File
@@ -1,6 +1,3 @@
# pylint: disable=protected-access
# The above line disables pylint's protected member access warnings for this file,
# allowing tests to access RedisAdapter's internal methods as needed for integration testing.
"""Integration tests for the RedisAdapter."""
from collections.abc import Generator
@@ -8,6 +5,7 @@ import pytest
import redis
from redis.commands.json.path import Path as RedisPath
from python_repositories.adapters.redis_adapter import RedisAdapter
from python_repositories.interfaces import JsonRepositoryInterface
@pytest.fixture(scope="module")
@@ -50,7 +48,7 @@ def clear_redis(raw_redis_client: redis.Redis) -> None:
def test_should_adhere_to_interface(redis_container: str) -> None:
"""Test that the RedisAdapter adheres to the expected interface."""
# Instantiation fails if interface not adhered to
assert issubclass(RedisAdapter, JsonRepositoryInterface)
_ = RedisAdapter()
@@ -134,7 +132,7 @@ def test_should_get_value(
# Arrange
key, data = data_in_redis
# Act
value = redis_adapter._get(key)
value = redis_adapter.get(key)
# Assert
assert value is not None
assert value == data
@@ -145,7 +143,7 @@ def test_should_get_none_for_missing_key(
) -> None:
"""Test that getting a non-existent key returns None."""
# Act
value = redis_adapter._get("nonexistent_key")
value = redis_adapter.get("nonexistent_key")
# Assert
assert value is None
@@ -159,7 +157,7 @@ def test_should_raise_value_error_on_invalid_get_key(
# Act & Assert
for key in invalid_keys:
with pytest.raises(ValueError):
redis_adapter._get(key) # type: ignore
redis_adapter.get(key) # type: ignore
def test_should_raise_connection_error_on_get_when_not_connected(
@@ -170,7 +168,7 @@ def test_should_raise_connection_error_on_get_when_not_connected(
adapter = RedisAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError):
adapter._get("some_key")
adapter.get("some_key")
def test_should_set_value(
@@ -180,12 +178,12 @@ def test_should_set_value(
"""Test that the RedisAdapter can set a value."""
# Arrange
key = "test_key"
received_data = redis_adapter._get(key)
received_data = redis_adapter.get(key)
assert received_data is None # Ensure key does not exist
# Act
redis_adapter._set(key, data)
redis_adapter.set(key, data)
# Assert
received_data = redis_adapter._get(key)
received_data = redis_adapter.get(key)
assert received_data is not None
assert received_data == data
@@ -198,13 +196,13 @@ def test_should_update_value(
# Arrange
key, _ = data_in_redis
new_data = {"new_key": "new_value"}
received_data = redis_adapter._get(key)
received_data = redis_adapter.get(key)
assert received_data is not None
assert received_data != new_data
# Act
redis_adapter._set(key, new_data)
redis_adapter.set(key, new_data)
# Assert
assert redis_adapter._get(key) == new_data
assert redis_adapter.get(key) == new_data
def test_should_raise_value_error_on_invalid_set_key(
@@ -217,7 +215,7 @@ def test_should_raise_value_error_on_invalid_set_key(
# Act & Assert
for key in invalid_keys:
with pytest.raises(ValueError):
redis_adapter._set(key, data) # type: ignore
redis_adapter.set(key, data) # type: ignore
def test_should_raise_value_error_on_invalid_set_data(
@@ -230,7 +228,7 @@ def test_should_raise_value_error_on_invalid_set_data(
# Act & Assert
for data in invalid_data:
with pytest.raises(ValueError):
redis_adapter._set(key, data) # type: ignore
redis_adapter.set(key, data) # type: ignore
def test_should_raise_connection_error_on_set_when_not_connected(
@@ -243,7 +241,7 @@ def test_should_raise_connection_error_on_set_when_not_connected(
key = "test_key"
# Act & Assert
with pytest.raises(ConnectionError):
adapter._set(key, data)
adapter.set(key, data)
def test_should_delete_key(
@@ -253,12 +251,12 @@ def test_should_delete_key(
"""Test that deleting a key removes it from Redis."""
# Arrange
key, _ = data_in_redis
received_data = redis_adapter._get(key)
received_data = redis_adapter.get(key)
assert received_data is not None # Ensure key exists
# Act
redis_adapter._delete(key)
redis_adapter.delete(key)
# Assert
assert redis_adapter._get(key) is None
assert redis_adapter.get(key) is None
def test_should_raise_value_error_on_invalid_delete_key(
@@ -270,7 +268,7 @@ def test_should_raise_value_error_on_invalid_delete_key(
# Act & Assert
for key in invalid_keys:
with pytest.raises(ValueError):
redis_adapter._delete(key) # type: ignore
redis_adapter.delete(key) # type: ignore
def test_should_raise_connection_error_on_delete_when_not_connected(
@@ -281,7 +279,7 @@ def test_should_raise_connection_error_on_delete_when_not_connected(
adapter = RedisAdapter()
# Act & Assert
with pytest.raises(ConnectionError):
adapter._delete("some_key")
adapter.delete("some_key")
def test_should_list_keys(
@@ -289,10 +287,10 @@ def test_should_list_keys(
) -> None:
"""Test listing keys matching a pattern returns correct keys."""
# Arrange
redis_adapter._set("key1", {"a": 1})
redis_adapter._set("key2", {"b": 2})
redis_adapter.set("key1", {"a": 1})
redis_adapter.set("key2", {"b": 2})
# Act
keys = redis_adapter._list_keys("key*")
keys = redis_adapter.list_keys("key*")
# Assert
assert set(keys) == {"key1", "key2"}
@@ -306,7 +304,7 @@ def test_should_raise_value_error_on_invalid_list_keys_pattern(
# Act & Assert
for pattern in invalid_patterns:
with pytest.raises(ValueError):
redis_adapter._list_keys(pattern) # type: ignore
redis_adapter.list_keys(pattern) # type: ignore
def test_should_raise_connection_error_on_list_keys_when_not_connected(
@@ -317,7 +315,7 @@ def test_should_raise_connection_error_on_list_keys_when_not_connected(
adapter = RedisAdapter()
# Act & Assert
with pytest.raises(ConnectionError):
adapter._list_keys("some_pattern")
adapter.list_keys("some_pattern")
# allows local debugging by running file as script
+36
View File
@@ -0,0 +1,36 @@
"""Redis test container without testcontainers' deprecated wait decorator."""
from typing import Any, cast
import redis
from testcontainers.core.container import DockerContainer
from testcontainers.core.waiting_utils import WaitStrategy, WaitStrategyTarget
REDIS_PORT = 6379
class _RedisPingWaitStrategy(WaitStrategy):
def __init__(self) -> None:
super().__init__()
self.with_transient_exceptions(redis.exceptions.ConnectionError)
def wait_until_ready(self, container: WaitStrategyTarget) -> None:
redis_container = cast("RedisTestContainer", container)
if not self._poll(lambda: redis_container.get_client().ping()):
raise redis.exceptions.ConnectionError("Could not connect to Redis")
class RedisTestContainer(DockerContainer):
"""Redis container using wait strategies instead of the deprecated decorator."""
def __init__(self, image: str, port: int = REDIS_PORT) -> None:
super().__init__(image, _wait_strategy=_RedisPingWaitStrategy())
self.port = port
self.with_exposed_ports(self.port)
def get_client(self, **kwargs: Any) -> redis.Redis:
return redis.Redis(
host=self.get_container_host_ip(),
port=self.get_exposed_port(self.port),
**kwargs,
)
+119
View File
@@ -0,0 +1,119 @@
"""Tests for optional dependency import behavior."""
from __future__ import annotations
import builtins
import importlib
import os
import subprocess
import sys
from collections.abc import Callable, Mapping, Sequence
from pathlib import Path
from types import ModuleType
from unittest.mock import patch
import pytest
import python_repositories
_ROOT = Path(__file__).resolve().parents[2]
def _block_backend_import(blocked_prefix: str) -> Callable[..., ModuleType]:
real_import = builtins.__import__
def fake_import(
name: str,
globals: Mapping[str, object] | None = None,
locals: Mapping[str, object] | None = None,
fromlist: Sequence[str] = (),
level: int = 0,
) -> ModuleType:
if name == blocked_prefix or name.startswith(f"{blocked_prefix}."):
raise ImportError(f"No module named '{name}'")
return real_import(name, globals, locals, fromlist, level)
return fake_import
def test_base_import_does_not_load_adapters() -> None:
"""Base package import does not eagerly load backend adapter modules."""
script = """
import sys
from python_repositories import JsonRepositoryInterface
assert JsonRepositoryInterface is not None
assert "python_repositories.adapters.redis_adapter" not in sys.modules
assert "python_repositories.adapters.minio_adapter" not in sys.modules
"""
result = subprocess.run(
[sys.executable, "-c", script],
cwd=_ROOT,
env={**os.environ, "PYTHONPATH": str(_ROOT)},
capture_output=True,
text=True,
)
assert result.returncode == 0, result.stderr or result.stdout
def test_lazy_adapter_load_succeeds_when_extra_present() -> None:
"""Adapters load when their optional dependencies are installed."""
from python_repositories import MinioAdapter, RedisAdapter
assert RedisAdapter.__name__ == "RedisAdapter"
assert MinioAdapter.__name__ == "MinioAdapter"
def test_redis_adapter_import_error_without_extra() -> None:
"""Missing redis extra raises ImportError with install hint."""
import python_repositories.adapters.redis_adapter as redis_adapter_module
with patch.object(builtins, "__import__", new=_block_backend_import("redis")):
with pytest.raises(ImportError, match=r"python-repositories\[redis\]"):
importlib.reload(redis_adapter_module)
importlib.reload(redis_adapter_module)
def test_minio_adapter_import_error_without_extra() -> None:
"""Missing minio extra raises ImportError with install hint."""
import python_repositories.adapters.minio_adapter as minio_adapter_module
with patch.object(builtins, "__import__", new=_block_backend_import("minio")):
with pytest.raises(ImportError, match=r"python-repositories\[minio\]"):
importlib.reload(minio_adapter_module)
importlib.reload(minio_adapter_module)
def test_top_level_lazy_import_propagates_redis_import_error() -> None:
"""Top-level RedisAdapter access surfaces adapter import errors."""
with patch(
"importlib.import_module",
side_effect=ImportError(
"Redis support requires the redis extra. "
"Install with: pip install python-repositories[redis]"
),
):
with pytest.raises(ImportError, match=r"python-repositories\[redis\]"):
_ = python_repositories.RedisAdapter
def test_top_level_lazy_import_propagates_minio_import_error() -> None:
"""Top-level MinioAdapter access surfaces adapter import errors."""
with patch(
"importlib.import_module",
side_effect=ImportError(
"MinIO support requires the minio extra. "
"Install with: pip install python-repositories[minio]"
),
):
with pytest.raises(ImportError, match=r"python-repositories\[minio\]"):
_ = python_repositories.MinioAdapter
def test_adapters_subpackage_lazy_import_succeeds() -> None:
"""Adapter subpackage imports delegate to the same lazy loader."""
from python_repositories.adapters import RedisAdapter
assert RedisAdapter.__name__ == "RedisAdapter"
Generated
+1035 -738
View File
File diff suppressed because it is too large Load Diff