Compare commits

..
10 Commits
Author SHA1 Message Date
CI Bot 2f19fcc972 chore: release v1.0.0 [skip ci] 2026-07-08 18:25:00 +00:00
brian 3bd65895ec Merge pull request '[breaking] Clarify MinIO get() error semantics to match Redis behavior' (#33) from cursor/minio-error-semantics into main
Code Quality Pipeline / code-quality (push) Successful in 47s
Test Python Package / unit-tests (push) Successful in 1m17s
Test Python Package / coverage-report (push) Successful in 16s
Release on merge to main / release (push) Successful in 16s
Test Python Package / integration-tests (push) Successful in 1m6s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/33
2026-07-08 20:24:41 +02:00
Brian Bjarke JensenandCursor 5311d49fa6 Clarify MinIO get() error semantics to match Redis behavior.
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / integration-tests (pull_request) Successful in 1m17s
Test Python Package / unit-tests (pull_request) Successful in 1m28s
Code Quality Pipeline / code-quality (pull_request) Successful in 1m36s
Test Python Package / coverage-report (pull_request) Successful in 16s
Return None only for missing objects and re-raise other S3 and network failures so callers can distinguish not-found from real errors.

Co-authored-by: Cursor <[email protected]>
2026-07-08 20:02:03 +02:00
CI Bot 703bb9521f chore: release v0.5.1 [skip ci] 2026-07-07 19:37:22 +00:00
brian 8e47ebf4c6 Merge pull request '[patch] Fix MinIO get_object response leak in MinioAdapter.get()' (#32) from fix/minio-get-object-response-cleanup into main
Release on merge to main / release (push) Successful in 14s
Test Python Package / unit-tests (push) Successful in 15s
Code Quality Pipeline / code-quality (push) Successful in 47s
Test Python Package / integration-tests (push) Successful in 54s
Test Python Package / coverage-report (push) Successful in 15s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/32
2026-07-07 21:37:04 +02:00
Brian Bjarke JensenandCursor 9a800bf553 Fetch live PR title in pr-title-check CI workflow.
Code Quality Pipeline / code-quality (pull_request) Successful in 24s
Test Python Package / integration-tests (pull_request) Successful in 1m18s
Test Python Package / unit-tests (pull_request) Successful in 1m24s
Test Python Package / coverage-report (pull_request) Successful in 12s
PR Title Check / check-title (pull_request) Successful in 6s
Rerunning the job now reads the current title from the Gitea API instead of the stale pull_request event payload.

Co-authored-by: Cursor <[email protected]>
2026-07-07 21:34:18 +02:00
Brian Bjarke JensenandCursor 8c34534187 Fix MinIO get_object response leak in MinioAdapter.get().
Test Python Package / integration-tests (pull_request) Successful in 27s
Code Quality Pipeline / code-quality (pull_request) Successful in 31s
Test Python Package / unit-tests (pull_request) Successful in 36s
Test Python Package / coverage-report (pull_request) Successful in 53s
PR Title Check / check-title (pull_request) Failing after 5s
Close and release the HTTP response in a finally block per SDK guidance, and add unit tests for success and read-failure cleanup.

Co-authored-by: Cursor <[email protected]>
2026-07-07 21:21:42 +02:00
brian b1210bddf2 Merge pull request 'Reach 100% combined coverage with targeted unit tests' (#31) from cursor/balanced-100-coverage into main
Test Python Package / coverage-report (push) Successful in 45s
Test Python Package / unit-tests (push) Successful in 14s
Code Quality Pipeline / code-quality (push) Successful in 33s
Release on merge to main / release (push) Successful in 36s
Test Python Package / integration-tests (push) Successful in 23s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/31
2026-07-07 21:05:51 +02:00
Brian Bjarke JensenandCursor 39b104383c Fix mypy unreachable-code warning in minio reconnect test.
PR Title Check / check-title (pull_request) Successful in 7s
Test Python Package / coverage-report (pull_request) Successful in 12s
Code Quality Pipeline / code-quality (pull_request) Successful in 48s
Test Python Package / unit-tests (pull_request) Successful in 19s
Test Python Package / integration-tests (pull_request) Successful in 28s
Drop bucket_name assertions that mypy treated as always false after manually clearing _bucket_name.

Co-authored-by: Cursor <[email protected]>
2026-07-07 21:04:39 +02:00
Brian Bjarke JensenandCursor beb2b5128e Reach 100% combined coverage with targeted unit tests.
Code Quality Pipeline / code-quality (pull_request) Failing after 29s
Test Python Package / integration-tests (pull_request) Successful in 35s
Test Python Package / unit-tests (pull_request) Successful in 42s
PR Title Check / check-title (pull_request) Successful in 52s
Test Python Package / coverage-report (pull_request) Successful in 12s
Add fast unit tests for lazy-import helpers and injected-client connect paths that integration tests miss, and raise the coverage floor to 100%.

Co-authored-by: Cursor <[email protected]>
2026-07-07 21:01:03 +02:00
10 changed files with 241 additions and 27 deletions
+7 -1
View File
@@ -16,8 +16,14 @@ jobs:
- name: Check PR title when source files change
env:
PR_TITLE: ${{ github.event.pull_request.title }}
API_URL: ${{ vars.API_URL }}
REPO_OWNER: ${{ github.repository_owner }}
REPO_NAME: ${{ github.event.repository.name }}
PR_NUMBER: ${{ github.event.pull_request.number }}
CI_RUNNER_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
run: |
git fetch origin "${{ github.base_ref }}"
PR_TITLE=$(scripts/ci/fetch-pr-title.sh)
echo "Live PR title: ${PR_TITLE}"
git diff --name-only "origin/${{ github.base_ref }}...HEAD" \
| scripts/ci/check-pr-title.sh "$PR_TITLE"
+2 -2
View File
@@ -1,6 +1,6 @@
[project]
name = "python-repositories"
version = "0.5.0"
version = "1.0.0"
description = "Various python repository interfaces exposed as a python package."
authors = [
{ name = "Brian Bjarke Jensen", email = "[email protected]" }
@@ -43,7 +43,7 @@ markers = [
source = ["python_repositories"]
[tool.coverage.report]
fail_under = 90
fail_under = 100
show_missing = true
precision = 2
@@ -152,6 +152,7 @@ class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
assert self._client is not None and self._bucket_name is not None
# Get data from bucket
# N.B. bucket name is set when connecting
response = None
try:
response = self._client.get_object(
bucket_name=self._bucket_name,
@@ -171,11 +172,12 @@ class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
self.logger.warning(
f"Object '{object_name}' not found in bucket '{self._bucket_name}'"
)
else:
self.logger.error(repr(exc))
except Exception as exc: # pylint: disable=broad-except
self.logger.error(repr(exc))
return None
raise
finally:
if response is not None:
response.close()
response.release_conn()
def delete(self, object_name: str) -> None:
"""Delete an object from the Minio bucket."""
@@ -9,7 +9,11 @@ class ObjectRepositoryInterface(ABC):
@abstractmethod
def get(self, object_name: str) -> BytesIO | None:
"""Get an object by name."""
"""Get an object by name.
Returns None when the object does not exist. Raises ConnectionError when
not connected. Other backend errors propagate to the caller.
"""
...
@abstractmethod
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -euo pipefail
# Fetch the current PR title from the Gitea API.
# Requires: API_URL, REPO_OWNER, REPO_NAME, PR_NUMBER, CI_RUNNER_TOKEN
: "${API_URL:?API_URL is required}"
: "${REPO_OWNER:?REPO_OWNER is required}"
: "${REPO_NAME:?REPO_NAME is required}"
: "${PR_NUMBER:?PR_NUMBER is required}"
: "${CI_RUNNER_TOKEN:?CI_RUNNER_TOKEN is required}"
curl -sf \
"${API_URL}/repos/${REPO_OWNER}/${REPO_NAME}/pulls/${PR_NUMBER}" \
-H "Authorization: token ${CI_RUNNER_TOKEN}" \
| python3 -c 'import json, sys; print(json.load(sys.stdin)["title"])'
+9 -16
View File
@@ -218,10 +218,8 @@ def test_should_log_warning_when_getting_nonexistent_object(
)
def test_should_log_error_when_getting_with_s3error_other_than_no_such_key(
caplog: pytest.LogCaptureFixture,
) -> None:
"""Test that the MinioAdapter logs an error for unhandled S3 errors."""
def test_should_reraise_s3error_other_than_no_such_key() -> None:
"""Test that the MinioAdapter re-raises unhandled S3 errors."""
mock_client = MagicMock(spec=Minio)
mock_client.bucket_exists.return_value = True
other_s3error = S3Error(
@@ -236,25 +234,20 @@ def test_should_log_error_when_getting_with_s3error_other_than_no_such_key(
)
mock_client.get_object.side_effect = other_s3error
adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
with caplog.at_level("ERROR"):
result = adapter.get("missing-object")
assert result is None
assert repr(other_s3error) in caplog.text
with pytest.raises(S3Error) as exc_info:
adapter.get("missing-object")
assert exc_info.value.code == "UnhandledError"
def test_should_log_error_when_getting_with_general_exception(
caplog: pytest.LogCaptureFixture,
) -> None:
"""Test that the MinioAdapter logs an error on general exceptions during get."""
def test_should_reraise_general_exception() -> None:
"""Test that the MinioAdapter re-raises general exceptions during get."""
mock_client = MagicMock(spec=Minio)
mock_client.bucket_exists.return_value = True
general_exception = Exception("General failure")
mock_client.get_object.side_effect = general_exception
adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
with caplog.at_level("ERROR"):
result = adapter.get("missing-object")
assert result is None
assert repr(general_exception) in caplog.text
with pytest.raises(Exception, match="General failure"):
adapter.get("missing-object")
def test_should_put_data(
+21
View File
@@ -117,3 +117,24 @@ def test_adapters_subpackage_lazy_import_succeeds() -> None:
from python_repositories.adapters import RedisAdapter
assert RedisAdapter.__name__ == "RedisAdapter"
def test_adapters_dir_exposes_lazy_exports() -> None:
"""dir(adapters) includes lazy adapter names for tab completion."""
import python_repositories.adapters as adapters
assert {"RedisAdapter", "MinioAdapter"}.issubset(set(dir(adapters)))
def test_adapters_getattr_raises_for_unknown() -> None:
"""Unknown adapter names raise AttributeError."""
import python_repositories.adapters as adapters
with pytest.raises(AttributeError, match="has no attribute 'NoSuchAdapter'"):
_ = adapters.NoSuchAdapter
def test_top_level_dir_exposes_lazy_exports() -> None:
"""dir(python_repositories) includes lazy adapter names for tab completion."""
assert "RedisAdapter" in dir(python_repositories)
assert "MinioAdapter" in dir(python_repositories)
+128 -1
View File
@@ -5,7 +5,8 @@ from __future__ import annotations
from unittest.mock import MagicMock
import pytest
from minio import Minio
from minio import Minio, S3Error
from urllib3.response import BaseHTTPResponse
from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.interfaces import ObjectRepositoryInterface
@@ -53,3 +54,129 @@ def test_raises_when_client_provided_without_config() -> None:
mock_client = MagicMock(spec=Minio)
with pytest.raises(ValueError, match="config is required"):
MinioAdapter(client=mock_client)
def test_connect_with_injected_client_succeeds() -> None:
mock_client = MagicMock(spec=Minio)
adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
adapter.connect()
mock_client.list_buckets.assert_called_once()
def test_connect_with_injected_client_raises_on_failure() -> None:
mock_client = MagicMock(spec=Minio)
mock_client.list_buckets.side_effect = Exception("connection lost")
adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
with pytest.raises(ConnectionError, match="Could not connect to Minio"):
adapter.connect()
def test_connect_disconnects_before_reconnect(
monkeypatch: pytest.MonkeyPatch,
) -> None:
stale_client = MagicMock(spec=Minio)
new_client = MagicMock(spec=Minio)
adapter = MinioAdapter(config=TEST_MINIO_CONFIG)
adapter._client = stale_client
adapter._bucket_name = None
monkeypatch.setattr(
"python_repositories.adapters.minio_adapter.minio.Minio",
lambda *args, **kwargs: new_client,
)
adapter.connect()
assert adapter._client is new_client
new_client.list_buckets.assert_called_once()
def test_get_closes_response_on_success() -> None:
"""get() must close and release the get_object HTTP response."""
mock_client = MagicMock(spec=Minio)
mock_client.bucket_exists.return_value = True
mock_response = MagicMock()
mock_response.read.side_effect = [b"data", b""]
mock_client.get_object.return_value = mock_response
adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
result = adapter.get("some-object")
assert result is not None
assert result.read() == b"data"
mock_response.close.assert_called_once()
mock_response.release_conn.assert_called_once()
def test_get_closes_response_when_read_fails() -> None:
"""get() must close and release the response even if read() raises."""
mock_client = MagicMock(spec=Minio)
mock_client.bucket_exists.return_value = True
mock_response = MagicMock()
mock_response.read.side_effect = OSError("connection reset")
mock_client.get_object.return_value = mock_response
adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
with pytest.raises(OSError, match="connection reset"):
adapter.get("some-object")
mock_response.close.assert_called_once()
mock_response.release_conn.assert_called_once()
def test_get_returns_none_for_no_such_key() -> None:
"""get() returns None when the object does not exist."""
mock_client = MagicMock(spec=Minio)
mock_client.bucket_exists.return_value = True
mock_client.get_object.side_effect = S3Error(
MagicMock(spec=BaseHTTPResponse),
"NoSuchKey",
"",
"",
"",
"",
bucket_name="test-bucket",
object_name="missing-object",
)
adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
result = adapter.get("missing-object")
assert result is None
def test_get_reraises_other_s3_errors() -> None:
"""get() re-raises S3 errors other than NoSuchKey."""
mock_client = MagicMock(spec=Minio)
mock_client.bucket_exists.return_value = True
other_s3error = S3Error(
MagicMock(spec=BaseHTTPResponse),
"AccessDenied",
"",
"",
"",
"",
bucket_name="test-bucket",
object_name="some-object",
)
mock_client.get_object.side_effect = other_s3error
adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
with pytest.raises(S3Error) as exc_info:
adapter.get("some-object")
assert exc_info.value.code == "AccessDenied"
def test_get_reraises_general_exception_from_get_object() -> None:
"""get() re-raises unexpected exceptions from get_object."""
mock_client = MagicMock(spec=Minio)
mock_client.bucket_exists.return_value = True
mock_client.get_object.side_effect = Exception("General failure")
adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
with pytest.raises(Exception, match="General failure"):
adapter.get("some-object")
+45
View File
@@ -69,3 +69,48 @@ def test_subclass_custom_env_var_name(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("CUSTOM_REDIS_URI", "redis://custom:6379")
adapter = CustomEnvRedisAdapter()
assert adapter._config.uri == "redis://custom:6379"
def test_connect_with_injected_client_succeeds_when_ping_ok() -> None:
mock_client = MagicMock(spec=redis.Redis)
mock_client.ping.return_value = True
adapter = RedisAdapter(config=TEST_REDIS_CONFIG, client=mock_client)
adapter.connect()
mock_client.ping.assert_called_once()
def test_connect_with_injected_client_raises_when_ping_false() -> None:
mock_client = MagicMock(spec=redis.Redis)
mock_client.ping.return_value = False
adapter = RedisAdapter(config=TEST_REDIS_CONFIG, client=mock_client)
with pytest.raises(ConnectionError, match="Could not connect to Redis"):
adapter.connect()
def test_connect_with_injected_client_raises_on_redis_error() -> None:
mock_client = MagicMock(spec=redis.Redis)
mock_client.ping.side_effect = redis.ConnectionError("connection lost")
adapter = RedisAdapter(config=TEST_REDIS_CONFIG, client=mock_client)
with pytest.raises(ConnectionError, match="Could not connect to Redis"):
adapter.connect()
def test_connect_closes_existing_non_injected_client(
monkeypatch: pytest.MonkeyPatch,
) -> None:
stale_client = MagicMock(spec=redis.Redis)
new_client = MagicMock(spec=redis.Redis)
new_client.ping.return_value = True
adapter = RedisAdapter(config=TEST_REDIS_CONFIG)
adapter._client = stale_client
monkeypatch.setattr("redis.Redis.from_url", lambda *args, **kwargs: new_client)
adapter.connect()
stale_client.close.assert_called_once()
assert adapter._client is new_client
Generated
+1 -1
View File
@@ -1056,7 +1056,7 @@ wheels = [
[[package]]
name = "python-repositories"
version = "0.5.0"
version = "1.0.0"
source = { editable = "." }
dependencies = [
{ name = "python-dotenv" },