Merge pull request '[minor] Add config and client injection with test reorganization' (#25) from cursor/config-client-injection into main
Code Quality Pipeline / code-quality (push) Successful in 31s
Release on merge to main / release (push) Successful in 14s
Test Python Package / test (push) Successful in 57s

Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/25
This commit was merged in pull request #25.
This commit is contained in:
2026-07-06 20:51:01 +02:00
28 changed files with 710 additions and 377 deletions
+9
View File
@@ -0,0 +1,9 @@
# Redis (requires redis extra)
REDIS_URI=redis://localhost:6379
# MinIO (requires minio extra)
MINIO_ENDPOINT=localhost:9000
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
MINIO_BUCKET=my-bucket
MINIO_SECURE=false
+43 -4
View File
@@ -42,6 +42,40 @@ Requires Redis with the RedisJSON module (e.g. redis-stack).
| `MINIO_ACCESS_KEY` | Access key | | `MINIO_ACCESS_KEY` | Access key |
| `MINIO_SECRET_KEY` | Secret key | | `MINIO_SECRET_KEY` | Secret key |
| `MINIO_BUCKET` | Bucket name (created on connect if missing) | | `MINIO_BUCKET` | Bucket name (created on connect if missing) |
| `MINIO_SECURE` | Use HTTPS (`true`/`false`; default: `true`) |
Copy [`.env.example`](.env.example) to `.env` for local development. `RedisConfig.from_env()` and `MinioConfig.from_env()` load `.env` automatically when resolving configuration from the environment.
## Configuration injection
Adapters accept optional `config` and `client` keyword arguments for explicit setup and testing:
```python
from python_repositories import RedisAdapter, RedisConfig, MinioAdapter, MinioConfig
redis = RedisAdapter(config=RedisConfig(uri="redis://localhost:6379"))
minio = MinioAdapter(
config=MinioConfig(
endpoint="localhost:9000",
access_key="minioadmin",
secret_key="minioadmin",
bucket="my-bucket",
secure=False,
)
)
```
When both `config` and `client` are provided, `connect()` skips client creation (the caller owns the client lifecycle). `config` is required whenever `client` is injected.
Load a `.env` file explicitly:
```python
from python_repositories import load_dotenv
load_dotenv() # optional — from_env() also loads .env by default
```
Calling `RedisAdapter()` or `MinioAdapter()` with no arguments still loads configuration from environment variables (and `.env` if present).
## Quick start ## Quick start
@@ -93,9 +127,12 @@ from python_repositories import (
ConnectionAwareInterface, ConnectionAwareInterface,
ContextAwareInterface, ContextAwareInterface,
JsonRepositoryInterface, JsonRepositoryInterface,
MinioAdapter,
MinioConfig,
ObjectRepositoryInterface, ObjectRepositoryInterface,
RedisAdapter, RedisAdapter,
MinioAdapter, RedisConfig,
load_dotenv,
) )
``` ```
@@ -104,9 +141,13 @@ from python_repositories import (
```bash ```bash
uv sync --all-extras uv sync --all-extras
uv run pre-commit install # once per clone — runs hooks on git commit uv run pre-commit install # once per clone — runs hooks on git commit
uv run pytest tests/integration/ -v uv run pytest tests/unit/ -v # fast, no Docker
uv run pytest -m "not integration" -v # all non-Docker tests
uv run pytest -v # full suite (requires Docker)
``` ```
Integration tests are marked with `@pytest.mark.integration` and require Docker (testcontainers). Run unit tests alone for quick local feedback.
`pre-commit` is included in the dev dependency group. `uv sync` installs the CLI, but git does not run hooks until you install them with `pre-commit install` (one time per clone). After that, commits run the checks defined in [`.pre-commit-config.yaml`](.pre-commit-config.yaml) (ruff, mypy, pyupgrade, prettier, and general file hygiene). `pre-commit` is included in the dev dependency group. `uv sync` installs the CLI, but git does not run hooks until you install them with `pre-commit install` (one time per clone). After that, commits run the checks defined in [`.pre-commit-config.yaml`](.pre-commit-config.yaml) (ruff, mypy, pyupgrade, prettier, and general file hygiene).
To run all hooks manually without committing: To run all hooks manually without committing:
@@ -115,8 +156,6 @@ To run all hooks manually without committing:
uv run pre-commit run --all-files uv run pre-commit run --all-files
``` ```
Integration tests require Docker (testcontainers).
## Releases ## Releases
Releases are automated when a pull request is merged to `main`. CI reads the **merged PR title** to decide whether and how to bump the version. Releases are automated when a pull request is merged to `main`. CI reads the **merged PR title** to decide whether and how to bump the version.
+5
View File
@@ -14,6 +14,7 @@ classifiers = [
"Operating System :: OS Independent", "Operating System :: OS Independent",
] ]
dependencies = [ dependencies = [
"python-dotenv>=1.0.0",
"python-utils>=0.1.0", "python-utils>=0.1.0",
"structlog>=25.4.0", "structlog>=25.4.0",
] ]
@@ -33,6 +34,10 @@ build-backend = "hatchling.build"
[tool.pytest.ini_options] [tool.pytest.ini_options]
testpaths = ["tests"] testpaths = ["tests"]
pythonpath = ["."] pythonpath = ["."]
addopts = "--import-mode=importlib"
markers = [
"integration: tests requiring Docker containers (deselect with '-m \"not integration\"')",
]
[tool.uv.sources] [tool.uv.sources]
python-utils = { index = "gitea" } python-utils = { index = "gitea" }
+4
View File
@@ -6,6 +6,7 @@ from typing import TYPE_CHECKING
# Interfaces are always available; they have no optional backend dependencies. # Interfaces are always available; they have no optional backend dependencies.
from . import adapters from . import adapters
from .config import MinioConfig, RedisConfig, load_dotenv
from .interfaces import ( from .interfaces import (
ConnectionAwareInterface, ConnectionAwareInterface,
ContextAwareInterface, ContextAwareInterface,
@@ -22,7 +23,10 @@ __all__ = [
"ConnectionAwareInterface", "ConnectionAwareInterface",
"ContextAwareInterface", "ContextAwareInterface",
"JsonRepositoryInterface", "JsonRepositoryInterface",
"MinioConfig",
"ObjectRepositoryInterface", "ObjectRepositoryInterface",
"RedisConfig",
"load_dotenv",
*adapters.__all__, *adapters.__all__,
] ]
+35 -22
View File
@@ -1,14 +1,12 @@
"""Definition of MinioAdapter class.""" """Definition of MinioAdapter class."""
from __future__ import annotations from __future__ import annotations
import os
from io import BytesIO from io import BytesIO
from python_utils import check_env
from python_repositories.adapters.connection_aware_adapter import ( from python_repositories.adapters.connection_aware_adapter import (
ConnectionAwareAdapter, ConnectionAwareAdapter,
) )
from python_repositories.config import MinioConfig
from python_repositories.interfaces import ObjectRepositoryInterface from python_repositories.interfaces import ObjectRepositoryInterface
try: try:
@@ -27,63 +25,78 @@ class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
access_key_env_var_name: str = "MINIO_ACCESS_KEY" access_key_env_var_name: str = "MINIO_ACCESS_KEY"
secret_key_env_var_name: str = "MINIO_SECRET_KEY" secret_key_env_var_name: str = "MINIO_SECRET_KEY"
bucket_env_var_name: str = "MINIO_BUCKET" bucket_env_var_name: str = "MINIO_BUCKET"
secure_env_var_name: str = "MINIO_SECURE"
chunk_size: int = 5 * 2**20 # 5 MiB chunk_size: int = 5 * 2**20 # 5 MiB
connection_name: str = "Minio" connection_name: str = "Minio"
def __init__(self) -> None: def __init__(
self,
*,
config: MinioConfig | None = None,
client: minio.Minio | None = None,
) -> None:
super().__init__() super().__init__()
check_env( if config is None:
{ if client is not None:
raise ValueError("config is required when client is provided")
config = MinioConfig.from_env(
self.endpoint_env_var_name, self.endpoint_env_var_name,
self.access_key_env_var_name, self.access_key_env_var_name,
self.secret_key_env_var_name, self.secret_key_env_var_name,
self.bucket_env_var_name, self.bucket_env_var_name,
}, self.secure_env_var_name,
) )
self._client: minio.Minio | None = None self._config = config
self._bucket_name: str | None = None self._client_injected = client is not None
self._client: minio.Minio | None = client
self._bucket_name: str | None = config.bucket if client is not None else None
if self._client_injected:
self._invalidate_health_cache()
def _is_client_ready(self) -> bool: def _is_client_ready(self) -> bool:
return self._client is not None and self._bucket_name is not None return self._client is not None and self._bucket_name is not None
def connect(self) -> None: def connect(self) -> None:
"""Connect to the Minio server.""" """Connect to the Minio server."""
if self._client_injected:
if self._client is not None:
try:
_ = self._client.list_buckets()
except Exception as exc: # pylint: disable=broad-except
raise ConnectionError(
f"Could not connect to Minio at {self._config.endpoint}"
) from exc
self._invalidate_health_cache()
return
if self._client is not None and self.is_connected(): if self._client is not None and self.is_connected():
self.logger.info("Already connected to Minio") self.logger.info("Already connected to Minio")
return return
if self._client is not None: if self._client is not None:
self.disconnect() self.disconnect()
# Prepare arguments endpoint = self._config.endpoint
endpoint = str(os.getenv(self.endpoint_env_var_name)) access_key = self._config.access_key
access_key = str(os.getenv(self.access_key_env_var_name)) secret_key = self._config.secret_key
secret_key = str(os.getenv(self.secret_key_env_var_name)) bucket = self._config.bucket
bucket = str(os.getenv(self.bucket_env_var_name))
# Connect client
client = minio.Minio( client = minio.Minio(
endpoint=endpoint, endpoint=endpoint,
access_key=access_key, access_key=access_key,
secret_key=secret_key, secret_key=secret_key,
secure=False, secure=self._config.secure,
) )
# Test the connection by listing buckets (will raise if connection fails)
try: try:
_ = client.list_buckets() _ = client.list_buckets()
except Exception as exc: # pylint: disable=broad-except except Exception as exc: # pylint: disable=broad-except
raise ConnectionError(f"Could not connect to Minio at {endpoint}") from exc raise ConnectionError(f"Could not connect to Minio at {endpoint}") from exc
# Ensure bucket exists
if not client.bucket_exists(bucket): if not client.bucket_exists(bucket):
self.logger.info(f"Creating bucket '{bucket}'") self.logger.info(f"Creating bucket '{bucket}'")
client.make_bucket(bucket) client.make_bucket(bucket)
# Persist information
self._client = client self._client = client
self._bucket_name = bucket self._bucket_name = bucket
self._invalidate_health_cache() self._invalidate_health_cache()
def disconnect(self) -> None: def disconnect(self) -> None:
"""Disconnect from the Minio server.""" """Disconnect from the Minio server."""
# Close connection # N.B. Minio client does not have a close method, but we include this for symmetry
# N.B. Minio client does not have a close method, but we include this for symmetry with other adapters
# Reset client
self._client = None self._client = None
self._bucket_name = None self._bucket_name = None
self._invalidate_health_cache() self._invalidate_health_cache()
+31 -13
View File
@@ -2,13 +2,11 @@
from __future__ import annotations from __future__ import annotations
from typing import cast from typing import cast
import os
from python_utils import check_env
from python_repositories.adapters.connection_aware_adapter import ( from python_repositories.adapters.connection_aware_adapter import (
ConnectionAwareAdapter, ConnectionAwareAdapter,
) )
from python_repositories.config import RedisConfig
from python_repositories.interfaces import JsonRepositoryInterface from python_repositories.interfaces import JsonRepositoryInterface
try: try:
@@ -29,24 +27,47 @@ class RedisAdapter(JsonRepositoryInterface, ConnectionAwareAdapter):
encoding: str = "UTF-8" encoding: str = "UTF-8"
connection_name: str = "Redis" connection_name: str = "Redis"
def __init__(self) -> None: def __init__(
self,
*,
config: RedisConfig | None = None,
client: redis.Redis | None = None,
) -> None:
super().__init__() super().__init__()
check_env(self.uri_env_var_name) if config is None:
self._client: redis.Redis | None = None if client is not None:
raise ValueError("config is required when client is provided")
config = RedisConfig.from_env(self.uri_env_var_name)
self._config = config
self._client_injected = client is not None
self._client: redis.Redis | None = client
self.path: str = RedisPath.root_path() self.path: str = RedisPath.root_path()
if self._client_injected:
self._invalidate_health_cache()
def _is_client_ready(self) -> bool: def _is_client_ready(self) -> bool:
return self._client is not None return self._client is not None
def connect(self) -> None: def connect(self) -> None:
"""Connect to the Redis server.""" """Connect to the Redis server."""
if self._client_injected:
if self._client is not None:
try:
if not self._client.ping():
raise ConnectionError(
f"Could not connect to Redis at {self._config.uri}"
)
except (redis.ConnectionError, redis.TimeoutError) as exc:
raise ConnectionError(
f"Could not connect to Redis at {self._config.uri}"
) from exc
self._invalidate_health_cache()
return
if self._client is not None: if self._client is not None:
self._client.close() self._client.close()
self._client = None self._client = None
self._invalidate_health_cache() self._invalidate_health_cache()
# Prepare arguments uri = self._config.uri
uri = str(os.getenv(self.uri_env_var_name))
# Connect client
try: try:
client = redis.Redis.from_url( client = redis.Redis.from_url(
url=uri, url=uri,
@@ -56,16 +77,13 @@ class RedisAdapter(JsonRepositoryInterface, ConnectionAwareAdapter):
raise ConnectionError(f"Could not connect to Redis at {uri}") raise ConnectionError(f"Could not connect to Redis at {uri}")
except (redis.ConnectionError, redis.TimeoutError) as exc: except (redis.ConnectionError, redis.TimeoutError) as exc:
raise ConnectionError(f"Could not connect to Redis at {uri}") from exc raise ConnectionError(f"Could not connect to Redis at {uri}") from exc
# Persist client
self._client = client self._client = client
self._invalidate_health_cache() self._invalidate_health_cache()
def disconnect(self) -> None: def disconnect(self) -> None:
"""Disconnect from the Redis server.""" """Disconnect from the Redis server."""
# Close connection if self._client is not None and not self._client_injected:
if self._client is not None:
self._client.close() self._client.close()
# Reset client
self._client = None self._client = None
self._invalidate_health_cache() self._invalidate_health_cache()
+9
View File
@@ -0,0 +1,9 @@
from .dotenv_loader import load_dotenv as load_dotenv
from .minio_config import MinioConfig as MinioConfig
from .redis_config import RedisConfig as RedisConfig
__all__ = [
"MinioConfig",
"RedisConfig",
"load_dotenv",
]
@@ -0,0 +1,14 @@
"""Load environment variables from a .env file."""
from __future__ import annotations
from pathlib import Path
from dotenv import load_dotenv as _load_dotenv
def load_dotenv(path: str | Path | None = None) -> bool:
"""Load .env into os.environ. Idempotent; returns True if a file was loaded."""
if path is None:
return bool(_load_dotenv())
return bool(_load_dotenv(path))
@@ -0,0 +1,65 @@
"""MinIO connection configuration."""
from __future__ import annotations
import os
from dataclasses import dataclass
from python_utils import check_env
from python_repositories.config.dotenv_loader import load_dotenv
_TRUTHY = frozenset({"1", "true", "yes", "on"})
_FALSY = frozenset({"0", "false", "no", "off"})
def _env_bool(name: str, default: bool) -> bool:
raw = os.getenv(name)
if raw is None:
return default
normalized = raw.strip().lower()
if normalized in _TRUTHY:
return True
if normalized in _FALSY:
return False
raise ValueError(f"Invalid boolean value for {name}: {raw!r}")
@dataclass(frozen=True)
class MinioConfig:
"""Configuration for connecting to MinIO."""
endpoint: str
access_key: str
secret_key: str
bucket: str
secure: bool = True
@classmethod
def from_env(
cls,
endpoint_env_var_name: str = "MINIO_ENDPOINT",
access_key_env_var_name: str = "MINIO_ACCESS_KEY",
secret_key_env_var_name: str = "MINIO_SECRET_KEY",
bucket_env_var_name: str = "MINIO_BUCKET",
secure_env_var_name: str = "MINIO_SECURE",
*,
use_dotenv: bool = True,
) -> MinioConfig:
"""Load configuration from environment variables."""
if use_dotenv:
load_dotenv()
env_var_names = {
endpoint_env_var_name,
access_key_env_var_name,
secret_key_env_var_name,
bucket_env_var_name,
}
check_env(env_var_names)
return cls(
endpoint=str(os.getenv(endpoint_env_var_name)),
access_key=str(os.getenv(access_key_env_var_name)),
secret_key=str(os.getenv(secret_key_env_var_name)),
bucket=str(os.getenv(bucket_env_var_name)),
secure=_env_bool(secure_env_var_name, default=True),
)
@@ -0,0 +1,30 @@
"""Redis connection configuration."""
from __future__ import annotations
import os
from dataclasses import dataclass
from python_utils import check_env
from python_repositories.config.dotenv_loader import load_dotenv
@dataclass(frozen=True)
class RedisConfig:
"""Configuration for connecting to Redis."""
uri: str
@classmethod
def from_env(
cls,
uri_env_var_name: str = "REDIS_URI",
*,
use_dotenv: bool = True,
) -> RedisConfig:
"""Load configuration from environment variables."""
if use_dotenv:
load_dotenv()
check_env(uri_env_var_name)
return cls(uri=str(os.getenv(uri_env_var_name)))
+12
View File
@@ -0,0 +1,12 @@
"""Shared test configuration constants."""
from python_repositories.config import MinioConfig, RedisConfig
TEST_REDIS_CONFIG = RedisConfig(uri="redis://localhost:6379")
TEST_MINIO_CONFIG = MinioConfig(
endpoint="localhost:9000",
access_key="minioadmin",
secret_key="minioadmin",
bucket="test-bucket",
secure=False,
)
+17 -31
View File
@@ -1,7 +1,6 @@
"""Integration tests configuration.""" """Integration tests configuration."""
import logging import logging
import os
from collections.abc import Generator from collections.abc import Generator
import pytest import pytest
@@ -10,6 +9,7 @@ import structlog
from minio import Minio from minio import Minio
from testcontainers.minio import MinioContainer from testcontainers.minio import MinioContainer
from python_repositories.config import MinioConfig, RedisConfig
from tests.integration.redis_container_test import REDIS_PORT, RedisTestContainer from tests.integration.redis_container_test import REDIS_PORT, RedisTestContainer
collect_ignore = ["redis_container_test.py"] collect_ignore = ["redis_container_test.py"]
@@ -22,7 +22,6 @@ MINIO_BUCKET = "test-bucket"
@pytest.fixture(scope="session", autouse=True) @pytest.fixture(scope="session", autouse=True)
def configure_logging() -> None: def configure_logging() -> None:
"""Configure logging for the test session.""" """Configure logging for the test session."""
# Configure structlog
structlog.configure( structlog.configure(
processors=[ processors=[
structlog.stdlib.filter_by_level, structlog.stdlib.filter_by_level,
@@ -35,40 +34,34 @@ def configure_logging() -> None:
wrapper_class=structlog.stdlib.BoundLogger, wrapper_class=structlog.stdlib.BoundLogger,
cache_logger_on_first_use=True, cache_logger_on_first_use=True,
) )
# Set up basic logging configuration
logging.basicConfig(level=logging.ERROR) logging.basicConfig(level=logging.ERROR)
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def redis_container() -> Generator[str, None, None]: def redis_container() -> Generator[str, None, None]:
"""Set up a Redis container for testing and yield the Redis URI.""" """Set up a Redis container for testing and yield the Redis URI."""
# Start container
container = RedisTestContainer( container = RedisTestContainer(
image="redis/redis-stack:7.2.0-v0", image="redis/redis-stack:7.2.0-v0",
) )
container.start() container.start()
# Set environment variable for Redis URI
redis_host = container.get_container_host_ip() redis_host = container.get_container_host_ip()
redis_port = container.get_exposed_port(REDIS_PORT) redis_port = container.get_exposed_port(REDIS_PORT)
redis_uri = f"redis://{redis_host}:{redis_port}" redis_uri = f"redis://{redis_host}:{redis_port}"
yield redis_uri yield redis_uri
# Stop container
container.stop() container.stop()
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def minio_container() -> Generator[dict[str, str], None, None]: def minio_container() -> Generator[dict[str, str], None, None]:
"""Set up a Minio container for testing and yield the Minio URI.""" """Set up a Minio container for testing and yield connection settings."""
# Start container
container = MinioContainer( container = MinioContainer(
image="minio/minio:latest", image="minio/minio:latest",
access_key=MINIO_ACCESS_KEY, access_key=MINIO_ACCESS_KEY,
secret_key=MINIO_SECRET_KEY, secret_key=MINIO_SECRET_KEY,
) )
container.start() container.start()
# Build environment variables dictionary
minio_host = container.get_container_host_ip() minio_host = container.get_container_host_ip()
minio_port = container.get_exposed_port(9000) minio_port = container.get_exposed_port(9000)
minio_endpoint = f"{minio_host}:{minio_port}" minio_endpoint = f"{minio_host}:{minio_port}"
@@ -77,38 +70,35 @@ def minio_container() -> Generator[dict[str, str], None, None]:
"MINIO_ACCESS_KEY": MINIO_ACCESS_KEY, "MINIO_ACCESS_KEY": MINIO_ACCESS_KEY,
"MINIO_SECRET_KEY": MINIO_SECRET_KEY, "MINIO_SECRET_KEY": MINIO_SECRET_KEY,
"MINIO_BUCKET": MINIO_BUCKET, "MINIO_BUCKET": MINIO_BUCKET,
"MINIO_SECURE": "false",
} }
yield env_vars yield env_vars
# Stop container
container.stop() container.stop()
@pytest.fixture(scope="session", autouse=True) @pytest.fixture(scope="session")
def set_environment_variables( def redis_config(redis_container: str) -> RedisConfig:
redis_container: str, """Provide RedisConfig built from the test container."""
minio_container: dict[str, str], return RedisConfig(uri=redis_container)
) -> Generator[dict[str, str], None, None]:
"""Set environment variables needed for tests."""
# Build environment variables dictionary
env_vars = {"REDIS_URI": redis_container}
env_vars.update(minio_container)
# Set environment variables
for key, value in env_vars.items():
os.environ[key] = value
yield env_vars
# Cleanup @pytest.fixture(scope="session")
for key in env_vars: def minio_config(minio_container: dict[str, str]) -> MinioConfig:
_ = os.environ.pop(key, default=None) """Provide MinioConfig built from the test container."""
return MinioConfig(
endpoint=minio_container["MINIO_ENDPOINT"],
access_key=minio_container["MINIO_ACCESS_KEY"],
secret_key=minio_container["MINIO_SECRET_KEY"],
bucket=minio_container["MINIO_BUCKET"],
secure=False,
)
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def raw_redis_client(redis_container: str) -> Generator[redis.Redis, None, None]: def raw_redis_client(redis_container: str) -> Generator[redis.Redis, None, None]:
"""Provide a raw Redis client connected to the test Redis container.""" """Provide a raw Redis client connected to the test Redis container."""
# Connect client
client = redis.Redis.from_url( client = redis.Redis.from_url(
url=redis_container, url=redis_container,
socket_connect_timeout=10, socket_connect_timeout=10,
@@ -116,7 +106,6 @@ def raw_redis_client(redis_container: str) -> Generator[redis.Redis, None, None]
yield client yield client
# Cleanup
client.flushall() client.flushall()
client.close() client.close()
@@ -124,21 +113,18 @@ def raw_redis_client(redis_container: str) -> Generator[redis.Redis, None, None]
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def raw_minio_client(minio_container: dict[str, str]) -> Generator[Minio, None, None]: def raw_minio_client(minio_container: dict[str, str]) -> Generator[Minio, None, None]:
"""Provide a raw Minio client connected to the test Minio container.""" """Provide a raw Minio client connected to the test Minio container."""
# Connect client
client = Minio( client = Minio(
endpoint=minio_container["MINIO_ENDPOINT"], endpoint=minio_container["MINIO_ENDPOINT"],
access_key=minio_container["MINIO_ACCESS_KEY"], access_key=minio_container["MINIO_ACCESS_KEY"],
secret_key=minio_container["MINIO_SECRET_KEY"], secret_key=minio_container["MINIO_SECRET_KEY"],
secure=False, secure=False,
) )
# Ensure bucket exists
bucket_name = minio_container["MINIO_BUCKET"] bucket_name = minio_container["MINIO_BUCKET"]
if not client.bucket_exists(bucket_name): if not client.bucket_exists(bucket_name):
client.make_bucket(bucket_name) client.make_bucket(bucket_name)
yield client yield client
# Cleanup
objects = client.list_objects(bucket_name, recursive=True) objects = client.list_objects(bucket_name, recursive=True)
for obj in objects: for obj in objects:
client.remove_object(bucket_name, obj.object_name) client.remove_object(bucket_name, obj.object_name)
+20 -5
View File
@@ -2,29 +2,46 @@
from collections.abc import Generator from collections.abc import Generator
from io import BytesIO from io import BytesIO
import os
import random import random
import pytest import pytest
from python_repositories.examples.artifact_object_repository import ( from python_repositories.examples.artifact_object_repository import (
ArtifactObjectRepository, ArtifactObjectRepository,
) )
from python_repositories.examples.user_json_repository import UserJsonRepository from python_repositories.examples.user_json_repository import UserJsonRepository
pytestmark = pytest.mark.integration
@pytest.fixture(scope="module", autouse=True)
def set_example_env(
redis_container: str,
minio_container: dict[str, str],
) -> Generator[None, None, None]:
"""Set env vars so example repositories can use from_env() defaults."""
env_vars = {"REDIS_URI": redis_container, **minio_container}
for key, value in env_vars.items():
os.environ[key] = value
yield
for key in env_vars:
_ = os.environ.pop(key, default=None)
@pytest.fixture(scope="module") @pytest.fixture(scope="module")
def user_data() -> Generator[dict[str, str]]: def user_data() -> Generator[dict[str, str], None, None]:
"""Provide sample user data for tests.""" """Provide sample user data for tests."""
yield {"name": "Alice", "email": "[email protected]"} yield {"name": "Alice", "email": "[email protected]"}
@pytest.fixture(scope="module") @pytest.fixture(scope="module")
def artifact_data() -> Generator[BytesIO]: def artifact_data() -> Generator[BytesIO, None, None]:
"""Provide sample artifact data for tests.""" """Provide sample artifact data for tests."""
yield BytesIO(random.randbytes(2**20)) yield BytesIO(random.randbytes(2**20))
def test_user_json_repository_save_and_get( def test_user_json_repository_save_and_get(
redis_container: str,
user_data: dict[str, str], user_data: dict[str, str],
) -> None: ) -> None:
"""Test that UserJsonRepository can save and retrieve a user.""" """Test that UserJsonRepository can save and retrieve a user."""
@@ -34,7 +51,6 @@ def test_user_json_repository_save_and_get(
def test_user_json_repository_delete( def test_user_json_repository_delete(
redis_container: str,
user_data: dict[str, str], user_data: dict[str, str],
) -> None: ) -> None:
"""Test that UserJsonRepository can delete a user.""" """Test that UserJsonRepository can delete a user."""
@@ -45,7 +61,6 @@ def test_user_json_repository_delete(
def test_artifact_object_repository_store_and_get( def test_artifact_object_repository_store_and_get(
minio_container: dict[str, str],
artifact_data: BytesIO, artifact_data: BytesIO,
) -> None: ) -> None:
"""Test that ArtifactObjectRepository can store and retrieve an artifact.""" """Test that ArtifactObjectRepository can store and retrieve an artifact."""
+69 -150
View File
@@ -1,17 +1,20 @@
"""Integration tests for the MinioAdapter.""" """Integration tests for the MinioAdapter."""
from collections.abc import Generator from collections.abc import Generator
import pytest
from unittest.mock import MagicMock
from minio import Minio
from io import BytesIO
import random
import os
import logging import logging
from minio import S3Error import random
from io import BytesIO
from unittest.mock import MagicMock
import pytest
from minio import Minio, S3Error
from urllib3.response import BaseHTTPResponse from urllib3.response import BaseHTTPResponse
from python_repositories.adapters.minio_adapter import MinioAdapter from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.interfaces import ObjectRepositoryInterface from python_repositories.config import MinioConfig
from tests.conftest import TEST_MINIO_CONFIG
pytestmark = pytest.mark.integration
def same_data( def same_data(
@@ -19,15 +22,10 @@ def same_data(
data_b: BytesIO, data_b: BytesIO,
) -> bool: ) -> bool:
"""Check if two BytesIO-objects contain the same data.""" """Check if two BytesIO-objects contain the same data."""
assert isinstance(data_a, BytesIO)
assert isinstance(data_b, BytesIO)
# prepare for being read
data_a.seek(0) data_a.seek(0)
data_b.seek(0) data_b.seek(0)
# convert to bytes
data_a_bytes = data_a.read() data_a_bytes = data_a.read()
data_b_bytes = data_b.read() data_b_bytes = data_b.read()
# compare size
if len(data_a_bytes) != len(data_b_bytes): if len(data_a_bytes) != len(data_b_bytes):
logging.error( logging.error(
"data has different length: %s and %s", "data has different length: %s and %s",
@@ -35,7 +33,6 @@ def same_data(
len(data_b_bytes), len(data_b_bytes),
) )
return False return False
# compare content
if data_a_bytes != data_b_bytes: if data_a_bytes != data_b_bytes:
logging.error("data has different bytes") logging.error("data has different bytes")
return False return False
@@ -45,20 +42,19 @@ def same_data(
@pytest.fixture(scope="module") @pytest.fixture(scope="module")
def data() -> Generator[BytesIO, None, None]: def data() -> Generator[BytesIO, None, None]:
"""Provide a sample data bytes for tests.""" """Provide a sample data bytes for tests."""
# Generate random bytes random_bytes = random.randbytes(2**21)
random_bytes = random.randbytes(2**21) # 2 MiB
yield BytesIO(random_bytes) yield BytesIO(random_bytes)
@pytest.fixture(scope="function") @pytest.fixture(scope="function")
def data_in_minio( def data_in_minio(
raw_minio_client: Minio, raw_minio_client: Minio,
minio_config: MinioConfig,
data: BytesIO, data: BytesIO,
) -> Generator[tuple[str, BytesIO], None, None]: ) -> Generator[tuple[str, BytesIO], None, None]:
"""Fixture to set up a known value in Minio before each test.""" """Fixture to set up a known value in Minio before each test."""
object_name = "test_object" object_name = "test_object"
bucket_name = str(os.getenv("MINIO_BUCKET")) bucket_name = minio_config.bucket
# Upload object
num_bytes = data.getbuffer().nbytes num_bytes = data.getbuffer().nbytes
data.seek(0) data.seek(0)
raw_minio_client.put_object( raw_minio_client.put_object(
@@ -68,19 +64,17 @@ def data_in_minio(
length=num_bytes, length=num_bytes,
part_size=MinioAdapter.chunk_size, part_size=MinioAdapter.chunk_size,
) )
# Reset data for reading in tests
data.seek(0) data.seek(0)
yield object_name, data yield object_name, data
# Cleanup
raw_minio_client.remove_object(bucket_name, object_name) raw_minio_client.remove_object(bucket_name, object_name)
@pytest.fixture(scope="module") @pytest.fixture(scope="module")
def minio_adapter() -> Generator[MinioAdapter, None, None]: def minio_adapter(minio_config: MinioConfig) -> Generator[MinioAdapter, None, None]:
"""Fixture to provide a connected MinioAdapter instance.""" """Fixture to provide a connected MinioAdapter instance."""
adapter = MinioAdapter() adapter = MinioAdapter(config=minio_config)
adapter.connect() adapter.connect()
yield adapter yield adapter
adapter.disconnect() adapter.disconnect()
@@ -89,10 +83,10 @@ def minio_adapter() -> Generator[MinioAdapter, None, None]:
@pytest.fixture(scope="function", autouse=True) @pytest.fixture(scope="function", autouse=True)
def clear_minio( def clear_minio(
raw_minio_client: Minio, raw_minio_client: Minio,
minio_config: MinioConfig,
) -> None: ) -> None:
"""Fixture to clear all Minio objects before each test.""" """Fixture to clear all Minio objects before each test."""
bucket_name = str(os.getenv("MINIO_BUCKET")) bucket_name = minio_config.bucket
# Clear all objects before each test
objects = raw_minio_client.list_objects(bucket_name, recursive=True) objects = raw_minio_client.list_objects(bucket_name, recursive=True)
for obj in objects: for obj in objects:
if not obj.object_name: if not obj.object_name:
@@ -100,24 +94,6 @@ def clear_minio(
raw_minio_client.remove_object(bucket_name, obj.object_name) raw_minio_client.remove_object(bucket_name, obj.object_name)
def test_should_adhere_to_interface() -> None:
"""Test that the MinioAdapter adheres to the expected interface."""
assert issubclass(MinioAdapter, ObjectRepositoryInterface)
_ = MinioAdapter()
def test_should_have_logger_when_instantiated() -> None:
"""Test that the MinioAdapter has a logger when instantiated."""
adapter = MinioAdapter()
assert hasattr(adapter, "logger")
def test_should_not_be_connected_when_instantiated() -> None:
"""Test that the MinioAdapter is not connected when instantiated."""
adapter = MinioAdapter()
assert not adapter.is_connected()
def test_should_log_info_when_already_connected( def test_should_log_info_when_already_connected(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
caplog: pytest.LogCaptureFixture, caplog: pytest.LogCaptureFixture,
@@ -128,13 +104,15 @@ def test_should_log_info_when_already_connected(
assert "Already connected to Minio" in caplog.text assert "Already connected to Minio" in caplog.text
def test_should_raise_connection_error_when_unable_to_connect( def test_should_raise_connection_error_when_unable_to_connect() -> None:
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test that the MinioAdapter raises a ConnectionError when unable to connect.""" """Test that the MinioAdapter raises a ConnectionError when unable to connect."""
# Arrange config = MinioConfig(
monkeypatch.setenv("MINIO_ENDPOINT", "invalid_uri") endpoint="invalid_uri",
adapter = MinioAdapter() access_key="minioadmin",
secret_key="minioadmin",
bucket="test-bucket",
)
adapter = MinioAdapter(config=config)
with pytest.raises(ConnectionError): with pytest.raises(ConnectionError):
adapter.connect() adapter.connect()
assert not adapter.is_connected() assert not adapter.is_connected()
@@ -142,38 +120,35 @@ def test_should_raise_connection_error_when_unable_to_connect(
def test_should_log_info_when_creating_expected_bucket( def test_should_log_info_when_creating_expected_bucket(
raw_minio_client: Minio, raw_minio_client: Minio,
minio_config: MinioConfig,
caplog: pytest.LogCaptureFixture, caplog: pytest.LogCaptureFixture,
) -> None: ) -> None:
"""Test that the MinioAdapter logs info when creating the expected bucket.""" """Test that the MinioAdapter logs info when creating the expected bucket."""
# Arrange bucket_name = minio_config.bucket
bucket_name = str(os.getenv("MINIO_BUCKET"))
raw_minio_client.remove_bucket(bucket_name) raw_minio_client.remove_bucket(bucket_name)
adapter = MinioAdapter() adapter = MinioAdapter(config=minio_config)
# Act
with caplog.at_level(logging.INFO): with caplog.at_level(logging.INFO):
adapter.connect() adapter.connect()
# Assert
assert f"Creating bucket '{bucket_name}'" in caplog.text assert f"Creating bucket '{bucket_name}'" in caplog.text
def test_should_log_error_on_exception_during_exit( def test_should_log_error_on_exception_during_exit(
minio_container: dict[str, str], minio_config: MinioConfig,
caplog: pytest.LogCaptureFixture, caplog: pytest.LogCaptureFixture,
) -> None: ) -> None:
"""Test that the MinioAdapter logs an error if an exception occurs during __exit__.""" """Test that the MinioAdapter logs an error if an exception occurs during __exit__."""
try: try:
with MinioAdapter() as adapter: with MinioAdapter(config=minio_config) as adapter:
assert adapter.is_connected() assert adapter.is_connected()
raise ValueError("Simulated error") raise ValueError("Simulated error")
except ValueError: except ValueError:
pass # Expected pass
# Assert error was logged
assert "Error while exiting context" in caplog.text assert "Error while exiting context" in caplog.text
def test_should_have_context_manager() -> None: def test_should_have_context_manager(minio_config: MinioConfig) -> None:
"""Test that the MinioAdapter can be used as a context manager.""" """Test that the MinioAdapter can be used as a context manager."""
with MinioAdapter() as adapter: with MinioAdapter(config=minio_config) as adapter:
assert adapter._client is not None assert adapter._client is not None
assert adapter._client is None assert adapter._client is None
@@ -183,11 +158,8 @@ def test_should_get_data(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter can get data from a bucket.""" """Test that the MinioAdapter can get data from a bucket."""
# Arrange
object_name, expected_data = data_in_minio object_name, expected_data = data_in_minio
# Act
received_data = minio_adapter.get(object_name) received_data = minio_adapter.get(object_name)
# Assert
assert received_data is not None assert received_data is not None
assert same_data(expected_data, received_data) assert same_data(expected_data, received_data)
@@ -196,11 +168,7 @@ def test_should_get_none_for_nonexistent_object(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter returns None for a nonexistent object.""" """Test that the MinioAdapter returns None for a nonexistent object."""
# Arrange received_data = minio_adapter.get("nonexistent_object")
object_name = "nonexistent_object"
# Act
received_data = minio_adapter.get(object_name)
# Assert
assert received_data is None assert received_data is None
@@ -208,21 +176,17 @@ def test_should_raise_value_error_on_invalid_get_object_name(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter raises ValueError when getting with an invalid object name.""" """Test that the MinioAdapter raises ValueError when getting with an invalid object name."""
# Arrange
invalid_object_names = ["", 123, None] invalid_object_names = ["", 123, None]
# Act & Assert
for object_name in invalid_object_names: for object_name in invalid_object_names:
with pytest.raises(ValueError): with pytest.raises(ValueError):
minio_adapter.get(object_name) # type: ignore minio_adapter.get(object_name) # type: ignore
def test_should_raise_connection_error_on_get_when_not_connected( def test_should_raise_connection_error_on_get_when_not_connected(
minio_adapter: MinioAdapter, minio_config: MinioConfig,
) -> None: ) -> None:
"""Test that the MinioAdapter raises ConnectionError when getting while not connected.""" """Test that the MinioAdapter raises ConnectionError when getting while not connected."""
# Arrange adapter = MinioAdapter(config=minio_config)
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError): with pytest.raises(ConnectionError):
adapter.get("some_object") adapter.get("some_object")
@@ -231,10 +195,9 @@ def test_should_log_warning_when_getting_nonexistent_object(
caplog: pytest.LogCaptureFixture, caplog: pytest.LogCaptureFixture,
) -> None: ) -> None:
"""Test that the MinioAdapter logs a warning when getting a nonexistent object.""" """Test that the MinioAdapter logs a warning when getting a nonexistent object."""
# Arrange mock_client = MagicMock(spec=Minio)
adapter = MinioAdapter() mock_client.bucket_exists.return_value = True
adapter._client = MagicMock(spec=Minio) mock_client.get_object.side_effect = S3Error(
adapter._client.get_object.side_effect = S3Error(
MagicMock(spec=BaseHTTPResponse), MagicMock(spec=BaseHTTPResponse),
"NoSuchKey", "NoSuchKey",
"", "",
@@ -244,12 +207,10 @@ def test_should_log_warning_when_getting_nonexistent_object(
bucket_name="test-bucket", bucket_name="test-bucket",
object_name="missing-object", object_name="missing-object",
) )
adapter._bucket_name = "test-bucket" adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
object_name = "missing-object" object_name = "missing-object"
# Act
with caplog.at_level("WARNING"): with caplog.at_level("WARNING"):
result = adapter.get(object_name) result = adapter.get(object_name)
# Assert
assert result is None assert result is None
assert ( assert (
f"Object '{object_name}' not found in bucket '{adapter._bucket_name}'" f"Object '{object_name}' not found in bucket '{adapter._bucket_name}'"
@@ -260,10 +221,9 @@ def test_should_log_warning_when_getting_nonexistent_object(
def test_should_log_error_when_getting_with_s3error_other_than_no_such_key( def test_should_log_error_when_getting_with_s3error_other_than_no_such_key(
caplog: pytest.LogCaptureFixture, caplog: pytest.LogCaptureFixture,
) -> None: ) -> None:
"""Test that the MinioAdapter logs an error when getting a nonexistent object.""" """Test that the MinioAdapter logs an error for unhandled S3 errors."""
# Arrange mock_client = MagicMock(spec=Minio)
adapter = MinioAdapter() mock_client.bucket_exists.return_value = True
adapter._client = MagicMock(spec=Minio)
other_s3error = S3Error( other_s3error = S3Error(
MagicMock(spec=BaseHTTPResponse), MagicMock(spec=BaseHTTPResponse),
"UnhandledError", "UnhandledError",
@@ -274,13 +234,10 @@ def test_should_log_error_when_getting_with_s3error_other_than_no_such_key(
bucket_name="test-bucket", bucket_name="test-bucket",
object_name="missing-object", object_name="missing-object",
) )
adapter._client.get_object.side_effect = other_s3error mock_client.get_object.side_effect = other_s3error
adapter._bucket_name = "test-bucket" adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
object_name = "missing-object"
# Act
with caplog.at_level("ERROR"): with caplog.at_level("ERROR"):
result = adapter.get(object_name) result = adapter.get("missing-object")
# Assert
assert result is None assert result is None
assert repr(other_s3error) in caplog.text assert repr(other_s3error) in caplog.text
@@ -288,18 +245,14 @@ def test_should_log_error_when_getting_with_s3error_other_than_no_such_key(
def test_should_log_error_when_getting_with_general_exception( def test_should_log_error_when_getting_with_general_exception(
caplog: pytest.LogCaptureFixture, caplog: pytest.LogCaptureFixture,
) -> None: ) -> None:
"""Test that the MinioAdapter logs an error when getting a nonexistent object.""" """Test that the MinioAdapter logs an error on general exceptions during get."""
# Arrange mock_client = MagicMock(spec=Minio)
adapter = MinioAdapter() mock_client.bucket_exists.return_value = True
adapter._client = MagicMock(spec=Minio)
general_exception = Exception("General failure") general_exception = Exception("General failure")
adapter._client.get_object.side_effect = general_exception mock_client.get_object.side_effect = general_exception
adapter._bucket_name = "test-bucket" adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
object_name = "missing-object"
# Act
with caplog.at_level("ERROR"): with caplog.at_level("ERROR"):
result = adapter.get(object_name) result = adapter.get("missing-object")
# Assert
assert result is None assert result is None
assert repr(general_exception) in caplog.text assert repr(general_exception) in caplog.text
@@ -307,21 +260,17 @@ def test_should_log_error_when_getting_with_general_exception(
def test_should_put_data( def test_should_put_data(
data: BytesIO, data: BytesIO,
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
minio_config: MinioConfig,
) -> None: ) -> None:
"""Test that the MinioAdapter can put data into a bucket.""" """Test that the MinioAdapter can put data into a bucket."""
# Arrange
object_name = "new_test_object" object_name = "new_test_object"
received_data = minio_adapter.get(object_name) received_data = minio_adapter.get(object_name)
assert received_data is None # ensure object does not exist yet assert received_data is None
# Act
minio_adapter.put(object_name, data) minio_adapter.put(object_name, data)
# Assert
received_data = minio_adapter.get(object_name) received_data = minio_adapter.get(object_name)
assert received_data is not None assert received_data is not None
assert same_data(data, received_data) assert same_data(data, received_data)
# Cleanup minio_adapter._client.remove_object(minio_config.bucket, object_name) # type: ignore[union-attr]
bucket_name = str(os.getenv("MINIO_BUCKET"))
minio_adapter._client.remove_object(bucket_name, object_name) # type: ignore
def test_should_update_data( def test_should_update_data(
@@ -329,15 +278,12 @@ def test_should_update_data(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter can update data in a bucket.""" """Test that the MinioAdapter can update data in a bucket."""
# Arrange
object_name, _ = data_in_minio object_name, _ = data_in_minio
new_data = BytesIO(random.randbytes(2**21)) # 2 MiB new_data = BytesIO(random.randbytes(2**21))
received_data = minio_adapter.get(object_name) received_data = minio_adapter.get(object_name)
assert received_data is not None assert received_data is not None
assert not same_data(received_data, new_data) assert not same_data(received_data, new_data)
# Act
minio_adapter.put(object_name, new_data) minio_adapter.put(object_name, new_data)
# Assert
received_data = minio_adapter.get(object_name) received_data = minio_adapter.get(object_name)
assert received_data is not None assert received_data is not None
assert same_data(new_data, received_data) assert same_data(new_data, received_data)
@@ -348,9 +294,7 @@ def test_should_raise_value_error_on_invalid_put_object_name(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter raises ValueError when putting with an invalid object name.""" """Test that the MinioAdapter raises ValueError when putting with an invalid object name."""
# Arrange
invalid_object_names = ["", 123, None] invalid_object_names = ["", 123, None]
# Act & Assert
for object_name in invalid_object_names: for object_name in invalid_object_names:
with pytest.raises(ValueError): with pytest.raises(ValueError):
minio_adapter.put(object_name, data) # type: ignore minio_adapter.put(object_name, data) # type: ignore
@@ -360,13 +304,11 @@ def test_should_raise_value_error_on_invalid_put_data(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter raises ValueError when putting with invalid data.""" """Test that the MinioAdapter raises ValueError when putting with invalid data."""
# Arrange
object_name = "valid_object_name" object_name = "valid_object_name"
invalid_data = ["not_bytesio", 123, None] invalid_data = ["not_bytesio", 123, None]
# Act & Assert for invalid in invalid_data:
for data in invalid_data:
with pytest.raises(ValueError): with pytest.raises(ValueError):
minio_adapter.put(object_name, data) # type: ignore minio_adapter.put(object_name, invalid) # type: ignore
def test_should_raise_value_error_on_invalid_put_content_type( def test_should_raise_value_error_on_invalid_put_content_type(
@@ -374,23 +316,19 @@ def test_should_raise_value_error_on_invalid_put_content_type(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter raises ValueError when putting with an invalid content type.""" """Test that the MinioAdapter raises ValueError when putting with an invalid content type."""
# Arrange
object_name = "valid_object_name" object_name = "valid_object_name"
invalid_content_types = ["", 123, None] invalid_content_types = ["", 123, None]
# Act & Assert
for content_type in invalid_content_types: for content_type in invalid_content_types:
with pytest.raises(ValueError): with pytest.raises(ValueError):
minio_adapter.put(object_name, data, content_type) # type: ignore minio_adapter.put(object_name, data, content_type) # type: ignore
def test_should_raise_connection_error_on_put_when_not_connected( def test_should_raise_connection_error_on_put_when_not_connected(
minio_config: MinioConfig,
data: BytesIO, data: BytesIO,
minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter raises ConnectionError when putting while not connected.""" """Test that the MinioAdapter raises ConnectionError when putting while not connected."""
# Arrange adapter = MinioAdapter(config=minio_config)
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError): with pytest.raises(ConnectionError):
adapter.put("some_object", data) adapter.put("some_object", data)
@@ -400,36 +338,28 @@ def test_should_delete_object(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter can delete an object from a bucket.""" """Test that the MinioAdapter can delete an object from a bucket."""
# Arrange
object_name, _ = data_in_minio object_name, _ = data_in_minio
received_data = minio_adapter.get(object_name) received_data = minio_adapter.get(object_name)
assert received_data is not None # ensure object exists assert received_data is not None
# Act
minio_adapter.delete(object_name) minio_adapter.delete(object_name)
# Assert assert minio_adapter.get(object_name) is None
received_data = minio_adapter.get(object_name)
assert received_data is None
def test_should_raise_value_error_on_invalid_delete_object_name( def test_should_raise_value_error_on_invalid_delete_object_name(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter raises ValueError when deleting with an invalid object name.""" """Test that the MinioAdapter raises ValueError when deleting with an invalid object name."""
# Arrange
invalid_object_names = ["", 123, None] invalid_object_names = ["", 123, None]
# Act & Assert
for object_name in invalid_object_names: for object_name in invalid_object_names:
with pytest.raises(ValueError): with pytest.raises(ValueError):
minio_adapter.delete(object_name) # type: ignore minio_adapter.delete(object_name) # type: ignore
def test_should_raise_connection_error_on_delete_when_not_connected( def test_should_raise_connection_error_on_delete_when_not_connected(
minio_adapter: MinioAdapter, minio_config: MinioConfig,
) -> None: ) -> None:
"""Test that the MinioAdapter raises ConnectionError when deleting while not connected.""" """Test that the MinioAdapter raises ConnectionError when deleting while not connected."""
# Arrange adapter = MinioAdapter(config=minio_config)
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError): with pytest.raises(ConnectionError):
adapter.delete("some_object") adapter.delete("some_object")
@@ -439,14 +369,11 @@ def test_should_list_objects(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter can list objects in a bucket.""" """Test that the MinioAdapter can list objects in a bucket."""
# Arrange
object_name, _ = data_in_minio object_name, _ = data_in_minio
new_data = BytesIO(random.randbytes(2**21)) # 2 MiB new_data = BytesIO(random.randbytes(2**21))
new_data_name = "another_test_object" new_data_name = "another_test_object"
minio_adapter.put(new_data_name, new_data) minio_adapter.put(new_data_name, new_data)
# Act
objects = minio_adapter.list_objects() objects = minio_adapter.list_objects()
# Assert
assert isinstance(objects, list) assert isinstance(objects, list)
assert len(objects) == 2 assert len(objects) == 2
assert object_name in objects assert object_name in objects
@@ -458,15 +385,12 @@ def test_should_list_objects_with_prefix(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter can list objects in a bucket with a prefix.""" """Test that the MinioAdapter can list objects in a bucket with a prefix."""
# Arrange
object_name, _ = data_in_minio object_name, _ = data_in_minio
new_data = BytesIO(random.randbytes(2**21)) # 2 MiB new_data = BytesIO(random.randbytes(2**21))
new_data_name = "prefix_test_object" new_data_name = "prefix_test_object"
minio_adapter.put(new_data_name, new_data) minio_adapter.put(new_data_name, new_data)
prefix = "prefix_" prefix = "prefix_"
# Act
objects = minio_adapter.list_objects(prefix) objects = minio_adapter.list_objects(prefix)
# Assert
assert isinstance(objects, list) assert isinstance(objects, list)
assert len(objects) == 1 assert len(objects) == 1
assert new_data_name in objects assert new_data_name in objects
@@ -477,25 +401,20 @@ def test_should_raise_value_error_on_invalid_list_objects_prefix(
minio_adapter: MinioAdapter, minio_adapter: MinioAdapter,
) -> None: ) -> None:
"""Test that the MinioAdapter raises ValueError when listing with an invalid prefix.""" """Test that the MinioAdapter raises ValueError when listing with an invalid prefix."""
# Arrange
invalid_prefixes = [123, None] invalid_prefixes = [123, None]
# Act & Assert
for prefix in invalid_prefixes: for prefix in invalid_prefixes:
with pytest.raises(ValueError): with pytest.raises(ValueError):
minio_adapter.list_objects(prefix) # type: ignore minio_adapter.list_objects(prefix) # type: ignore
def test_should_raise_connection_error_on_list_objects_when_not_connected( def test_should_raise_connection_error_on_list_objects_when_not_connected(
minio_adapter: MinioAdapter, minio_config: MinioConfig,
) -> None: ) -> None:
"""Test that the MinioAdapter raises ConnectionError when listing while not connected.""" """Test that the MinioAdapter raises ConnectionError when listing while not connected."""
# Arrange adapter = MinioAdapter(config=minio_config)
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError): with pytest.raises(ConnectionError):
adapter.list_objects() adapter.list_objects()
# allows local debugging by running file as script
if __name__ == "__main__": if __name__ == "__main__":
pytest.main(["-s", "-v", __file__]) pytest.main(["-s", "-v", __file__])
+27 -92
View File
@@ -1,11 +1,15 @@
"""Integration tests for the RedisAdapter.""" """Integration tests for the RedisAdapter."""
from collections.abc import Generator from collections.abc import Generator
import pytest import pytest
import redis import redis
from redis.commands.json.path import Path as RedisPath from redis.commands.json.path import Path as RedisPath
from python_repositories.adapters.redis_adapter import RedisAdapter from python_repositories.adapters.redis_adapter import RedisAdapter
from python_repositories.interfaces import JsonRepositoryInterface from python_repositories.config import RedisConfig
pytestmark = pytest.mark.integration
@pytest.fixture(scope="module") @pytest.fixture(scope="module")
@@ -26,14 +30,13 @@ def data_in_redis(
yield key, data yield key, data
# Cleanup
raw_redis_client.delete(key) raw_redis_client.delete(key)
@pytest.fixture(scope="module") @pytest.fixture(scope="module")
def redis_adapter(redis_container: str) -> Generator[RedisAdapter, None, None]: def redis_adapter(redis_config: RedisConfig) -> Generator[RedisAdapter, None, None]:
"""Fixture to provide a connected RedisAdapter instance.""" """Fixture to provide a connected RedisAdapter instance."""
adapter = RedisAdapter() adapter = RedisAdapter(config=redis_config)
adapter.connect() adapter.connect()
yield adapter yield adapter
adapter.disconnect() adapter.disconnect()
@@ -42,38 +45,12 @@ def redis_adapter(redis_container: str) -> Generator[RedisAdapter, None, None]:
@pytest.fixture(scope="function", autouse=True) @pytest.fixture(scope="function", autouse=True)
def clear_redis(raw_redis_client: redis.Redis) -> None: def clear_redis(raw_redis_client: redis.Redis) -> None:
"""Fixture to clear all Redis keys before each test.""" """Fixture to clear all Redis keys before each test."""
# Clear all keys before each test
raw_redis_client.flushall() raw_redis_client.flushall()
def test_should_adhere_to_interface(redis_container: str) -> None: def test_should_raise_connection_error_when_unable_to_connect() -> None:
"""Test that the RedisAdapter adheres to the expected interface."""
assert issubclass(RedisAdapter, JsonRepositoryInterface)
_ = RedisAdapter()
def test_should_have_logger_when_instantiated(redis_container: str) -> None:
"""Test that the RedisAdapter has a logger when instantiated."""
adapter = RedisAdapter()
assert hasattr(adapter, "logger")
assert adapter.logger is not None
def test_should_not_be_connected_when_instantiated(redis_container: str) -> None:
"""Test that the RedisAdapter is not connected when instantiated."""
adapter = RedisAdapter()
assert adapter._client is None
assert not adapter.is_connected()
def test_should_raise_connection_error_when_unable_to_connect(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test that the RedisAdapter raises ConnectionError when unable to connect.""" """Test that the RedisAdapter raises ConnectionError when unable to connect."""
# Arrange adapter = RedisAdapter(config=RedisConfig(uri="redis://invalid:6379"))
monkeypatch.setenv("REDIS_URI", "redis://invalid:6379")
adapter = RedisAdapter()
# Act & Assert
with pytest.raises(ConnectionError): with pytest.raises(ConnectionError):
adapter.connect() adapter.connect()
assert adapter._client is None assert adapter._client is None
@@ -84,42 +61,37 @@ def test_connect_raises_connection_error_when_unable_to_ping(
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
) -> None: ) -> None:
"""Test that the RedisAdapter raises ConnectionError when ping fails.""" """Test that the RedisAdapter raises ConnectionError when ping fails."""
# Set an invalid URI
monkeypatch.setenv("REDIS_URI", "redis://invalid:6379")
# Monkeypatch redis.Redis.from_url to return a mock client
class MockRedis: class MockRedis:
"""A mock Redis client that simulates a failed ping.""" """A mock Redis client that simulates a failed ping."""
def ping(self) -> bool: def ping(self) -> bool:
"""Simulate a failed ping.""" return False
return False # Simulate failed ping
monkeypatch.setattr("redis.Redis.from_url", lambda *a, **kw: MockRedis()) monkeypatch.setattr("redis.Redis.from_url", lambda *a, **kw: MockRedis())
adapter = RedisAdapter() adapter = RedisAdapter(config=RedisConfig(uri="redis://invalid:6379"))
with pytest.raises(ConnectionError, match="Could not connect to Redis"): with pytest.raises(ConnectionError, match="Could not connect to Redis"):
adapter.connect() adapter.connect()
def test_should_log_error_on_exception_during_exit( def test_should_log_error_on_exception_during_exit(
redis_container: str, redis_config: RedisConfig,
caplog: pytest.LogCaptureFixture, caplog: pytest.LogCaptureFixture,
) -> None: ) -> None:
"""Test that the RedisAdapter logs an error when an exception occurs during context exit.""" """Test that the RedisAdapter logs an error when an exception occurs during context exit."""
try: try:
with RedisAdapter() as adapter: with RedisAdapter(config=redis_config) as adapter:
assert adapter.is_connected() assert adapter.is_connected()
raise ValueError("Simulated error") raise ValueError("Simulated error")
except ValueError: except ValueError:
pass # Expected pass
# Assert error was logged
assert "Error while exiting context" in caplog.text assert "Error while exiting context" in caplog.text
def test_should_have_context_manager(redis_container: str) -> None: def test_should_have_context_manager(redis_config: RedisConfig) -> None:
"""Test that the RedisAdapter can be used as a context manager.""" """Test that the RedisAdapter can be used as a context manager."""
with RedisAdapter() as adapter: with RedisAdapter(config=redis_config) as adapter:
assert adapter._client is not None assert adapter._client is not None
assert adapter._client is None assert adapter._client is None
@@ -129,11 +101,8 @@ def test_should_get_value(
redis_adapter: RedisAdapter, redis_adapter: RedisAdapter,
) -> None: ) -> None:
"""Test that the RedisAdapter can get a value.""" """Test that the RedisAdapter can get a value."""
# Arrange
key, data = data_in_redis key, data = data_in_redis
# Act
value = redis_adapter.get(key) value = redis_adapter.get(key)
# Assert
assert value is not None assert value is not None
assert value == data assert value == data
@@ -142,9 +111,7 @@ def test_should_get_none_for_missing_key(
redis_adapter: RedisAdapter, redis_adapter: RedisAdapter,
) -> None: ) -> None:
"""Test that getting a non-existent key returns None.""" """Test that getting a non-existent key returns None."""
# Act
value = redis_adapter.get("nonexistent_key") value = redis_adapter.get("nonexistent_key")
# Assert
assert value is None assert value is None
@@ -152,21 +119,17 @@ def test_should_raise_value_error_on_invalid_get_key(
redis_adapter: RedisAdapter, redis_adapter: RedisAdapter,
) -> None: ) -> None:
"""Test that the RedisAdapter raises ValueError when getting with an invalid key.""" """Test that the RedisAdapter raises ValueError when getting with an invalid key."""
# Arrange
invalid_keys = ["", 123, None] invalid_keys = ["", 123, None]
# Act & Assert
for key in invalid_keys: for key in invalid_keys:
with pytest.raises(ValueError): with pytest.raises(ValueError):
redis_adapter.get(key) # type: ignore redis_adapter.get(key) # type: ignore
def test_should_raise_connection_error_on_get_when_not_connected( def test_should_raise_connection_error_on_get_when_not_connected(
redis_adapter: RedisAdapter, redis_config: RedisConfig,
) -> None: ) -> None:
"""Test that the RedisAdapter raises ConnectionError when getting while not connected.""" """Test that the RedisAdapter raises ConnectionError when getting while not connected."""
# Arrange adapter = RedisAdapter(config=redis_config)
adapter = RedisAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError): with pytest.raises(ConnectionError):
adapter.get("some_key") adapter.get("some_key")
@@ -176,13 +139,10 @@ def test_should_set_value(
redis_adapter: RedisAdapter, redis_adapter: RedisAdapter,
) -> None: ) -> None:
"""Test that the RedisAdapter can set a value.""" """Test that the RedisAdapter can set a value."""
# Arrange
key = "test_key" key = "test_key"
received_data = redis_adapter.get(key) received_data = redis_adapter.get(key)
assert received_data is None # Ensure key does not exist assert received_data is None
# Act
redis_adapter.set(key, data) redis_adapter.set(key, data)
# Assert
received_data = redis_adapter.get(key) received_data = redis_adapter.get(key)
assert received_data is not None assert received_data is not None
assert received_data == data assert received_data == data
@@ -193,15 +153,12 @@ def test_should_update_value(
redis_adapter: RedisAdapter, redis_adapter: RedisAdapter,
) -> None: ) -> None:
"""Test that the RedisAdapter can update an existing value.""" """Test that the RedisAdapter can update an existing value."""
# Arrange
key, _ = data_in_redis key, _ = data_in_redis
new_data = {"new_key": "new_value"} new_data = {"new_key": "new_value"}
received_data = redis_adapter.get(key) received_data = redis_adapter.get(key)
assert received_data is not None assert received_data is not None
assert received_data != new_data assert received_data != new_data
# Act
redis_adapter.set(key, new_data) redis_adapter.set(key, new_data)
# Assert
assert redis_adapter.get(key) == new_data assert redis_adapter.get(key) == new_data
@@ -210,9 +167,7 @@ def test_should_raise_value_error_on_invalid_set_key(
data: dict[str, str], data: dict[str, str],
) -> None: ) -> None:
"""Test that the RedisAdapter raises ValueError when setting with an invalid key.""" """Test that the RedisAdapter raises ValueError when setting with an invalid key."""
# Arrange
invalid_keys = ["", 123, None] invalid_keys = ["", 123, None]
# Act & Assert
for key in invalid_keys: for key in invalid_keys:
with pytest.raises(ValueError): with pytest.raises(ValueError):
redis_adapter.set(key, data) # type: ignore redis_adapter.set(key, data) # type: ignore
@@ -222,26 +177,21 @@ def test_should_raise_value_error_on_invalid_set_data(
redis_adapter: RedisAdapter, redis_adapter: RedisAdapter,
) -> None: ) -> None:
"""Test that the RedisAdapter raises ValueError when setting with invalid data.""" """Test that the RedisAdapter raises ValueError when setting with invalid data."""
# Arrange
key = "test_key" key = "test_key"
invalid_data = ["", 123, None, [], {}] invalid_data = ["", 123, None, [], {}]
# Act & Assert
for data in invalid_data: for data in invalid_data:
with pytest.raises(ValueError): with pytest.raises(ValueError):
redis_adapter.set(key, data) # type: ignore redis_adapter.set(key, data) # type: ignore
def test_should_raise_connection_error_on_set_when_not_connected( def test_should_raise_connection_error_on_set_when_not_connected(
redis_adapter: RedisAdapter, redis_config: RedisConfig,
data: dict[str, str], data: dict[str, str],
) -> None: ) -> None:
"""Test that the RedisAdapter raises ConnectionError when setting while not connected.""" """Test that the RedisAdapter raises ConnectionError when setting while not connected."""
# Arrange adapter = RedisAdapter(config=redis_config)
adapter = RedisAdapter()
key = "test_key"
# Act & Assert
with pytest.raises(ConnectionError): with pytest.raises(ConnectionError):
adapter.set(key, data) adapter.set("test_key", data)
def test_should_delete_key( def test_should_delete_key(
@@ -249,13 +199,10 @@ def test_should_delete_key(
redis_adapter: RedisAdapter, redis_adapter: RedisAdapter,
) -> None: ) -> None:
"""Test that deleting a key removes it from Redis.""" """Test that deleting a key removes it from Redis."""
# Arrange
key, _ = data_in_redis key, _ = data_in_redis
received_data = redis_adapter.get(key) received_data = redis_adapter.get(key)
assert received_data is not None # Ensure key exists assert received_data is not None
# Act
redis_adapter.delete(key) redis_adapter.delete(key)
# Assert
assert redis_adapter.get(key) is None assert redis_adapter.get(key) is None
@@ -263,21 +210,17 @@ def test_should_raise_value_error_on_invalid_delete_key(
redis_adapter: RedisAdapter, redis_adapter: RedisAdapter,
) -> None: ) -> None:
"""Test that the RedisAdapter raises ValueError when deleting with an invalid key.""" """Test that the RedisAdapter raises ValueError when deleting with an invalid key."""
# Arrange
invalid_keys = ["", 123, None] invalid_keys = ["", 123, None]
# Act & Assert
for key in invalid_keys: for key in invalid_keys:
with pytest.raises(ValueError): with pytest.raises(ValueError):
redis_adapter.delete(key) # type: ignore redis_adapter.delete(key) # type: ignore
def test_should_raise_connection_error_on_delete_when_not_connected( def test_should_raise_connection_error_on_delete_when_not_connected(
redis_adapter: RedisAdapter, redis_config: RedisConfig,
) -> None: ) -> None:
"""Test that the RedisAdapter raises ConnectionError when deleting while not connected.""" """Test that the RedisAdapter raises ConnectionError when deleting while not connected."""
# Arrange adapter = RedisAdapter(config=redis_config)
adapter = RedisAdapter()
# Act & Assert
with pytest.raises(ConnectionError): with pytest.raises(ConnectionError):
adapter.delete("some_key") adapter.delete("some_key")
@@ -286,12 +229,9 @@ def test_should_list_keys(
redis_adapter: RedisAdapter, redis_adapter: RedisAdapter,
) -> None: ) -> None:
"""Test listing keys matching a pattern returns correct keys.""" """Test listing keys matching a pattern returns correct keys."""
# Arrange
redis_adapter.set("key1", {"a": 1}) redis_adapter.set("key1", {"a": 1})
redis_adapter.set("key2", {"b": 2}) redis_adapter.set("key2", {"b": 2})
# Act
keys = redis_adapter.list_keys("key*") keys = redis_adapter.list_keys("key*")
# Assert
assert set(keys) == {"key1", "key2"} assert set(keys) == {"key1", "key2"}
@@ -299,25 +239,20 @@ def test_should_raise_value_error_on_invalid_list_keys_pattern(
redis_adapter: RedisAdapter, redis_adapter: RedisAdapter,
) -> None: ) -> None:
"""Test that the RedisAdapter raises ValueError when listing keys with an invalid pattern.""" """Test that the RedisAdapter raises ValueError when listing keys with an invalid pattern."""
# Arrange
invalid_patterns = ["", 123, None] invalid_patterns = ["", 123, None]
# Act & Assert
for pattern in invalid_patterns: for pattern in invalid_patterns:
with pytest.raises(ValueError): with pytest.raises(ValueError):
redis_adapter.list_keys(pattern) # type: ignore redis_adapter.list_keys(pattern) # type: ignore
def test_should_raise_connection_error_on_list_keys_when_not_connected( def test_should_raise_connection_error_on_list_keys_when_not_connected(
redis_adapter: RedisAdapter, redis_config: RedisConfig,
) -> None: ) -> None:
"""Test that the RedisAdapter raises ConnectionError when listing keys while not connected.""" """Test that the RedisAdapter raises ConnectionError when listing keys while not connected."""
# Arrange adapter = RedisAdapter(config=redis_config)
adapter = RedisAdapter()
# Act & Assert
with pytest.raises(ConnectionError): with pytest.raises(ConnectionError):
adapter.list_keys("some_pattern") adapter.list_keys("some_pattern")
# allows local debugging by running file as script
if __name__ == "__main__": if __name__ == "__main__":
pytest.main(["-s", "-v", __file__]) pytest.main(["-s", "-v", __file__])
@@ -1,4 +1,4 @@
"""Tests for optional dependency import behavior.""" """Unit tests for lazy adapter loading in adapters subpackage."""
from __future__ import annotations from __future__ import annotations
+27
View File
@@ -0,0 +1,27 @@
"""Unit test fixtures for mocked adapters."""
from __future__ import annotations
from unittest.mock import MagicMock
import pytest
import redis
from minio import Minio
from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.adapters.redis_adapter import RedisAdapter
from tests.conftest import TEST_MINIO_CONFIG, TEST_REDIS_CONFIG
@pytest.fixture
def redis_adapter() -> RedisAdapter:
"""Provide a RedisAdapter with an injected mock client."""
mock_client = MagicMock(spec=redis.Redis)
return RedisAdapter(config=TEST_REDIS_CONFIG, client=mock_client)
@pytest.fixture
def minio_adapter() -> MinioAdapter:
"""Provide a MinioAdapter with an injected mock client."""
mock_client = MagicMock(spec=Minio)
return MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
@@ -1,39 +1,25 @@
"""Tests for TTL-cached connection health checks on adapters.""" """Unit tests for TTL-cached connection health checks on ConnectionAwareAdapter."""
from __future__ import annotations from __future__ import annotations
from typing import cast
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
import pytest
import redis import redis
from python_repositories.adapters.minio_adapter import MinioAdapter from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.adapters.redis_adapter import RedisAdapter from python_repositories.adapters.redis_adapter import RedisAdapter
from tests.conftest import TEST_MINIO_CONFIG, TEST_REDIS_CONFIG
@pytest.fixture
def redis_adapter(monkeypatch: pytest.MonkeyPatch) -> RedisAdapter:
monkeypatch.setenv("REDIS_URI", "redis://localhost:6379")
return RedisAdapter()
@pytest.fixture
def minio_adapter(monkeypatch: pytest.MonkeyPatch) -> MinioAdapter:
monkeypatch.setenv("MINIO_ENDPOINT", "localhost:9000")
monkeypatch.setenv("MINIO_ACCESS_KEY", "minioadmin")
monkeypatch.setenv("MINIO_SECRET_KEY", "minioadmin")
monkeypatch.setenv("MINIO_BUCKET", "test-bucket")
return MinioAdapter()
class TestRedisConnectionHealth: class TestRedisConnectionHealth:
def test_not_connected_when_no_client(self, redis_adapter: RedisAdapter) -> None: def test_not_connected_when_no_client(self) -> None:
assert not redis_adapter.is_connected() adapter = RedisAdapter(config=TEST_REDIS_CONFIG)
assert not adapter.is_connected()
def test_connected_when_probe_succeeds(self, redis_adapter: RedisAdapter) -> None: def test_connected_when_probe_succeeds(self, redis_adapter: RedisAdapter) -> None:
mock_client = MagicMock(spec=redis.Redis) mock_client = cast(MagicMock, redis_adapter._client)
mock_client.ping.return_value = True mock_client.ping.return_value = True
redis_adapter._client = mock_client
assert redis_adapter.is_connected() assert redis_adapter.is_connected()
mock_client.ping.assert_called_once() mock_client.ping.assert_called_once()
@@ -41,16 +27,15 @@ class TestRedisConnectionHealth:
def test_stale_connection_when_probe_fails( def test_stale_connection_when_probe_fails(
self, redis_adapter: RedisAdapter self, redis_adapter: RedisAdapter
) -> None: ) -> None:
mock_client = MagicMock(spec=redis.Redis) cast(MagicMock, redis_adapter._client).ping.side_effect = redis.ConnectionError(
mock_client.ping.side_effect = redis.ConnectionError("connection lost") "connection lost"
redis_adapter._client = mock_client )
assert not redis_adapter.is_connected() assert not redis_adapter.is_connected()
def test_cache_hit_avoids_second_probe(self, redis_adapter: RedisAdapter) -> None: def test_cache_hit_avoids_second_probe(self, redis_adapter: RedisAdapter) -> None:
mock_client = MagicMock(spec=redis.Redis) mock_client = cast(MagicMock, redis_adapter._client)
mock_client.ping.return_value = True mock_client.ping.return_value = True
redis_adapter._client = mock_client
with patch( with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic", "python_repositories.adapters.connection_aware_adapter.time.monotonic",
@@ -62,9 +47,8 @@ class TestRedisConnectionHealth:
mock_client.ping.assert_called_once() mock_client.ping.assert_called_once()
def test_cache_miss_runs_probe_again(self, redis_adapter: RedisAdapter) -> None: def test_cache_miss_runs_probe_again(self, redis_adapter: RedisAdapter) -> None:
mock_client = MagicMock(spec=redis.Redis) mock_client = cast(MagicMock, redis_adapter._client)
mock_client.ping.return_value = True mock_client.ping.return_value = True
redis_adapter._client = mock_client
with patch( with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic", "python_repositories.adapters.connection_aware_adapter.time.monotonic",
@@ -76,9 +60,8 @@ class TestRedisConnectionHealth:
assert mock_client.ping.call_count == 2 assert mock_client.ping.call_count == 2
def test_disconnect_clears_cache(self, redis_adapter: RedisAdapter) -> None: def test_disconnect_clears_cache(self, redis_adapter: RedisAdapter) -> None:
mock_client = MagicMock(spec=redis.Redis) mock_client = cast(MagicMock, redis_adapter._client)
mock_client.ping.return_value = True mock_client.ping.return_value = True
redis_adapter._client = mock_client
with patch( with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic", "python_repositories.adapters.connection_aware_adapter.time.monotonic",
@@ -87,34 +70,35 @@ class TestRedisConnectionHealth:
assert redis_adapter.is_connected() assert redis_adapter.is_connected()
redis_adapter.disconnect() redis_adapter.disconnect()
redis_adapter._client = mock_client reinjected = RedisAdapter(
config=redis_adapter._config,
client=mock_client,
)
with patch( with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic", "python_repositories.adapters.connection_aware_adapter.time.monotonic",
return_value=100.0, return_value=100.0,
): ):
assert redis_adapter.is_connected() assert reinjected.is_connected()
assert mock_client.ping.call_count == 2 assert mock_client.ping.call_count == 2
class TestMinioConnectionHealth: class TestMinioConnectionHealth:
def test_not_connected_when_no_client(self, minio_adapter: MinioAdapter) -> None: def test_not_connected_when_no_client(self) -> None:
assert not minio_adapter.is_connected() adapter = MinioAdapter(config=TEST_MINIO_CONFIG)
assert not adapter.is_connected()
def test_not_connected_when_bucket_name_missing( def test_not_connected_when_bucket_name_missing(
self, minio_adapter: MinioAdapter self, minio_adapter: MinioAdapter
) -> None: ) -> None:
minio_adapter._client = MagicMock()
minio_adapter._bucket_name = None minio_adapter._bucket_name = None
assert not minio_adapter.is_connected() assert not minio_adapter.is_connected()
def test_connected_when_probe_succeeds(self, minio_adapter: MinioAdapter) -> None: def test_connected_when_probe_succeeds(self, minio_adapter: MinioAdapter) -> None:
mock_client = MagicMock() mock_client = cast(MagicMock, minio_adapter._client)
mock_client.bucket_exists.return_value = True mock_client.bucket_exists.return_value = True
minio_adapter._client = mock_client
minio_adapter._bucket_name = "test-bucket"
assert minio_adapter.is_connected() assert minio_adapter.is_connected()
mock_client.bucket_exists.assert_called_once_with("test-bucket") mock_client.bucket_exists.assert_called_once_with("test-bucket")
@@ -122,18 +106,15 @@ class TestMinioConnectionHealth:
def test_stale_connection_when_probe_fails( def test_stale_connection_when_probe_fails(
self, minio_adapter: MinioAdapter self, minio_adapter: MinioAdapter
) -> None: ) -> None:
mock_client = MagicMock() cast(MagicMock, minio_adapter._client).bucket_exists.side_effect = Exception(
mock_client.bucket_exists.side_effect = Exception("connection lost") "connection lost"
minio_adapter._client = mock_client )
minio_adapter._bucket_name = "test-bucket"
assert not minio_adapter.is_connected() assert not minio_adapter.is_connected()
def test_cache_hit_avoids_second_probe(self, minio_adapter: MinioAdapter) -> None: def test_cache_hit_avoids_second_probe(self, minio_adapter: MinioAdapter) -> None:
mock_client = MagicMock() mock_client = cast(MagicMock, minio_adapter._client)
mock_client.bucket_exists.return_value = True mock_client.bucket_exists.return_value = True
minio_adapter._client = mock_client
minio_adapter._bucket_name = "test-bucket"
with patch( with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic", "python_repositories.adapters.connection_aware_adapter.time.monotonic",
@@ -145,10 +126,8 @@ class TestMinioConnectionHealth:
mock_client.bucket_exists.assert_called_once() mock_client.bucket_exists.assert_called_once()
def test_cache_miss_runs_probe_again(self, minio_adapter: MinioAdapter) -> None: def test_cache_miss_runs_probe_again(self, minio_adapter: MinioAdapter) -> None:
mock_client = MagicMock() mock_client = cast(MagicMock, minio_adapter._client)
mock_client.bucket_exists.return_value = True mock_client.bucket_exists.return_value = True
minio_adapter._client = mock_client
minio_adapter._bucket_name = "test-bucket"
with patch( with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic", "python_repositories.adapters.connection_aware_adapter.time.monotonic",
@@ -160,10 +139,8 @@ class TestMinioConnectionHealth:
assert mock_client.bucket_exists.call_count == 2 assert mock_client.bucket_exists.call_count == 2
def test_disconnect_clears_cache(self, minio_adapter: MinioAdapter) -> None: def test_disconnect_clears_cache(self, minio_adapter: MinioAdapter) -> None:
mock_client = MagicMock() mock_client = cast(MagicMock, minio_adapter._client)
mock_client.bucket_exists.return_value = True mock_client.bucket_exists.return_value = True
minio_adapter._client = mock_client
minio_adapter._bucket_name = "test-bucket"
with patch( with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic", "python_repositories.adapters.connection_aware_adapter.time.monotonic",
@@ -172,13 +149,15 @@ class TestMinioConnectionHealth:
assert minio_adapter.is_connected() assert minio_adapter.is_connected()
minio_adapter.disconnect() minio_adapter.disconnect()
minio_adapter._client = mock_client reinjected = MinioAdapter(
minio_adapter._bucket_name = "test-bucket" config=minio_adapter._config,
client=mock_client,
)
with patch( with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic", "python_repositories.adapters.connection_aware_adapter.time.monotonic",
return_value=100.0, return_value=100.0,
): ):
assert minio_adapter.is_connected() assert reinjected.is_connected()
assert mock_client.bucket_exists.call_count == 2 assert mock_client.bucket_exists.call_count == 2
@@ -1,4 +1,4 @@
"""Integration tests for ConnectionAwareInterface.""" """Unit tests for ConnectionAwareInterface."""
import pytest import pytest
from python_repositories.interfaces.connection_aware_interface import ( from python_repositories.interfaces.connection_aware_interface import (
@@ -1,4 +1,4 @@
"""Integration tests for ContextAwareInterface.""" """Unit tests for ContextAwareInterface."""
from __future__ import annotations from __future__ import annotations
+26
View File
@@ -0,0 +1,26 @@
"""Unit tests for dotenv_loader."""
from __future__ import annotations
import os
from pathlib import Path
import pytest
from python_repositories.config.dotenv_loader import load_dotenv
def test_load_dotenv_loads_file(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
env_file = tmp_path / ".env"
env_file.write_text("DOTENV_TEST_VAR=loaded_value\n")
monkeypatch.delenv("DOTENV_TEST_VAR", raising=False)
loaded = load_dotenv(env_file)
assert loaded is True
assert os.getenv("DOTENV_TEST_VAR") == "loaded_value"
def test_load_dotenv_returns_false_for_missing_file(tmp_path: Path) -> None:
missing = tmp_path / "missing.env"
assert load_dotenv(missing) is False
@@ -1,4 +1,4 @@
"""Integration tests for JsonRepositoryInterface.""" """Unit tests for JsonRepositoryInterface."""
import pytest import pytest
from python_repositories.interfaces.json_repository_interface import ( from python_repositories.interfaces.json_repository_interface import (
+55
View File
@@ -0,0 +1,55 @@
"""Unit tests for MinioAdapter instantiation and injection."""
from __future__ import annotations
from unittest.mock import MagicMock
import pytest
from minio import Minio
from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.interfaces import ObjectRepositoryInterface
from tests.conftest import TEST_MINIO_CONFIG
def test_should_adhere_to_interface() -> None:
assert issubclass(MinioAdapter, ObjectRepositoryInterface)
_ = MinioAdapter(config=TEST_MINIO_CONFIG)
def test_should_have_logger_when_instantiated() -> None:
adapter = MinioAdapter(config=TEST_MINIO_CONFIG)
assert hasattr(adapter, "logger")
assert adapter.logger is not None
def test_should_not_be_connected_when_instantiated() -> None:
adapter = MinioAdapter(config=TEST_MINIO_CONFIG)
assert not adapter.is_connected()
def test_constructs_with_injected_config_without_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
for var in (
"MINIO_ENDPOINT",
"MINIO_ACCESS_KEY",
"MINIO_SECRET_KEY",
"MINIO_BUCKET",
):
monkeypatch.delenv(var, raising=False)
adapter = MinioAdapter(config=TEST_MINIO_CONFIG)
assert adapter._config == TEST_MINIO_CONFIG
def test_injected_client_sets_bucket_name() -> None:
mock_client = MagicMock(spec=Minio)
adapter = MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
assert adapter._bucket_name == "test-bucket"
assert adapter._client is mock_client
def test_raises_when_client_provided_without_config() -> None:
mock_client = MagicMock(spec=Minio)
with pytest.raises(ValueError, match="config is required"):
MinioAdapter(client=mock_client)
+73
View File
@@ -0,0 +1,73 @@
"""Unit tests for MinioConfig."""
from __future__ import annotations
import pytest
from python_repositories.config import MinioConfig
def _set_required_minio_env(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("MINIO_ENDPOINT", "localhost:9000")
monkeypatch.setenv("MINIO_ACCESS_KEY", "access")
monkeypatch.setenv("MINIO_SECRET_KEY", "secret")
monkeypatch.setenv("MINIO_BUCKET", "my-bucket")
def test_from_env_loads_all_fields(monkeypatch: pytest.MonkeyPatch) -> None:
_set_required_minio_env(monkeypatch)
config = MinioConfig.from_env(use_dotenv=False)
assert config.endpoint == "localhost:9000"
assert config.access_key == "access"
assert config.secret_key == "secret"
assert config.bucket == "my-bucket"
assert config.secure is True
def test_from_env_defaults_secure_to_true_when_unset(
monkeypatch: pytest.MonkeyPatch,
) -> None:
_set_required_minio_env(monkeypatch)
monkeypatch.delenv("MINIO_SECURE", raising=False)
config = MinioConfig.from_env(use_dotenv=False)
assert config.secure is True
@pytest.mark.parametrize(
("value", "expected"),
[
("true", True),
("1", True),
("yes", True),
("false", False),
("0", False),
("no", False),
],
)
def test_from_env_parses_secure(
monkeypatch: pytest.MonkeyPatch,
value: str,
expected: bool,
) -> None:
_set_required_minio_env(monkeypatch)
monkeypatch.setenv("MINIO_SECURE", value)
config = MinioConfig.from_env(use_dotenv=False)
assert config.secure is expected
def test_from_env_raises_for_invalid_secure(monkeypatch: pytest.MonkeyPatch) -> None:
_set_required_minio_env(monkeypatch)
monkeypatch.setenv("MINIO_SECURE", "not-a-bool")
with pytest.raises(ValueError, match="MINIO_SECURE"):
MinioConfig.from_env(use_dotenv=False)
def test_from_env_raises_when_endpoint_missing(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("MINIO_ENDPOINT", raising=False)
monkeypatch.setenv("MINIO_ACCESS_KEY", "access")
monkeypatch.setenv("MINIO_SECRET_KEY", "secret")
monkeypatch.setenv("MINIO_BUCKET", "my-bucket")
with pytest.raises(Exception):
MinioConfig.from_env(use_dotenv=False)
@@ -1,4 +1,4 @@
"""Integration tests for ObjectRepositoryInterface.""" """Unit tests for ObjectRepositoryInterface."""
from io import BytesIO from io import BytesIO
+71
View File
@@ -0,0 +1,71 @@
"""Unit tests for RedisAdapter instantiation and injection."""
from __future__ import annotations
from unittest.mock import MagicMock
import pytest
import redis
from python_repositories.adapters.redis_adapter import RedisAdapter
from python_repositories.interfaces import JsonRepositoryInterface
from tests.conftest import TEST_REDIS_CONFIG
def test_should_adhere_to_interface() -> None:
assert issubclass(RedisAdapter, JsonRepositoryInterface)
_ = RedisAdapter(config=TEST_REDIS_CONFIG)
def test_should_have_logger_when_instantiated() -> None:
adapter = RedisAdapter(config=TEST_REDIS_CONFIG)
assert hasattr(adapter, "logger")
assert adapter.logger is not None
def test_should_not_be_connected_when_instantiated() -> None:
adapter = RedisAdapter(config=TEST_REDIS_CONFIG)
assert adapter._client is None
assert not adapter.is_connected()
def test_constructs_with_injected_config_without_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("REDIS_URI", raising=False)
adapter = RedisAdapter(config=TEST_REDIS_CONFIG)
assert adapter._config == TEST_REDIS_CONFIG
def test_constructs_with_injected_client_without_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("REDIS_URI", raising=False)
mock_client = MagicMock(spec=redis.Redis)
adapter = RedisAdapter(config=TEST_REDIS_CONFIG, client=mock_client)
assert adapter._client is mock_client
assert adapter._client_injected is True
def test_raises_when_client_provided_without_config() -> None:
mock_client = MagicMock(spec=redis.Redis)
with pytest.raises(ValueError, match="config is required"):
RedisAdapter(client=mock_client)
def test_disconnect_does_not_close_injected_client() -> None:
mock_client = MagicMock(spec=redis.Redis)
adapter = RedisAdapter(config=TEST_REDIS_CONFIG, client=mock_client)
adapter.disconnect()
mock_client.close.assert_not_called()
assert adapter._client is None
class CustomEnvRedisAdapter(RedisAdapter):
uri_env_var_name = "CUSTOM_REDIS_URI"
def test_subclass_custom_env_var_name(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("CUSTOM_REDIS_URI", "redis://custom:6379")
adapter = CustomEnvRedisAdapter()
assert adapter._config.uri == "redis://custom:6379"
+27
View File
@@ -0,0 +1,27 @@
"""Unit tests for RedisConfig."""
from __future__ import annotations
import pytest
from python_repositories.config import RedisConfig
def test_from_env_loads_uri(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("REDIS_URI", "redis://example:6379")
config = RedisConfig.from_env(use_dotenv=False)
assert config.uri == "redis://example:6379"
def test_from_env_raises_when_uri_missing(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("REDIS_URI", raising=False)
with pytest.raises(Exception):
RedisConfig.from_env(use_dotenv=False)
def test_from_env_respects_custom_env_var_name(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("CUSTOM_REDIS_URI", "redis://custom:6379")
config = RedisConfig.from_env("CUSTOM_REDIS_URI", use_dotenv=False)
assert config.uri == "redis://custom:6379"
Generated
+2
View File
@@ -1059,6 +1059,7 @@ name = "python-repositories"
version = "0.4.1" version = "0.4.1"
source = { editable = "." } source = { editable = "." }
dependencies = [ dependencies = [
{ name = "python-dotenv" },
{ name = "python-utils" }, { name = "python-utils" },
{ name = "structlog" }, { name = "structlog" },
] ]
@@ -1087,6 +1088,7 @@ dev = [
[package.metadata] [package.metadata]
requires-dist = [ requires-dist = [
{ name = "minio", marker = "extra == 'minio'", specifier = ">=7.2.16" }, { name = "minio", marker = "extra == 'minio'", specifier = ">=7.2.16" },
{ name = "python-dotenv", specifier = ">=1.0.0" },
{ name = "python-utils", specifier = ">=0.1.0", index = "https://gitea.lille-vemmelund.dk/api/packages/brian/pypi/simple/" }, { name = "python-utils", specifier = ">=0.1.0", index = "https://gitea.lille-vemmelund.dk/api/packages/brian/pypi/simple/" },
{ name = "redis", marker = "extra == 'redis'", specifier = ">=6.4.0" }, { name = "redis", marker = "extra == 'redis'", specifier = ">=6.4.0" },
{ name = "structlog", specifier = ">=25.4.0" }, { name = "structlog", specifier = ">=25.4.0" },