Compare commits

..
14 Commits
Author SHA1 Message Date
brian 2bb361e3a3 Merge pull request 'Add public repository interfaces and subclassable adapter CRUD API.' (#11) from cursor/public-repository-api into main
Code Quality Pipeline / code-quality (push) Successful in 30s
Test Python Package / test (push) Successful in 52s
Publish Python Package / build-and-publish (push) Successful in 22s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/11
2026-06-28 20:05:14 +02:00
Brian Bjarke JensenandCursor c5169df10d Use pre-commit for Prettier in CI and document local hook setup.
Test Python Package / test (pull_request) Successful in 50s
Code Quality Pipeline / code-quality (pull_request) Successful in 31s
Run prettier via pre-commit in code-quality workflow and add README instructions for installing and running hooks during development.

Co-authored-by: Cursor <[email protected]>
2026-06-28 19:56:25 +02:00
Brian Bjarke JensenandCursor 497533f0e3 Fix mypy context manager typing for adapter subclasses.
Code Quality Pipeline / code-quality (pull_request) Failing after 26s
Test Python Package / test (pull_request) Successful in 1m7s
Return Self from __enter__ so with-blocks preserve subclass types, and align mypy python_version with the 3.12 runtime target.

Co-authored-by: Cursor <[email protected]>
2026-06-28 19:42:15 +02:00
Brian Bjarke JensenandCursor e98fd3b90d Add public repository interfaces and subclassable adapter CRUD API.
Code Quality Pipeline / code-quality (pull_request) Failing after 19s
Test Python Package / test (pull_request) Successful in 50s
Define split JSON and object repository ABCs, promote adapter methods to public CRUD, add example domain repositories, modernize interface stubs, and make the package installable for tests.

Co-authored-by: Cursor <[email protected]>
2026-06-28 10:58:33 +02:00
brian f4280f15c1 Merge pull request 'Remove non-existent Mongo adapter from README.' (#10) from cleanup/remove-mongo-adapter-mentions into main
Test Python Package / test (push) Successful in 1m12s
Code Quality Pipeline / code-quality (push) Successful in 24s
Reviewed-on: https://gitea.lille-vemmelund.dk/brian/python-repositories/pulls/10
2026-06-21 23:05:47 +02:00
Brian Bjarke Jensen d9b9fa49a2 updated package settings
Code Quality Pipeline / code-quality (pull_request) Successful in 41s
Test Python Package / test (pull_request) Successful in 1m18s
2026-06-21 22:45:29 +02:00
Brian Bjarke Jensen 9d25f7c722 updated package settings
Code Quality Pipeline / code-quality (pull_request) Failing after 39s
Test Python Package / test (pull_request) Successful in 1m23s
2026-06-21 22:43:14 +02:00
Brian Bjarke Jensen 163581526d updated package settings
Code Quality Pipeline / code-quality (pull_request) Failing after 21s
Test Python Package / test (pull_request) Failing after 12s
2026-06-21 22:39:50 +02:00
Brian Bjarke Jensen 933c4ee7c5 updated package settings
Test Python Package / test (pull_request) Successful in 1m27s
Code Quality Pipeline / code-quality (pull_request) Successful in 41s
2026-06-21 22:34:50 +02:00
Brian Bjarke Jensen 2b0d98f84c updated package settings
Code Quality Pipeline / code-quality (pull_request) Failing after 51s
Test Python Package / test (pull_request) Failing after 2m1s
2026-06-21 22:26:41 +02:00
Brian Bjarke JensenandCursor 729fa48505 Remove non-existent Mongo adapter from README.
Code Quality Pipeline / code-quality (pull_request) Failing after 2m9s
Test Python Package / test (pull_request) Failing after 2m10s
The package only ships Redis and MinIO adapters, so the docs should not reference a mongo extra.

Co-authored-by: Cursor <[email protected]>
2026-06-21 22:17:44 +02:00
Brian Bjarke Jensen fa011be862 Merge pull request 'added support for content type when putting object in minio' (#9) from add-support-for-content-type-in-minio into main
Code Quality Pipeline / code-quality (push) Successful in 26s
Test Python Package / test (push) Successful in 33s
Publish Python Package / build-and-publish (push) Successful in 15s
Dependency Update Bot / update-check (push) Failing after 10s
Security Audit / safety (push) Failing after 2m9s
Reviewed-on: brian/python-repositories#9
2025-09-18 19:40:27 +02:00
Brian Bjarke Jensen 2e9330949c ruff format fix
Code Quality Pipeline / code-quality (pull_request) Successful in 29s
Test Python Package / test (pull_request) Successful in 35s
2025-09-18 19:38:22 +02:00
Brian Bjarke Jensen 3ab8da92fb added support for content type when putting object in minio
Code Quality Pipeline / code-quality (pull_request) Failing after 24s
Test Python Package / test (pull_request) Successful in 1m9s
2025-09-18 19:34:05 +02:00
22 changed files with 1234 additions and 847 deletions
+1 -3
View File
@@ -39,6 +39,4 @@ jobs:
run: uv run pyupgrade --py313-plus $(git ls-files '*.py') && git diff --exit-code
- name: Prettier format
run: |
npm install --save-dev --save-exact prettier
npx prettier --check .
run: uv run pre-commit run prettier --all-files
+106 -9
View File
@@ -1,17 +1,114 @@
# python-repositories
Various python repository interfaces exposed as a python package.
Unified repository interfaces and technology-specific adapters for Python projects.
Subclass an adapter in your own repository to add domain-specific methods while reusing connection management and CRUD operations.
## Architecture
| Layer | Responsibility |
| ---------------- | ----------------------------------------------------------------- |
| **Interfaces** | Abstract contracts for connection, context, and CRUD |
| **Adapters** | Technology-specific base classes (`RedisAdapter`, `MinioAdapter`) |
| **Your project** | Subclass an adapter and add domain methods |
## Optional dependencies
This package supports interacting with multiple different backends:
- Redis
- Mongo
- MinIO
To add support for a specific backend install this package with one or more of these optional packages:
Install with the extras you need:
```bash
uv add python-repositories[redis, mongo, minio]
uv add python-repositories[redis]
uv add python-repositories[minio]
uv add python-repositories[redis,minio]
```
### Redis (`JsonRepositoryInterface`)
Requires Redis with the RedisJSON module (e.g. redis-stack).
| Environment variable | Description |
| -------------------- | ---------------------------------------------------- |
| `REDIS_URI` | Redis connection URL (e.g. `redis://localhost:6379`) |
### MinIO (`ObjectRepositoryInterface`)
| Environment variable | Description |
| -------------------- | ------------------------------------------- |
| `MINIO_ENDPOINT` | MinIO server endpoint |
| `MINIO_ACCESS_KEY` | Access key |
| `MINIO_SECRET_KEY` | Secret key |
| `MINIO_BUCKET` | Bucket name (created on connect if missing) |
## Quick start
### JSON documents with Redis
```python
from python_repositories.examples.user_json_repository import UserJsonRepository
with UserJsonRepository() as repo:
repo.save_user("alice", {"name": "Alice", "email": "[email protected]"})
user = repo.get_user("alice")
repo.delete_user("alice")
```
### Binary objects with MinIO
```python
from io import BytesIO
from python_repositories.examples.artifact_object_repository import (
ArtifactObjectRepository,
)
with ArtifactObjectRepository() as repo:
repo.store_artifact("report-1", BytesIO(b"pdf bytes here"))
data = repo.get_artifact("report-1")
```
### Subclassing in your own project
```python
from python_repositories import RedisAdapter
class UserRepository(RedisAdapter):
def _key(self, user_id: str) -> str:
return f"user:{user_id}"
def get_user(self, user_id: str) -> dict | None:
return self.get(self._key(user_id))
def save_user(self, user_id: str, user: dict) -> None:
self.set(self._key(user_id), user)
```
## Public API
```python
from python_repositories import (
ConnectionAwareInterface,
ContextAwareInterface,
JsonRepositoryInterface,
ObjectRepositoryInterface,
RedisAdapter,
MinioAdapter,
)
```
## Development
```bash
uv sync --all-extras
uv run pre-commit install # once per clone — runs hooks on git commit
uv run pytest tests/integration/ -v
```
`pre-commit` is included in the dev dependency group. `uv sync` installs the CLI, but git does not run hooks until you install them with `pre-commit install` (one time per clone). After that, commits run the checks defined in [`.pre-commit-config.yaml`](.pre-commit-config.yaml) (ruff, mypy, pyupgrade, prettier, and general file hygiene).
To run all hooks manually without committing:
```bash
uv run pre-commit run --all-files
```
Integration tests require Docker (testcontainers).
+11 -3
View File
@@ -26,21 +26,29 @@ minio = [
"minio>=7.2.16",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["."]
[tool.uv.sources]
python-utils = { index = "gitea" }
[[tool.uv.index]]
name = "threadripper-proxpi-cache"
url = "http://10.0.0.2:5001/index/"
url = "https://proxpi.lille-vemmelund.dk/index/"
default = true
[[tool.uv.index]]
name = "gitea"
url = "https://gitea.gt-proj.com/api/packages/brian/pypi/simple/"
url = "https://gitea.lille-vemmelund.dk/api/packages/brian/pypi/simple/"
explicit = true
[tool.mypy]
python_version = "3.10"
python_version = "3.12"
warn_return_any = true # nudge to use stricter types
warn_unused_configs = true # nudge to remove unused configs
disallow_untyped_defs = true # disallow untyped function definitions
+15 -3
View File
@@ -1,6 +1,18 @@
"""python_repositories: Unified repository interfaces and adapters."""
from . import adapters
from . import interfaces
from .adapters import MinioAdapter, RedisAdapter
from .interfaces import (
ConnectionAwareInterface,
ContextAwareInterface,
JsonRepositoryInterface,
ObjectRepositoryInterface,
)
__all__ = ["adapters", "interfaces"]
__all__ = [
"ConnectionAwareInterface",
"ContextAwareInterface",
"JsonRepositoryInterface",
"ObjectRepositoryInterface",
"RedisAdapter",
"MinioAdapter",
]
+17 -6
View File
@@ -4,12 +4,14 @@ from __future__ import annotations
import os
from io import BytesIO
from importlib.util import find_spec
from typing import Self
import structlog
from python_utils import check_env
from python_repositories.interfaces import (
ContextAwareInterface,
ConnectionAwareInterface,
ContextAwareInterface,
ObjectRepositoryInterface,
)
# Handle optional dependencies
@@ -18,6 +20,7 @@ if find_spec("minio") is not None:
class MinioAdapter(
ObjectRepositoryInterface,
ContextAwareInterface,
ConnectionAwareInterface,
):
@@ -47,7 +50,7 @@ class MinioAdapter(
self._client: minio.Minio | None = None
self._bucket_name: str | None = None
def __enter__(self) -> MinioAdapter:
def __enter__(self) -> Self:
"""Enter the context."""
self.connect()
return self
@@ -113,13 +116,20 @@ class MinioAdapter(
self.logger.debug(res)
return res
def _put(self, object_name: str, data: BytesIO) -> None:
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
"""Put an object into the Minio bucket."""
# Check input
if not isinstance(object_name, str) or len(object_name) == 0:
raise ValueError("object_name must be a non-empty string")
if not isinstance(data, BytesIO) or data.getbuffer().nbytes == 0:
raise ValueError("data must be a non-empty BytesIO object")
if not isinstance(content_type, str) or len(content_type) == 0:
raise ValueError("content_type must be a non-empty string")
# Check connection
if self._client is None or not self.is_connected:
raise ConnectionError("Not connected to Minio")
@@ -134,12 +144,13 @@ class MinioAdapter(
data=data,
length=num_bytes,
part_size=self.chunk_size,
content_type=content_type,
)
self.logger.debug(
f"Put object '{object_name}' into bucket '{self._bucket_name}'"
)
def _get(self, object_name: str) -> BytesIO | None:
def get(self, object_name: str) -> BytesIO | None:
"""Get an object from the Minio bucket."""
# Check input
if not isinstance(object_name, str) or len(object_name) == 0:
@@ -174,7 +185,7 @@ class MinioAdapter(
self.logger.error(repr(exc))
return None
def _delete(self, object_name: str) -> None:
def delete(self, object_name: str) -> None:
"""Delete an object from the Minio bucket."""
# Check input
if not isinstance(object_name, str) or len(object_name) == 0:
@@ -192,7 +203,7 @@ class MinioAdapter(
f"Deleted object '{object_name}' from bucket '{self._bucket_name}'"
)
def _list_objects(self, prefix: str = "") -> list[str]:
def list_objects(self, prefix: str = "") -> list[str]:
"""List objects in the Minio bucket with an optional prefix."""
# Check input
if not isinstance(prefix, str):
@@ -1,7 +1,7 @@
"""Definition of RedisAdapter class."""
from __future__ import annotations
from typing import cast
from typing import Self, cast
from importlib.util import find_spec
import os
import structlog
@@ -9,8 +9,9 @@ import structlog
from python_utils import check_env
from python_repositories.interfaces import (
ContextAwareInterface,
ConnectionAwareInterface,
ContextAwareInterface,
JsonRepositoryInterface,
)
# Handle optional dependencies
@@ -20,6 +21,7 @@ if find_spec("redis") is not None:
class RedisAdapter(
JsonRepositoryInterface,
ContextAwareInterface,
ConnectionAwareInterface,
):
@@ -40,7 +42,7 @@ class RedisAdapter(
self._client: redis.Redis | None = None
self.path: str = RedisPath.root_path()
def __enter__(self) -> RedisAdapter:
def __enter__(self) -> Self:
"""Enter the context."""
self.connect()
return self
@@ -92,7 +94,7 @@ class RedisAdapter(
self.logger.debug(res)
return res
def _set(self, key: str, data: dict) -> None:
def set(self, key: str, data: dict) -> None:
"""Set a JSON object in Redis."""
# Check input
if not isinstance(key, str) or len(key) == 0:
@@ -106,7 +108,7 @@ class RedisAdapter(
self._client.json().set(key, self.path, data)
self.logger.debug(f"Set {key} to {data}")
def _get(self, key: str) -> dict | None:
def get(self, key: str) -> dict | None:
"""Get a JSON object from Redis."""
# Check input
if not isinstance(key, str) or len(key) == 0:
@@ -122,7 +124,7 @@ class RedisAdapter(
self.logger.debug(f"Got {data} from {key}")
return data
def _delete(self, key: str) -> None:
def delete(self, key: str) -> None:
"""Delete data from Redis."""
# Check input
if not isinstance(key, str) or len(key) == 0:
@@ -134,7 +136,7 @@ class RedisAdapter(
self._client.json().delete(key)
self.logger.debug(f"Deleted {key}")
def _list_keys(self, pattern: str) -> list[str]:
def list_keys(self, pattern: str) -> list[str]:
"""List keys in Redis matching a pattern."""
# Check input
if not isinstance(pattern, str) or len(pattern) == 0:
+1
View File
@@ -0,0 +1 @@
"""Example domain repositories built on technology adapters."""
@@ -0,0 +1,23 @@
"""Example domain repository backed by MinIO objects."""
from io import BytesIO
from python_repositories.adapters.minio_adapter import MinioAdapter
class ArtifactObjectRepository(MinioAdapter):
"""Example: domain repository backed by MinIO objects."""
def _object_name(self, artifact_id: str) -> str:
return f"artifacts/{artifact_id}"
def get_artifact(self, artifact_id: str) -> BytesIO | None:
return self.get(self._object_name(artifact_id))
def store_artifact(
self,
artifact_id: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
self.put(self._object_name(artifact_id), data, content_type)
@@ -0,0 +1,19 @@
"""Example domain repository backed by Redis JSON."""
from python_repositories.adapters.redis_adapter import RedisAdapter
class UserJsonRepository(RedisAdapter):
"""Example: domain repository backed by Redis JSON."""
def _key(self, user_id: str) -> str:
return f"user:{user_id}"
def get_user(self, user_id: str) -> dict | None:
return self.get(self._key(user_id))
def save_user(self, user_id: str, user: dict) -> None:
self.set(self._key(user_id), user)
def delete_user(self, user_id: str) -> None:
self.delete(self._key(user_id))
@@ -2,8 +2,16 @@ from .connection_aware_interface import (
ConnectionAwareInterface as ConnectionAwareInterface,
)
from .context_aware_interface import ContextAwareInterface as ContextAwareInterface
from .json_repository_interface import (
JsonRepositoryInterface as JsonRepositoryInterface,
)
from .object_repository_interface import (
ObjectRepositoryInterface as ObjectRepositoryInterface,
)
__all__ = [
"ConnectionAwareInterface",
"ContextAwareInterface",
"JsonRepositoryInterface",
"ObjectRepositoryInterface",
]
@@ -9,15 +9,15 @@ class ConnectionAwareInterface(ABC):
@abstractmethod
def connect(self) -> None:
"""Connect to resource."""
raise NotImplementedError()
...
@abstractmethod
def disconnect(self) -> None:
"""Disconnect from resource."""
raise NotImplementedError()
...
@property
@abstractmethod
def is_connected(self) -> bool:
"""Check if connected to resource."""
raise NotImplementedError()
...
@@ -1,21 +1,22 @@
"""Definition of ConnectionAwareInterface abstract base class."""
"""Definition of ContextAwareInterface abstract base class."""
from __future__ import annotations
from abc import ABC, abstractmethod
from typing import Self
class ContextAwareInterface(ABC):
"""Interface that defined context-related methods."""
"""Interface that defines context-related methods."""
@abstractmethod
def __enter__(self) -> ContextAwareInterface:
def __enter__(self) -> Self:
"""Enter the context."""
raise NotImplementedError()
...
@abstractmethod
def __exit__(
self, exc_type: type | None, exc_val: object | None, exc_tb: object | None
) -> None:
"""Exit the context."""
raise NotImplementedError()
...
@@ -0,0 +1,27 @@
"""Definition of JsonRepositoryInterface abstract base class."""
from abc import ABC, abstractmethod
class JsonRepositoryInterface(ABC):
"""Interface that defines JSON document CRUD methods."""
@abstractmethod
def get(self, key: str) -> dict | None:
"""Get a JSON object by key."""
...
@abstractmethod
def set(self, key: str, data: dict) -> None:
"""Set a JSON object by key."""
...
@abstractmethod
def delete(self, key: str) -> None:
"""Delete a JSON object by key."""
...
@abstractmethod
def list_keys(self, pattern: str) -> list[str]:
"""List keys matching a glob pattern."""
...
@@ -0,0 +1,33 @@
"""Definition of ObjectRepositoryInterface abstract base class."""
from abc import ABC, abstractmethod
from io import BytesIO
class ObjectRepositoryInterface(ABC):
"""Interface that defines binary object CRUD methods."""
@abstractmethod
def get(self, object_name: str) -> BytesIO | None:
"""Get an object by name."""
...
@abstractmethod
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
"""Put an object by name."""
...
@abstractmethod
def delete(self, object_name: str) -> None:
"""Delete an object by name."""
...
@abstractmethod
def list_objects(self, prefix: str = "") -> list[str]:
"""List object names with an optional prefix."""
...
@@ -54,66 +54,3 @@ def test_instantiation_fails_when_is_connected_not_implemented() -> None:
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_connect_raises_not_implemented_if_not_overwritten() -> None:
"""Test that connect raises NotImplementedError if not implemented."""
class Incomplete(ConnectionAwareInterface):
"""A class that does not implement connect."""
def connect(self) -> None:
super().connect() # type: ignore
def disconnect(self) -> None:
pass
@property
def is_connected(self) -> bool:
return False
instance = Incomplete()
with pytest.raises(NotImplementedError):
instance.connect()
def test_disconnect_raises_not_implemented_if_not_overwritten() -> None:
"""Test that disconnect raises NotImplementedError if not implemented."""
class Incomplete(ConnectionAwareInterface):
"""A class that does not implement disconnect."""
def connect(self) -> None:
pass
def disconnect(self) -> None:
super().disconnect() # type: ignore
@property
def is_connected(self) -> bool:
return False
instance = Incomplete()
with pytest.raises(NotImplementedError):
instance.disconnect()
def test_is_connected_raises_not_implemented_if_not_overwritten() -> None:
"""Test that is_connected raises NotImplementedError if not implemented."""
class Incomplete(ConnectionAwareInterface):
"""A class that does not implement is_connected."""
def connect(self) -> None:
pass
def disconnect(self) -> None:
pass
@property
def is_connected(self) -> bool:
return super().is_connected # type: ignore
instance = Incomplete()
with pytest.raises(NotImplementedError):
_ = instance.is_connected
@@ -1,6 +1,7 @@
"""Integration tests for ContextAwareInterface."""
from __future__ import annotations
import pytest
from python_repositories.interfaces.context_aware_interface import ContextAwareInterface
@@ -31,42 +32,3 @@ def test_instantiation_fails_when_exit_not_implemented() -> None:
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_enter_raises_not_implemented_if_not_overwritten() -> None:
"""Test that __enter__ raises NotImplementedError if not implemented."""
class Incomplete(ContextAwareInterface):
"""A class that does not implement __enter__."""
def __enter__(self) -> Incomplete:
super().__enter__() # type: ignore
return self
def __exit__(
self, exc_type: type | None, exc_val: object | None, exc_tb: object | None
) -> None:
pass
instance = Incomplete()
with pytest.raises(NotImplementedError):
instance.__enter__()
def test_exit_raises_not_implemented_if_not_overwritten() -> None:
"""Test that __exit__ raises NotImplementedError if not implemented."""
class Incomplete(ContextAwareInterface):
"""A class that does not implement __exit__."""
def __enter__(self) -> ContextAwareInterface:
return self
def __exit__(
self, exc_type: type | None, exc_val: object | None, exc_tb: object | None
) -> None:
super().__exit__(exc_type, exc_val, exc_tb) # type: ignore
instance = Incomplete()
with pytest.raises(NotImplementedError):
instance.__exit__(None, None, None)
+57
View File
@@ -0,0 +1,57 @@
"""Integration tests for example domain repositories."""
from collections.abc import Generator
from io import BytesIO
import random
import pytest
from python_repositories.examples.artifact_object_repository import (
ArtifactObjectRepository,
)
from python_repositories.examples.user_json_repository import UserJsonRepository
@pytest.fixture(scope="module")
def user_data() -> Generator[dict[str, str]]:
"""Provide sample user data for tests."""
yield {"name": "Alice", "email": "[email protected]"}
@pytest.fixture(scope="module")
def artifact_data() -> Generator[BytesIO]:
"""Provide sample artifact data for tests."""
yield BytesIO(random.randbytes(2**20))
def test_user_json_repository_save_and_get(
redis_container: str,
user_data: dict[str, str],
) -> None:
"""Test that UserJsonRepository can save and retrieve a user."""
with UserJsonRepository() as repo:
repo.save_user("alice", user_data)
assert repo.get_user("alice") == user_data
def test_user_json_repository_delete(
redis_container: str,
user_data: dict[str, str],
) -> None:
"""Test that UserJsonRepository can delete a user."""
with UserJsonRepository() as repo:
repo.save_user("alice", user_data)
repo.delete_user("alice")
assert repo.get_user("alice") is None
def test_artifact_object_repository_store_and_get(
minio_container: dict[str, str],
artifact_data: BytesIO,
) -> None:
"""Test that ArtifactObjectRepository can store and retrieve an artifact."""
with ArtifactObjectRepository() as repo:
repo.store_artifact("report-1", artifact_data)
received = repo.get_artifact("report-1")
assert received is not None
artifact_data.seek(0)
assert received.read() == artifact_data.read()
@@ -0,0 +1,82 @@
"""Integration tests for JsonRepositoryInterface."""
import pytest
from python_repositories.interfaces.json_repository_interface import (
JsonRepositoryInterface,
)
def test_instantiation_fails_when_get_not_implemented() -> None:
"""Test that instantiation fails if get is not implemented."""
class Incomplete(JsonRepositoryInterface):
"""A class that does not implement get."""
def set(self, key: str, data: dict) -> None:
pass
def delete(self, key: str) -> None:
pass
def list_keys(self, pattern: str) -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_set_not_implemented() -> None:
"""Test that instantiation fails if set is not implemented."""
class Incomplete(JsonRepositoryInterface):
"""A class that does not implement set."""
def get(self, key: str) -> dict | None:
return None
def delete(self, key: str) -> None:
pass
def list_keys(self, pattern: str) -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_delete_not_implemented() -> None:
"""Test that instantiation fails if delete is not implemented."""
class Incomplete(JsonRepositoryInterface):
"""A class that does not implement delete."""
def get(self, key: str) -> dict | None:
return None
def set(self, key: str, data: dict) -> None:
pass
def list_keys(self, pattern: str) -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_list_keys_not_implemented() -> None:
"""Test that instantiation fails if list_keys is not implemented."""
class Incomplete(JsonRepositoryInterface):
"""A class that does not implement list_keys."""
def get(self, key: str) -> dict | None:
return None
def set(self, key: str, data: dict) -> None:
pass
def delete(self, key: str) -> None:
pass
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
+43 -31
View File
@@ -1,6 +1,3 @@
# pylint: disable=protected-access
# The above line disables pylint's protected member access warnings for this file,
# allowing tests to access MinioAdapter's internal methods as needed for integration testing.
"""Integration tests for the MinioAdapter."""
from collections.abc import Generator
@@ -13,6 +10,7 @@ import os
import logging
from minio import S3Error
from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.interfaces import ObjectRepositoryInterface
def same_data(
@@ -103,7 +101,7 @@ def clear_minio(
def test_should_adhere_to_interface() -> None:
"""Test that the MinioAdapter adheres to the expected interface."""
# Instantiation fails if interface not adhered to
assert issubclass(MinioAdapter, ObjectRepositoryInterface)
_ = MinioAdapter()
@@ -187,7 +185,7 @@ def test_should_get_data(
# Arrange
object_name, expected_data = data_in_minio
# Act
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
# Assert
assert received_data is not None
assert same_data(expected_data, received_data)
@@ -200,7 +198,7 @@ def test_should_get_none_for_nonexistent_object(
# Arrange
object_name = "nonexistent_object"
# Act
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
# Assert
assert received_data is None
@@ -214,7 +212,7 @@ def test_should_raise_value_error_on_invalid_get_object_name(
# Act & Assert
for object_name in invalid_object_names:
with pytest.raises(ValueError):
minio_adapter._get(object_name) # type: ignore
minio_adapter.get(object_name) # type: ignore
def test_should_raise_connection_error_on_get_when_not_connected(
@@ -225,7 +223,7 @@ def test_should_raise_connection_error_on_get_when_not_connected(
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError):
adapter._get("some_object")
adapter.get("some_object")
def test_should_log_warning_when_getting_nonexistent_object(
@@ -249,7 +247,7 @@ def test_should_log_warning_when_getting_nonexistent_object(
object_name = "missing-object"
# Act
with caplog.at_level("WARNING"):
result = adapter._get(object_name)
result = adapter.get(object_name)
# Assert
assert result is None
assert (
@@ -280,7 +278,7 @@ def test_should_log_error_when_getting_with_s3error_other_than_no_such_key(
object_name = "missing-object"
# Act
with caplog.at_level("ERROR"):
result = adapter._get(object_name)
result = adapter.get(object_name)
# Assert
assert result is None
assert repr(other_s3error) in caplog.text
@@ -299,7 +297,7 @@ def test_should_log_error_when_getting_with_general_exception(
object_name = "missing-object"
# Act
with caplog.at_level("ERROR"):
result = adapter._get(object_name)
result = adapter.get(object_name)
# Assert
assert result is None
assert repr(general_exception) in caplog.text
@@ -312,12 +310,12 @@ def test_should_put_data(
"""Test that the MinioAdapter can put data into a bucket."""
# Arrange
object_name = "new_test_object"
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is None # ensure object does not exist yet
# Act
minio_adapter._put(object_name, data)
minio_adapter.put(object_name, data)
# Assert
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is not None
assert same_data(data, received_data)
# Cleanup
@@ -333,13 +331,13 @@ def test_should_update_data(
# Arrange
object_name, _ = data_in_minio
new_data = BytesIO(random.randbytes(2**21)) # 2 MiB
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is not None
assert not same_data(received_data, new_data)
# Act
minio_adapter._put(object_name, new_data)
minio_adapter.put(object_name, new_data)
# Assert
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is not None
assert same_data(new_data, received_data)
@@ -354,7 +352,7 @@ def test_should_raise_value_error_on_invalid_put_object_name(
# Act & Assert
for object_name in invalid_object_names:
with pytest.raises(ValueError):
minio_adapter._put(object_name, data) # type: ignore
minio_adapter.put(object_name, data) # type: ignore
def test_should_raise_value_error_on_invalid_put_data(
@@ -367,7 +365,21 @@ def test_should_raise_value_error_on_invalid_put_data(
# Act & Assert
for data in invalid_data:
with pytest.raises(ValueError):
minio_adapter._put(object_name, data) # type: ignore
minio_adapter.put(object_name, data) # type: ignore
def test_should_raise_value_error_on_invalid_put_content_type(
data: BytesIO,
minio_adapter: MinioAdapter,
) -> None:
"""Test that the MinioAdapter raises ValueError when putting with an invalid content type."""
# Arrange
object_name = "valid_object_name"
invalid_content_types = ["", 123, None]
# Act & Assert
for content_type in invalid_content_types:
with pytest.raises(ValueError):
minio_adapter.put(object_name, data, content_type) # type: ignore
def test_should_raise_connection_error_on_put_when_not_connected(
@@ -379,7 +391,7 @@ def test_should_raise_connection_error_on_put_when_not_connected(
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError):
adapter._put("some_object", data)
adapter.put("some_object", data)
def test_should_delete_object(
@@ -389,12 +401,12 @@ def test_should_delete_object(
"""Test that the MinioAdapter can delete an object from a bucket."""
# Arrange
object_name, _ = data_in_minio
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is not None # ensure object exists
# Act
minio_adapter._delete(object_name)
minio_adapter.delete(object_name)
# Assert
received_data = minio_adapter._get(object_name)
received_data = minio_adapter.get(object_name)
assert received_data is None
@@ -407,7 +419,7 @@ def test_should_raise_value_error_on_invalid_delete_object_name(
# Act & Assert
for object_name in invalid_object_names:
with pytest.raises(ValueError):
minio_adapter._delete(object_name) # type: ignore
minio_adapter.delete(object_name) # type: ignore
def test_should_raise_connection_error_on_delete_when_not_connected(
@@ -418,7 +430,7 @@ def test_should_raise_connection_error_on_delete_when_not_connected(
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError):
adapter._delete("some_object")
adapter.delete("some_object")
def test_should_list_objects(
@@ -430,9 +442,9 @@ def test_should_list_objects(
object_name, _ = data_in_minio
new_data = BytesIO(random.randbytes(2**21)) # 2 MiB
new_data_name = "another_test_object"
minio_adapter._put(new_data_name, new_data)
minio_adapter.put(new_data_name, new_data)
# Act
objects = minio_adapter._list_objects()
objects = minio_adapter.list_objects()
# Assert
assert isinstance(objects, list)
assert len(objects) == 2
@@ -449,10 +461,10 @@ def test_should_list_objects_with_prefix(
object_name, _ = data_in_minio
new_data = BytesIO(random.randbytes(2**21)) # 2 MiB
new_data_name = "prefix_test_object"
minio_adapter._put(new_data_name, new_data)
minio_adapter.put(new_data_name, new_data)
prefix = "prefix_"
# Act
objects = minio_adapter._list_objects(prefix)
objects = minio_adapter.list_objects(prefix)
# Assert
assert isinstance(objects, list)
assert len(objects) == 1
@@ -469,7 +481,7 @@ def test_should_raise_value_error_on_invalid_list_objects_prefix(
# Act & Assert
for prefix in invalid_prefixes:
with pytest.raises(ValueError):
minio_adapter._list_objects(prefix) # type: ignore
minio_adapter.list_objects(prefix) # type: ignore
def test_should_raise_connection_error_on_list_objects_when_not_connected(
@@ -480,7 +492,7 @@ def test_should_raise_connection_error_on_list_objects_when_not_connected(
adapter = MinioAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError):
adapter._list_objects()
adapter.list_objects()
# allows local debugging by running file as script
@@ -0,0 +1,99 @@
"""Integration tests for ObjectRepositoryInterface."""
from io import BytesIO
import pytest
from python_repositories.interfaces.object_repository_interface import (
ObjectRepositoryInterface,
)
def test_instantiation_fails_when_get_not_implemented() -> None:
"""Test that instantiation fails if get is not implemented."""
class Incomplete(ObjectRepositoryInterface):
"""A class that does not implement get."""
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
pass
def delete(self, object_name: str) -> None:
pass
def list_objects(self, prefix: str = "") -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_put_not_implemented() -> None:
"""Test that instantiation fails if put is not implemented."""
class Incomplete(ObjectRepositoryInterface):
"""A class that does not implement put."""
def get(self, object_name: str) -> BytesIO | None:
return None
def delete(self, object_name: str) -> None:
pass
def list_objects(self, prefix: str = "") -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_delete_not_implemented() -> None:
"""Test that instantiation fails if delete is not implemented."""
class Incomplete(ObjectRepositoryInterface):
"""A class that does not implement delete."""
def get(self, object_name: str) -> BytesIO | None:
return None
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
pass
def list_objects(self, prefix: str = "") -> list[str]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_list_objects_not_implemented() -> None:
"""Test that instantiation fails if list_objects is not implemented."""
class Incomplete(ObjectRepositoryInterface):
"""A class that does not implement list_objects."""
def get(self, object_name: str) -> BytesIO | None:
return None
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
pass
def delete(self, object_name: str) -> None:
pass
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
+25 -27
View File
@@ -1,6 +1,3 @@
# pylint: disable=protected-access
# The above line disables pylint's protected member access warnings for this file,
# allowing tests to access RedisAdapter's internal methods as needed for integration testing.
"""Integration tests for the RedisAdapter."""
from collections.abc import Generator
@@ -8,6 +5,7 @@ import pytest
import redis
from redis.commands.json.path import Path as RedisPath
from python_repositories.adapters.redis_adapter import RedisAdapter
from python_repositories.interfaces import JsonRepositoryInterface
@pytest.fixture(scope="module")
@@ -50,7 +48,7 @@ def clear_redis(raw_redis_client: redis.Redis) -> None:
def test_should_adhere_to_interface(redis_container: str) -> None:
"""Test that the RedisAdapter adheres to the expected interface."""
# Instantiation fails if interface not adhered to
assert issubclass(RedisAdapter, JsonRepositoryInterface)
_ = RedisAdapter()
@@ -134,7 +132,7 @@ def test_should_get_value(
# Arrange
key, data = data_in_redis
# Act
value = redis_adapter._get(key)
value = redis_adapter.get(key)
# Assert
assert value is not None
assert value == data
@@ -145,7 +143,7 @@ def test_should_get_none_for_missing_key(
) -> None:
"""Test that getting a non-existent key returns None."""
# Act
value = redis_adapter._get("nonexistent_key")
value = redis_adapter.get("nonexistent_key")
# Assert
assert value is None
@@ -159,7 +157,7 @@ def test_should_raise_value_error_on_invalid_get_key(
# Act & Assert
for key in invalid_keys:
with pytest.raises(ValueError):
redis_adapter._get(key) # type: ignore
redis_adapter.get(key) # type: ignore
def test_should_raise_connection_error_on_get_when_not_connected(
@@ -170,7 +168,7 @@ def test_should_raise_connection_error_on_get_when_not_connected(
adapter = RedisAdapter() # not connected
# Act & Assert
with pytest.raises(ConnectionError):
adapter._get("some_key")
adapter.get("some_key")
def test_should_set_value(
@@ -180,12 +178,12 @@ def test_should_set_value(
"""Test that the RedisAdapter can set a value."""
# Arrange
key = "test_key"
received_data = redis_adapter._get(key)
received_data = redis_adapter.get(key)
assert received_data is None # Ensure key does not exist
# Act
redis_adapter._set(key, data)
redis_adapter.set(key, data)
# Assert
received_data = redis_adapter._get(key)
received_data = redis_adapter.get(key)
assert received_data is not None
assert received_data == data
@@ -198,13 +196,13 @@ def test_should_update_value(
# Arrange
key, _ = data_in_redis
new_data = {"new_key": "new_value"}
received_data = redis_adapter._get(key)
received_data = redis_adapter.get(key)
assert received_data is not None
assert received_data != new_data
# Act
redis_adapter._set(key, new_data)
redis_adapter.set(key, new_data)
# Assert
assert redis_adapter._get(key) == new_data
assert redis_adapter.get(key) == new_data
def test_should_raise_value_error_on_invalid_set_key(
@@ -217,7 +215,7 @@ def test_should_raise_value_error_on_invalid_set_key(
# Act & Assert
for key in invalid_keys:
with pytest.raises(ValueError):
redis_adapter._set(key, data) # type: ignore
redis_adapter.set(key, data) # type: ignore
def test_should_raise_value_error_on_invalid_set_data(
@@ -230,7 +228,7 @@ def test_should_raise_value_error_on_invalid_set_data(
# Act & Assert
for data in invalid_data:
with pytest.raises(ValueError):
redis_adapter._set(key, data) # type: ignore
redis_adapter.set(key, data) # type: ignore
def test_should_raise_connection_error_on_set_when_not_connected(
@@ -243,7 +241,7 @@ def test_should_raise_connection_error_on_set_when_not_connected(
key = "test_key"
# Act & Assert
with pytest.raises(ConnectionError):
adapter._set(key, data)
adapter.set(key, data)
def test_should_delete_key(
@@ -253,12 +251,12 @@ def test_should_delete_key(
"""Test that deleting a key removes it from Redis."""
# Arrange
key, _ = data_in_redis
received_data = redis_adapter._get(key)
received_data = redis_adapter.get(key)
assert received_data is not None # Ensure key exists
# Act
redis_adapter._delete(key)
redis_adapter.delete(key)
# Assert
assert redis_adapter._get(key) is None
assert redis_adapter.get(key) is None
def test_should_raise_value_error_on_invalid_delete_key(
@@ -270,7 +268,7 @@ def test_should_raise_value_error_on_invalid_delete_key(
# Act & Assert
for key in invalid_keys:
with pytest.raises(ValueError):
redis_adapter._delete(key) # type: ignore
redis_adapter.delete(key) # type: ignore
def test_should_raise_connection_error_on_delete_when_not_connected(
@@ -281,7 +279,7 @@ def test_should_raise_connection_error_on_delete_when_not_connected(
adapter = RedisAdapter()
# Act & Assert
with pytest.raises(ConnectionError):
adapter._delete("some_key")
adapter.delete("some_key")
def test_should_list_keys(
@@ -289,10 +287,10 @@ def test_should_list_keys(
) -> None:
"""Test listing keys matching a pattern returns correct keys."""
# Arrange
redis_adapter._set("key1", {"a": 1})
redis_adapter._set("key2", {"b": 2})
redis_adapter.set("key1", {"a": 1})
redis_adapter.set("key2", {"b": 2})
# Act
keys = redis_adapter._list_keys("key*")
keys = redis_adapter.list_keys("key*")
# Assert
assert set(keys) == {"key1", "key2"}
@@ -306,7 +304,7 @@ def test_should_raise_value_error_on_invalid_list_keys_pattern(
# Act & Assert
for pattern in invalid_patterns:
with pytest.raises(ValueError):
redis_adapter._list_keys(pattern) # type: ignore
redis_adapter.list_keys(pattern) # type: ignore
def test_should_raise_connection_error_on_list_keys_when_not_connected(
@@ -317,7 +315,7 @@ def test_should_raise_connection_error_on_list_keys_when_not_connected(
adapter = RedisAdapter()
# Act & Assert
with pytest.raises(ConnectionError):
adapter._list_keys("some_pattern")
adapter.list_keys("some_pattern")
# allows local debugging by running file as script
Generated
+648 -648
View File
File diff suppressed because it is too large Load Diff