PLBA2-2. Реализация

This commit is contained in:
2026-10-10 19:36:32 +03:00
parent 473cde126f
commit 71998a6b13
120 changed files with 3675 additions and 21 deletions
+78 -19
View File
@@ -1,35 +1,94 @@
# PLBA v2 # PLBA v2
PLBA v2 is a compact, transport-neutral runtime for business microservices. It owns application lifecycle, worker supervision, versioned configuration snapshots, traces, structured logging, health aggregation, workflows and an optional audit port. PLBA v2 is a transport-neutral Python library for application lifecycle,
workers, health aggregation, configuration snapshots, workflows, business
tracing, and structured technical logs. It is a separate package from PLBA v1.
It deliberately does not own task queues, RabbitMQ clients, gRPC servers, HTTP servers, a database driver, or business logic. A service supplies those as adapters and owns its entrypoint. The application owns its entrypoint and composition root, business logic, HTTP
host and listener, database schema and migrations, and message queues. PLBA2
provides components and adapters that the application wires together; it does
not start a service or create a second HTTP port.
## Status ## Capabilities
This repository contains the first executable skeleton and public contracts. It is a breaking, separate package from PLBA v1; no v1 compatibility facade is provided. - `runtime/application-identity` resolves `application_id` from `config.yaml`,
then `[project].name` in `pyproject.toml`, then UUIDv4.
- `runtime/worker` registers uniquely named workers, applies the `disabled`
flag, runs sequential scheduled cycles, combines application and automatic
health, and supports graceful or force stop.
- `runtime/application-lifecycle` starts and stops all enabled workers and
reports application lifecycle states.
- `health/aggregation` combines worker and contributor states, criticality,
missing components, and application lifecycle into a health report.
- `api/http/control-api` is a mountable ASGI adapter for `GET /health`,
`POST /runtime/start`, `POST /runtime/stop`, and `GET /runtime/status`.
- `configuration/application-config-file` loads an application-typed root
`config.yaml` and atomically reloads valid snapshots without a restart.
- `configuration/application-settings` provides a typed `SettingsReader` and a
PostgreSQL adapter for the application's `settings` table.
- `workflow/orchestration` executes named object-based steps through declared
transitions and persists run and step snapshots through an application port.
- `observability/business-tracing` provides trace contracts and a PostgreSQL
writer for application-owned `trace_contexts` and `trace_messages` tables.
- `observability/technical-logging` formats JSON log records with application
and business trace identifiers for stdout.
## Run checks ## Install
```bash ```bash
python -m unittest discover -s tests -v pip install .
``` ```
## Configuration-service shape The PostgreSQL adapters accept the application's pool and schema. PLBA2 does
not install or manage a database driver. The HTTP adapter uses ASGI and does
not install or manage an HTTP server.
## Composition outline
The application loads and validates configuration before building dependent
components. It supplies a decoder for its own typed fields:
```python ```python
runtime = Runtime( config_loader = ApplicationConfigLoader(
configuration=ConfigurationSnapshotManager( Path("config.yaml"), application_decoder=ServiceConfig.from_mapping
client=configuration_grpc_adapter,
scope="configuration-service",
validator=validate_configuration,
),
workers=[grpc_server_worker, configuration_changed_consumer],
) )
config_manager = ApplicationConfigManager(config_loader)
initial_config = config_manager.load_initial()
await runtime.start() identity = ApplicationIdentityResolver(
# The RabbitMQ adapter invokes the registered callback after configuration.changed. TomlProjectNameReader(Path("pyproject.toml"))
# The manager refetches and atomically swaps the validated snapshot. )
await runtime.stop(timeout_seconds=20) runtime_context = identity.resolve(initial_config.application_id)
workers = WorkerRegistry()
workers.register(WorkerDefinition("inventory_sync", sync_inventory))
supervisor = WorkerSupervisor(workers, config_manager)
runtime = ApplicationRuntime(runtime_context, supervisor)
``` ```
The adapter authenticates using bootstrap values read by the service from environment/secrets. Business code reads `manager.current()` and never makes a synchronous configuration-service call. The application creates its HTTP host and mounts `ControlApi` on the same
listener. `RuntimeHealthInputProvider` connects the runtime, worker supervisor,
and registered health contributors to `HealthAggregator`. The application
starts its own server and chooses when to call `runtime.start()` and
`runtime.stop()`.
For configuration changes, run `config_manager.watch(...)` in an application
task and handle reload errors in the supplied callback. Existing snapshots
remain stable; an invalid update leaves the latest valid snapshot active.
## Database ownership
The application creates and seeds `settings` in its migrations. It also creates
`trace_contexts` and `trace_messages` in its migrations. PLBA2 does not create
tables, run application migrations, copy YAML into the database, expose a
business-history API, publish business events to RabbitMQ, or provide shared
worker-message queues. See [database contracts](docs/database-contracts.md) and
[configuration](docs/configuration.md) for the consumer-facing contracts.
See [integration guidance](docs/integration.md) for wiring the HTTP, database,
workflow, trace, and logging adapters into an application.
## Development
```bash
python3 -m pytest
```
+40
View File
@@ -0,0 +1,40 @@
# Application configuration
PLBA2 reads `config.yaml` from the application root. The application supplies a
typed decoder for its own fields; PLBA2 validates the common identity and worker
fields and keeps the application model in the same snapshot.
```yaml
application_id: inventory-service
workers:
inventory_sync:
disabled: false
interval_seconds: 60
cleanup:
pause_seconds: 300
database_url: "..."
```
`application_id` is optional. A non-empty value is used unchanged; otherwise
identity resolution uses `[project].name` from `pyproject.toml`, then a UUIDv4.
The resolved value is passed to `ApplicationRuntime` through `RuntimeContext`
before dependent components start.
Each worker entry may set `disabled` (default `false`) and one schedule:
`interval_seconds` or `pause_seconds`. Enabled workers need exactly one positive
finite schedule. A disabled worker is skipped at application start and does not
need a schedule. Changing `disabled` takes effect on the next application start;
valid schedule updates are used when the next worker cycle is planned.
The application defines every other YAML field and its typed model. This file is
for rarely changed technical parameters. Business parameters, feature toggles,
and technical values that must change remotely while the process is running
belong in the application-owned `settings` table, read through `SettingsReader`.
Load and validate the initial file before starting dependent components. A
missing, malformed, or invalid initial file raises `ApplicationConfigLoadError`
and blocks startup. After startup, `ApplicationConfigManager.watch` checks for
file changes. It swaps in a fully validated snapshot atomically; previously
returned snapshots remain unchanged. If an update is invalid or unreadable,
`reload_if_changed` returns a failure and the manager keeps the last valid
snapshot. The watcher passes that error to the application's callback.
+77
View File
@@ -0,0 +1,77 @@
# Database contracts
PLBA2 reads and writes application-owned tables through adapters. The application
owns the schema, migrations, initial values, and retention policy. PLBA2 does not
create tables or run migrations.
## `settings`
The application creates `settings` in its own schema and loads initial rows in
its migration. The table has this contract:
| Field | Type | Constraints | Purpose |
| --- | --- | --- | --- |
| `id` | `BIGINT` | Identity primary key, not null | Row identifier and final ordering key |
| `config_key` | `VARCHAR(128)` | Nullable | Application-defined scope or entity |
| `setting_key` | `VARCHAR(128)` | Not null | Stable setting name |
| `setting_value` | `TEXT` | Not null | Text value decoded by the application |
| `sort_order` | `INT` | Not null, default `0` | Order for repeated keys |
| `is_enabled` | `BOOLEAN` | Not null, default `TRUE` | Includes or excludes a row from reads |
| `description` | `TEXT` | Nullable | Human-readable explanation |
Indexes should support the adapter's filters and ordering, for example:
```sql
CREATE INDEX settings_key_enabled_order_idx
ON settings (setting_key, is_enabled, sort_order, id);
CREATE INDEX settings_scope_lookup_order_idx
ON settings (setting_key, config_key, is_enabled, sort_order, id);
```
There is no uniqueness constraint on `setting_key`; several enabled rows may
form an ordered list. Reads filter by `setting_key`, an optional `config_key`,
and `is_enabled = TRUE`, then order by `sort_order`, `id`. The value of a feature
toggle belongs in `setting_value`; `is_enabled` only controls whether the row is
read. `config_key` relationships are owned by the application.
`settings` stores business parameters, feature toggles, and technical values
that must change remotely while the process runs. Rarely changed technical
parameters remain in the root `config.yaml`; PLBA2 does not copy YAML into the
database.
## `trace_contexts` and `trace_messages`
The application creates both trace tables in its own schema through its own
migration. Public code calls the context identifier `business_trace_id`; both
tables store it as `trace_id`.
`trace_contexts` contains:
| Field | Type | Constraints | Purpose |
| --- | --- | --- | --- |
| `trace_id` | `UUID` | Primary key, not null | Public `business_trace_id` |
| `parent_id` | `UUID` | Nullable | Parent context identifier |
| `alias` | `TEXT` | Not null | Short context name |
| `trace_type` | `TEXT` | Not null | Business operation type |
| `event_time` | `TIMESTAMPTZ` | Not null | Context creation time |
| `attrs_json` | `JSONB` | Not null | Context attributes, including `application_id` |
Create indexes on `parent_id` and `event_time`.
`trace_messages` contains:
| Field | Type | Constraints | Purpose |
| --- | --- | --- | --- |
| `id` | `BIGINT` | Identity primary key, not null | Message identifier |
| `trace_id` | `UUID` | Not null, references `trace_contexts.trace_id` with `ON DELETE CASCADE` | Owning context |
| `event_time` | `TIMESTAMPTZ` | Not null | Message time |
| `step` | `TEXT` | Not null | Business step name |
| `status` | `TEXT` | Not null | Step result |
| `level` | `TEXT` | Not null | Business message level |
| `message` | `TEXT` | Not null | Human-readable business message |
| `attrs_json` | `JSONB` | Not null | Message attributes, including `application_id` |
Create indexes on `trace_id`, `event_time`, and `status`. The PostgreSQL
adapter only inserts rows, adds the runtime `application_id` to each
`attrs_json`, and passes database failures to the application. It does not
create tables, run migrations, publish RabbitMQ events, or expose a history API.
+52
View File
@@ -0,0 +1,52 @@
# Integrating PLBA2
The application keeps its entrypoint and composes the library before it starts
its own HTTP host. The root `config.yaml` must load successfully before
dependent components are created.
## HTTP, runtime, and health
Register `WorkerDefinition` objects in a `WorkerRegistry`, pass the registry and
`ApplicationConfigManager` to `WorkerSupervisor`, resolve an
`ApplicationIdentityResolver` result into `RuntimeContext`, and construct
`ApplicationRuntime`. Register non-worker checks in `HealthContributorRegistry`
and connect it to `HealthAggregator` with `RuntimeHealthInputProvider`.
Mount `ControlApi(runtime, health_aggregator)` in the application's ASGI host.
The host owns the socket and listener; the adapter contributes `/health`,
`/runtime/start`, `/runtime/stop`, and `/runtime/status` to that same listener.
The application chooses when to start and stop the runtime.
## Configuration and settings
Pass an application decoder to `ApplicationConfigLoader`. Its typed model
validates application-specific fields while PLBA2 validates `application_id`
and worker settings. `ApplicationConfigManager` implements
`WorkerConfigReader`; its watcher atomically swaps valid snapshots and passes
reload failures to the application's callback.
Create `settings` and seed its initial values in an application migration. Pass
the application's pool and schema to `PostgresSettingsRepository`, then expose
it through `SettingsReader` and typed `SettingKey` definitions. The adapter
filters enabled rows and returns values in `sort_order`, `id` order. The full
schema contract is in [database-contracts.md](database-contracts.md).
## Workflow and business trace
Define a `WorkflowDefinition` from named `WorkflowNode` objects and explicit
transition maps. Give `WorkflowEngine` an application-owned implementation of
`WorkflowPersistence`; that adapter decides how and where to store snapshots.
PLBA2 does not connect workflow persistence to a database itself.
Create `trace_contexts` and `trace_messages` in the application's migrations.
Pass the application's pool, schema, and resolved application ID to
`PostgresBusinessTraceWriter`. Bind the public trace ID with
`business_trace_context(trace_id)` around async business work so technical logs
inherit it.
## Technical logs
Attach `create_stdout_handler(runtime_context.application_id)` to the
application's logger. It uses the standard library `StreamHandler` and emits
one JSON object per record. The platform remains responsible for log delivery
and retention.
+5 -2
View File
@@ -5,10 +5,13 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "plba-v2" name = "plba-v2"
version = "0.1.0" version = "0.1.0"
description = "Portable lifecycle and observability runtime for business microservices" description = "Portable application lifecycle and observability components"
readme = "README.md" readme = "README.md"
requires-python = ">=3.11" requires-python = ">=3.11"
dependencies = [] dependencies = ["PyYAML>=6.0"]
[project.optional-dependencies]
test = ["pytest>=8.0"]
[tool.setuptools.packages.find] [tool.setuptools.packages.find]
where = ["src"] where = ["src"]
+5
View File
@@ -0,0 +1,5 @@
"""Portable lifecycle and observability components for Python applications."""
from plba.runtime.context import RuntimeContext
__all__ = ["RuntimeContext"]
Binary file not shown.
+18
View File
@@ -0,0 +1,18 @@
from plba.config.loader import ApplicationConfigLoadError, ApplicationConfigLoader
from plba.config.manager import (
ApplicationConfigManager,
ApplicationConfigUnavailable,
ConfigReloadResult,
ConfigReloadStatus,
)
from plba.config.models import ApplicationConfigSnapshot
__all__ = [
"ApplicationConfigLoadError",
"ApplicationConfigLoader",
"ApplicationConfigManager",
"ApplicationConfigSnapshot",
"ApplicationConfigUnavailable",
"ConfigReloadResult",
"ConfigReloadStatus",
]
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+106
View File
@@ -0,0 +1,106 @@
from collections.abc import Callable, Mapping
import math
from pathlib import Path
from typing import Any, Generic, TypeVar
import yaml
from plba.config.models import LoadedApplicationConfig
from plba.runtime.worker.config import WorkerConfig, WorkerConfigurationError
T = TypeVar("T")
ApplicationConfigDecoder = Callable[[Mapping[str, Any]], T]
class ApplicationConfigLoadError(ValueError):
pass
class ApplicationConfigLoader(Generic[T]):
_WORKER_FIELDS = {"disabled", "interval_seconds", "pause_seconds"}
def __init__(
self, config_path: Path, application_decoder: ApplicationConfigDecoder[T]
) -> None:
self._config_path = config_path
self._application_decoder = application_decoder
@property
def config_path(self) -> Path:
return self._config_path
def signature(self) -> tuple[int, int, int, int]:
stat = self._config_path.stat()
return stat.st_mtime_ns, stat.st_ctime_ns, stat.st_size, stat.st_ino
def load(self) -> LoadedApplicationConfig[T]:
try:
with self._config_path.open("r", encoding="utf-8") as config_file:
raw_config = yaml.safe_load(config_file)
except (OSError, UnicodeDecodeError, yaml.YAMLError) as error:
raise ApplicationConfigLoadError("could not read config.yaml") from error
if not isinstance(raw_config, dict):
raise ApplicationConfigLoadError("config.yaml root must be a mapping")
try:
application_id = self._application_id(raw_config)
workers = self._workers(raw_config)
application_config = self._application_decoder(raw_config)
except ApplicationConfigLoadError:
raise
except Exception as error:
raise ApplicationConfigLoadError("config.yaml does not match the application model") from error
return LoadedApplicationConfig(application_id, workers, application_config)
@staticmethod
def _application_id(raw_config: Mapping[str, Any]) -> str | None:
if "application_id" not in raw_config:
return None
application_id = raw_config["application_id"]
if not isinstance(application_id, str):
raise ApplicationConfigLoadError("application_id must be a string")
return application_id
@staticmethod
def _workers(raw_config: Mapping[str, Any]) -> dict[str, WorkerConfig]:
raw_workers = raw_config.get("workers", {})
if not isinstance(raw_workers, dict):
raise ApplicationConfigLoadError("workers must be a mapping")
workers: dict[str, WorkerConfig] = {}
for name, raw_settings in raw_workers.items():
if not isinstance(name, str) or not name or name.strip() != name:
raise ApplicationConfigLoadError("worker names must be non-empty strings")
if not isinstance(raw_settings, dict):
raise ApplicationConfigLoadError(f"workers.{name} must be a mapping")
unknown = set(raw_settings) - ApplicationConfigLoader._WORKER_FIELDS
if unknown:
raise ApplicationConfigLoadError(
f"workers.{name} contains unsupported fields: {sorted(unknown)}"
)
workers[name] = ApplicationConfigLoader._worker_config(name, raw_settings)
return workers
@staticmethod
def _worker_config(name: str, raw: Mapping[str, Any]) -> WorkerConfig:
disabled = raw.get("disabled", False)
if not isinstance(disabled, bool):
raise ApplicationConfigLoadError(f"workers.{name}.disabled must be boolean")
interval = ApplicationConfigLoader._duration(name, "interval_seconds", raw)
pause = ApplicationConfigLoader._duration(name, "pause_seconds", raw)
config = WorkerConfig(disabled, interval, pause)
try:
config.validate_schedule()
except WorkerConfigurationError as error:
raise ApplicationConfigLoadError(f"workers.{name} has an invalid schedule") from error
return config
@staticmethod
def _duration(name: str, field: str, raw: Mapping[str, Any]) -> float | None:
if field not in raw:
return None
value = raw[field]
if isinstance(value, bool) or not isinstance(value, (int, float)):
raise ApplicationConfigLoadError(f"workers.{name}.{field} must be numeric")
duration = float(value)
if not math.isfinite(duration):
raise ApplicationConfigLoadError(f"workers.{name}.{field} must be finite")
return duration
+161
View File
@@ -0,0 +1,161 @@
import asyncio
import math
from collections.abc import Awaitable, Callable
from copy import deepcopy
from dataclasses import dataclass
from enum import StrEnum
import logging
from threading import RLock
from typing import Generic, TypeVar
from plba.config.loader import ApplicationConfigLoadError, ApplicationConfigLoader
from plba.config.models import ApplicationConfigSnapshot, LoadedApplicationConfig
from plba.runtime.worker.config import WorkerConfig
T = TypeVar("T")
class ConfigReloadStatus(StrEnum):
UNCHANGED = "unchanged"
RELOADED = "reloaded"
FAILED = "failed"
@dataclass(frozen=True, slots=True)
class ConfigReloadResult(Generic[T]):
status: ConfigReloadStatus
snapshot: ApplicationConfigSnapshot[T] | None = None
error: Exception | None = None
class ApplicationConfigUnavailable(RuntimeError):
pass
class ApplicationConfigManager(Generic[T]):
def __init__(
self, loader: ApplicationConfigLoader[T], logger: logging.Logger | None = None
) -> None:
self._loader = loader
self._logger = logger or logging.getLogger(__name__)
self._lock = RLock()
self._snapshot: ApplicationConfigSnapshot[T] | None = None
self._revision = 0
self._last_seen_signature: tuple[int, int, int, int] | None = None
def load_initial(self) -> ApplicationConfigSnapshot[T]:
try:
loaded = self._loader.load()
signature = self._loader.signature()
except Exception as error:
self._logger.error(
"initial application config load failed",
extra={"attributes": {"error_type": type(error).__name__}},
)
raise
with self._lock:
self._revision += 1
self._snapshot = self._snapshot_from(loaded)
self._last_seen_signature = signature
snapshot = self.current()
self._logger.info(
"initial application config loaded",
extra={"attributes": {"revision": snapshot.revision}},
)
return snapshot
def current(self) -> ApplicationConfigSnapshot[T]:
with self._lock:
if self._snapshot is None:
raise ApplicationConfigUnavailable("application config is not loaded")
snapshot = self._snapshot
return ApplicationConfigSnapshot(
application_id=snapshot.application_id,
workers=snapshot.workers,
application_config=deepcopy(snapshot.application_config),
revision=snapshot.revision,
)
def get_worker_config(self, worker_name: str) -> WorkerConfig:
with self._lock:
if self._snapshot is None:
raise ApplicationConfigUnavailable("application config is not loaded")
return self._snapshot.workers.get(worker_name, WorkerConfig())
def reload_if_changed(self) -> ConfigReloadResult[T]:
try:
signature = self._loader.signature()
except OSError as error:
with self._lock:
if self._last_seen_signature is None:
return ConfigReloadResult(ConfigReloadStatus.UNCHANGED)
return self._failure(error, None)
with self._lock:
if signature == self._last_seen_signature:
return ConfigReloadResult(ConfigReloadStatus.UNCHANGED)
self._last_seen_signature = signature
try:
loaded = self._loader.load()
except ApplicationConfigLoadError as error:
return self._failure(error, signature)
with self._lock:
self._revision += 1
self._snapshot = self._snapshot_from(loaded)
snapshot = self.current()
self._logger.info(
"application config reloaded",
extra={"attributes": {"revision": snapshot.revision}},
)
return ConfigReloadResult(ConfigReloadStatus.RELOADED, snapshot=snapshot)
async def watch(
self,
stop_event: asyncio.Event,
interval_seconds: float,
on_reload_failed: Callable[[Exception], Awaitable[None]],
sleep: Callable[[float], Awaitable[None]] = asyncio.sleep,
) -> None:
if (
isinstance(interval_seconds, bool)
or not isinstance(interval_seconds, (int, float))
or not math.isfinite(interval_seconds)
or interval_seconds <= 0
):
raise ValueError("interval_seconds must be a positive finite number")
while not stop_event.is_set():
sleep_task = asyncio.ensure_future(sleep(interval_seconds))
stop_task = asyncio.create_task(stop_event.wait())
done, pending = await asyncio.wait(
(sleep_task, stop_task), return_when=asyncio.FIRST_COMPLETED
)
if sleep_task in done:
await sleep_task
for task in pending:
task.cancel()
await asyncio.gather(*pending, return_exceptions=True)
if stop_task in done:
break
result = await asyncio.to_thread(self.reload_if_changed)
if result.status is ConfigReloadStatus.FAILED and result.error is not None:
await on_reload_failed(result.error)
def _failure(
self, error: Exception, signature: tuple[int, int, int, int] | None
) -> ConfigReloadResult[T]:
with self._lock:
self._last_seen_signature = signature
self._logger.error(
"application config reload failed",
extra={"attributes": {"error_type": type(error).__name__}},
)
return ConfigReloadResult(ConfigReloadStatus.FAILED, error=error)
def _snapshot_from(
self, loaded: LoadedApplicationConfig[T]
) -> ApplicationConfigSnapshot[T]:
return ApplicationConfigSnapshot(
application_id=loaded.application_id,
workers=loaded.workers,
application_config=loaded.application_config,
revision=self._revision,
)
+27
View File
@@ -0,0 +1,27 @@
from collections.abc import Mapping
from dataclasses import dataclass
from types import MappingProxyType
from typing import Generic, TypeVar
from plba.runtime.worker.config import WorkerConfig
T = TypeVar("T")
@dataclass(frozen=True, slots=True)
class ApplicationConfigSnapshot(Generic[T]):
application_id: str | None
workers: Mapping[str, WorkerConfig]
application_config: T
revision: int
def __post_init__(self) -> None:
object.__setattr__(self, "workers", MappingProxyType(dict(self.workers)))
@dataclass(frozen=True, slots=True)
class LoadedApplicationConfig(Generic[T]):
application_id: str | None
workers: Mapping[str, WorkerConfig]
application_config: T
+27
View File
@@ -0,0 +1,27 @@
from plba.health.aggregator import HealthAggregator, HealthInputInvalid, HealthInputReader
from plba.health.contributors import (
DuplicateHealthContributorError,
HealthContributor,
HealthContributorRegistry,
)
from plba.health.models import (
HealthComponent,
HealthExpectation,
HealthInput,
HealthReport,
)
from plba.health.provider import RuntimeHealthInputProvider
__all__ = [
"DuplicateHealthContributorError",
"HealthAggregator",
"HealthComponent",
"HealthContributor",
"HealthContributorRegistry",
"HealthExpectation",
"HealthInput",
"HealthInputInvalid",
"HealthInputReader",
"HealthReport",
"RuntimeHealthInputProvider",
]
Binary file not shown.
Binary file not shown.
Binary file not shown.
+118
View File
@@ -0,0 +1,118 @@
import json
from collections.abc import Mapping
from typing import Protocol
from plba.health.models import (
HealthComponent,
HealthExpectation,
HealthInput,
HealthReport,
)
from plba.runtime.worker.models import HealthStatus
class HealthInputInvalid(ValueError):
pass
class HealthInputReader(Protocol):
async def read(self) -> HealthInput: ...
class HealthAggregator:
_LIFECYCLES = {"new", "starting", "running", "stopping", "stopped", "failed"}
def __init__(self, input_provider: HealthInputReader) -> None:
self._input_provider = input_provider
async def aggregate(self) -> HealthReport:
health_input = await self._input_provider.read()
self._validate(health_input)
components = self._fill_missing(health_input)
status = self._aggregate_status(health_input.lifecycle, components)
return HealthReport(
application_id=health_input.application_id,
lifecycle=health_input.lifecycle,
status=status,
components=tuple(sorted(components, key=lambda item: item.name)),
)
@staticmethod
def _validate(health_input: HealthInput) -> None:
if not isinstance(health_input, HealthInput):
raise HealthInputInvalid("health input has an unsupported type")
if not isinstance(health_input.application_id, str) or not health_input.application_id:
raise HealthInputInvalid("application_id must be a non-empty string")
if not isinstance(health_input.lifecycle, str) or health_input.lifecycle not in HealthAggregator._LIFECYCLES:
raise HealthInputInvalid("lifecycle is not supported")
if any(not isinstance(item, HealthExpectation) for item in health_input.expected_components):
raise HealthInputInvalid("expected component has an unsupported type")
if any(not isinstance(item, HealthComponent) for item in health_input.components):
raise HealthInputInvalid("component has an unsupported type")
expected_names = [item.name for item in health_input.expected_components]
component_names = [item.name for item in health_input.components]
if any(not isinstance(name, str) for name in expected_names + component_names):
raise HealthInputInvalid("component names must be strings")
if any(not isinstance(item.critical, bool) for item in health_input.expected_components):
raise HealthInputInvalid("expected component critical flags must be boolean")
if len(expected_names) != len(set(expected_names)):
raise HealthInputInvalid("expected component names must be unique")
if len(component_names) != len(set(component_names)):
raise HealthInputInvalid("component names must be unique")
if any(not name for name in expected_names + component_names):
raise HealthInputInvalid("component names must be non-empty")
for component in health_input.components:
if not isinstance(component.status, HealthStatus):
raise HealthInputInvalid(f"invalid status for component {component.name}")
if not isinstance(component.critical, bool):
raise HealthInputInvalid(f"invalid critical flag for {component.name}")
if component.detail is not None and not isinstance(component.detail, str):
raise HealthInputInvalid(f"invalid detail for component {component.name}")
if not isinstance(component.meta, Mapping):
raise HealthInputInvalid(f"invalid meta for component {component.name}")
if any(not isinstance(key, str) for key in component.meta):
raise HealthInputInvalid(f"meta keys must be strings for {component.name}")
try:
json.dumps(dict(component.meta), allow_nan=False)
except (TypeError, ValueError) as error:
raise HealthInputInvalid(
f"meta is not JSON serializable for component {component.name}"
) from error
@staticmethod
def _fill_missing(health_input: HealthInput) -> list[HealthComponent]:
components = {item.name: item for item in health_input.components}
for expected in health_input.expected_components:
if expected.name not in components:
components[expected.name] = HealthComponent(
name=expected.name,
status=(
HealthStatus.UNHEALTHY
if expected.critical
else HealthStatus.DEGRADED
),
critical=expected.critical,
detail="component has not reported health",
)
return list(components.values())
@staticmethod
def _aggregate_status(lifecycle: str, components: list[HealthComponent]) -> str:
if lifecycle == "stopped":
return "stopped"
statuses = [HealthStatus.OK]
if lifecycle in {"new", "failed"}:
statuses.append(HealthStatus.UNHEALTHY)
elif lifecycle in {"starting", "stopping"}:
statuses.append(HealthStatus.DEGRADED)
order = {
HealthStatus.OK: 0,
HealthStatus.DEGRADED: 1,
HealthStatus.UNHEALTHY: 2,
}
for component in components:
status = component.status
if status is HealthStatus.UNHEALTHY and not component.critical:
status = HealthStatus.DEGRADED
statuses.append(status)
return max(statuses, key=order.__getitem__).value
+35
View File
@@ -0,0 +1,35 @@
from collections.abc import Awaitable, Callable
from dataclasses import dataclass
from plba.runtime.worker.models import ComponentHealth
HealthReader = Callable[[], Awaitable[ComponentHealth]]
@dataclass(frozen=True, slots=True)
class HealthContributor:
name: str
health_reader: HealthReader
critical: bool = True
class DuplicateHealthContributorError(ValueError):
pass
class HealthContributorRegistry:
def __init__(self) -> None:
self._contributors: dict[str, HealthContributor] = {}
def register(self, contributor: HealthContributor) -> None:
if not contributor.name or contributor.name.strip() != contributor.name:
raise ValueError("health contributor name must be non-empty")
if contributor.name in self._contributors:
raise DuplicateHealthContributorError(
f"health contributor already registered: {contributor.name}"
)
self._contributors[contributor.name] = contributor
def all(self) -> tuple[HealthContributor, ...]:
return tuple(self._contributors.values())
+37
View File
@@ -0,0 +1,37 @@
from collections.abc import Mapping
from dataclasses import dataclass, field
from typing import Any
from plba.runtime.worker.models import HealthStatus
@dataclass(frozen=True, slots=True)
class HealthExpectation:
name: str
critical: bool
@dataclass(frozen=True, slots=True)
class HealthComponent:
name: str
status: HealthStatus
critical: bool
detail: str | None = None
meta: Mapping[str, Any] = field(default_factory=dict)
@dataclass(frozen=True, slots=True)
class HealthInput:
application_id: str
lifecycle: str
expected_components: tuple[HealthExpectation, ...]
components: tuple[HealthComponent, ...]
@dataclass(frozen=True, slots=True)
class HealthReport:
application_id: str
lifecycle: str
status: str
components: tuple[HealthComponent, ...]
+65
View File
@@ -0,0 +1,65 @@
import asyncio
from plba.health.contributors import HealthContributorRegistry
from plba.health.models import HealthComponent, HealthExpectation, HealthInput
from plba.runtime.application import ApplicationRuntime
from plba.runtime.worker.models import ComponentHealth
from plba.runtime.worker.supervisor import WorkerSupervisor
class RuntimeHealthInputProvider:
def __init__(
self,
runtime: ApplicationRuntime,
worker_supervisor: WorkerSupervisor,
contributors: HealthContributorRegistry,
) -> None:
self._runtime = runtime
self._worker_supervisor = worker_supervisor
self._contributors = contributors
async def read(self) -> HealthInput:
worker_expectations = self._worker_supervisor.expected_workers()
contributor_definitions = self._contributors.all()
expected = tuple(
[HealthExpectation(name, critical) for name, critical in worker_expectations]
+ [
HealthExpectation(item.name, item.critical)
for item in contributor_definitions
]
)
actual = [
HealthComponent(
name=snapshot.name,
status=snapshot.health.status,
critical=snapshot.health.critical,
detail=snapshot.health.detail,
meta=snapshot.health.meta,
)
for snapshot in self._worker_supervisor.snapshots()
]
contributor_health = await asyncio.gather(
*(item.health_reader() for item in contributor_definitions)
)
actual.extend(
self._component_from_contributor(item.name, item.critical, health)
for item, health in zip(contributor_definitions, contributor_health)
)
return HealthInput(
application_id=self._runtime.context.application_id,
lifecycle=self._runtime.lifecycle.value,
expected_components=expected,
components=tuple(actual),
)
@staticmethod
def _component_from_contributor(
name: str, critical: bool, health: ComponentHealth
) -> HealthComponent:
return HealthComponent(
name=name,
status=health.status,
critical=critical,
detail=health.detail,
meta=dict(health.meta),
)
+3
View File
@@ -0,0 +1,3 @@
from plba.http.control_api import ControlApi
__all__ = ["ControlApi"]
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+41
View File
@@ -0,0 +1,41 @@
import json
from typing import Any
from plba.http.errors import ControlRequestError
class ASGIHttpTransport:
async def read_body(self, receive: Any) -> bytes:
chunks: list[bytes] = []
while True:
message = await receive()
if message.get("type") == "http.disconnect":
raise ControlRequestError("client disconnected before request completed")
if message.get("type") != "http.request":
continue
chunks.append(message.get("body", b""))
if not message.get("more_body", False):
return b"".join(chunks)
async def send_json(self, send: Any, status: int, payload: dict[str, Any]) -> None:
body = json.dumps(payload, allow_nan=False).encode("utf-8")
await send(
{
"type": "http.response.start",
"status": status,
"headers": [
(b"content-type", b"application/json; charset=utf-8"),
(b"content-length", str(len(body)).encode("ascii")),
],
}
)
await send({"type": "http.response.body", "body": body})
async def handle_lifespan(self, receive: Any, send: Any) -> None:
while True:
message = await receive()
if message.get("type") == "lifespan.startup":
await send({"type": "lifespan.startup.complete"})
elif message.get("type") == "lifespan.shutdown":
await send({"type": "lifespan.shutdown.complete"})
return
+81
View File
@@ -0,0 +1,81 @@
from typing import Any
from plba.health.aggregator import HealthAggregator, HealthInputInvalid
from plba.http.asgi import ASGIHttpTransport
from plba.http.errors import ControlRequestError
from plba.http.requests import StopRequest
from plba.http.serialization import health_report_payload
from plba.runtime.application import ApplicationRuntime, RuntimeLifecycleError
class ControlApi:
"""ASGI route adapter; it owns no server, socket, or listener."""
def __init__(
self, runtime: ApplicationRuntime, health_aggregator: HealthAggregator
) -> None:
self._runtime = runtime
self._health_aggregator = health_aggregator
self._transport = ASGIHttpTransport()
async def __call__(self, scope: dict[str, Any], receive: Any, send: Any) -> None:
scope_type = scope.get("type")
if scope_type == "lifespan":
await self._transport.handle_lifespan(receive, send)
return
if scope_type != "http":
return
try:
status, payload = await self._dispatch(scope, receive)
except HealthInputInvalid as error:
status, payload = 500, self._error("HEALTH_INPUT_INVALID", error)
except RuntimeLifecycleError as error:
status, payload = 409, self._error("APPLICATION_CONTROL_CONFLICT", error)
except ControlRequestError as error:
status, payload = 400, self._error("HTTP_CONTROL_REJECTED", error)
except Exception as error:
status, payload = 500, self._error("CONTROL_API_FAILED", error)
await self._transport.send_json(send, status, payload)
async def _dispatch(self, scope: dict[str, Any], receive: Any) -> tuple[int, dict[str, Any]]:
method = scope.get("method", "").upper()
path = scope.get("path", "")
query = scope.get("query_string", b"")
if path in {"/health", "/runtime/status", "/runtime/start", "/runtime/stop"} and query:
raise ControlRequestError("query parameters are not accepted")
if path == "/health":
if method != "GET":
return 405, self._error("METHOD_NOT_ALLOWED", "GET required")
await self._require_empty_body(receive)
report = await self._health_aggregator.aggregate()
return 200, health_report_payload(report)
if path == "/runtime/start":
if method != "POST":
return 405, self._error("METHOD_NOT_ALLOWED", "POST required")
await self._require_empty_body(receive)
result = await self._runtime.start()
return 200, {"status": result.lifecycle.value}
if path == "/runtime/status":
if method != "GET":
return 405, self._error("METHOD_NOT_ALLOWED", "GET required")
await self._require_empty_body(receive)
return 200, {"status": self._runtime.lifecycle.value}
if path == "/runtime/stop":
if method != "POST":
return 405, self._error("METHOD_NOT_ALLOWED", "POST required")
request = StopRequest.parse(await self._transport.read_body(receive))
result = await self._runtime.stop(
force=request.force, timeout_seconds=request.timeout_seconds
)
status_code = 200 if result.lifecycle.value == "stopped" else 202
return status_code, {"status": result.lifecycle.value}
return 404, self._error("NOT_FOUND", "control route not found")
async def _require_empty_body(self, receive: Any) -> None:
if await self._transport.read_body(receive):
raise ControlRequestError("request body is not accepted")
@staticmethod
def _error(code: str, error: Exception | str) -> dict[str, str]:
return {"error": code, "message": str(error)}
+2
View File
@@ -0,0 +1,2 @@
class ControlRequestError(ValueError):
pass
+39
View File
@@ -0,0 +1,39 @@
import json
import math
from dataclasses import dataclass
from typing import Any
from plba.http.errors import ControlRequestError
@dataclass(frozen=True, slots=True)
class StopRequest:
force: bool = False
timeout_seconds: float | None = None
@classmethod
def parse(cls, body: bytes) -> "StopRequest":
if not body:
raise ControlRequestError("stop request must contain a JSON object")
try:
decoded: Any = json.loads(body.decode("utf-8"))
except (json.JSONDecodeError, UnicodeDecodeError) as error:
raise ControlRequestError("stop request must contain valid UTF-8 JSON") from error
if not isinstance(decoded, dict):
raise ControlRequestError("stop request must be a JSON object")
if set(decoded) - {"force", "timeout_seconds"}:
raise ControlRequestError("stop request contains unsupported fields")
force = decoded.get("force", False)
timeout_seconds = decoded.get("timeout_seconds")
if not isinstance(force, bool):
raise ControlRequestError("force must be a boolean")
if "timeout_seconds" in decoded and timeout_seconds is None:
raise ControlRequestError("timeout_seconds cannot be null")
if timeout_seconds is not None and (
isinstance(timeout_seconds, bool)
or not isinstance(timeout_seconds, (int, float))
or not math.isfinite(timeout_seconds)
or timeout_seconds <= 0
):
raise ControlRequestError("timeout_seconds must be a positive finite number")
return cls(force=force, timeout_seconds=timeout_seconds)
+21
View File
@@ -0,0 +1,21 @@
from typing import Any
from plba.health.models import HealthReport
def health_report_payload(report: HealthReport) -> dict[str, Any]:
return {
"application_id": report.application_id,
"lifecycle": report.lifecycle,
"status": report.status,
"components": [
{
"name": item.name,
"status": item.status.value,
"critical": item.critical,
"detail": item.detail,
"meta": dict(item.meta),
}
for item in report.components
],
}
+3
View File
@@ -0,0 +1,3 @@
from plba.logging.technical import TechnicalJsonFormatter, create_stdout_handler
__all__ = ["TechnicalJsonFormatter", "create_stdout_handler"]
+57
View File
@@ -0,0 +1,57 @@
from collections.abc import Mapping, Callable
from datetime import datetime, timezone
import json
import logging
import sys
from typing import Any, TextIO
from plba.tracing.context import current_business_trace_id
class TechnicalJsonFormatter(logging.Formatter):
def __init__(
self,
application_id: str,
trace_id_reader: Callable[[], str | None] = current_business_trace_id,
) -> None:
super().__init__()
if not application_id:
raise ValueError("application_id must be non-empty")
self._application_id = application_id
self._trace_id_reader = trace_id_reader
def format(self, record: logging.LogRecord) -> str:
timestamp = datetime.fromtimestamp(record.created, timezone.utc)
payload: dict[str, Any] = {
"timestamp": timestamp.isoformat(timespec="milliseconds").replace(
"+00:00", "Z"
),
"level": record.levelname,
"logger": record.name,
"message": record.getMessage(),
"application_id": self._application_id,
}
attributes = getattr(record, "attributes", None)
if attributes is not None:
if not isinstance(attributes, Mapping) or any(
not isinstance(key, str) for key in attributes
):
raise ValueError("log attributes must be a string-keyed object")
payload["attributes"] = dict(attributes)
business_trace_id = self._trace_id_reader()
if business_trace_id is not None:
payload["business_trace_id"] = business_trace_id
try:
return json.dumps(
payload, ensure_ascii=False, allow_nan=False, separators=(",", ":")
)
except (TypeError, ValueError) as error:
raise ValueError("technical log attributes must be JSON-safe") from error
def create_stdout_handler(
application_id: str, stream: TextIO | None = None
) -> logging.StreamHandler[TextIO]:
handler = logging.StreamHandler(stream if stream is not None else sys.stdout)
handler.setFormatter(TechnicalJsonFormatter(application_id))
return handler
+25
View File
@@ -0,0 +1,25 @@
from plba.runtime.application import (
ApplicationLifecycle,
ApplicationRuntime,
ApplicationStartResult,
ApplicationStopResult,
RuntimeLifecycleError,
)
from plba.runtime.application_identity import (
ApplicationIdentityResolver,
ProjectNameReader,
TomlProjectNameReader,
)
from plba.runtime.context import RuntimeContext
__all__ = [
"ApplicationLifecycle",
"ApplicationIdentityResolver",
"ApplicationRuntime",
"ApplicationStartResult",
"ApplicationStopResult",
"ProjectNameReader",
"RuntimeContext",
"RuntimeLifecycleError",
"TomlProjectNameReader",
]
Binary file not shown.
+88
View File
@@ -0,0 +1,88 @@
import logging
from plba.runtime.context import RuntimeContext
from plba.runtime.lifecycle import (
ApplicationLifecycle,
ApplicationStartResult,
ApplicationStopResult,
RuntimeLifecycleError,
)
from plba.runtime.stop_coordinator import ApplicationStopCoordinator
from plba.runtime.worker.supervisor import WorkerSupervisor
class ApplicationRuntime:
def __init__(
self,
context: RuntimeContext,
worker_supervisor: WorkerSupervisor,
logger: logging.Logger | None = None,
) -> None:
self.context = context
self._worker_supervisor = worker_supervisor
self._logger = logger or logging.getLogger(__name__)
self._lifecycle = ApplicationLifecycle.NEW
self._stop_coordinator = ApplicationStopCoordinator(
worker_supervisor,
lambda: self._lifecycle,
self._set_lifecycle,
context.application_id,
self._logger,
)
@property
def lifecycle(self) -> ApplicationLifecycle:
return self._lifecycle
async def start(self) -> ApplicationStartResult:
if self._lifecycle not in {
ApplicationLifecycle.NEW,
ApplicationLifecycle.STOPPED,
}:
raise RuntimeLifecycleError(
f"application cannot start from {self._lifecycle.value}"
)
self._lifecycle = ApplicationLifecycle.STARTING
self._logger.info(
"application start requested",
extra={"attributes": {"application_id": self.context.application_id}},
)
try:
workers = await self._worker_supervisor.start_all()
except Exception as error:
self._lifecycle = ApplicationLifecycle.FAILED
self._logger.error(
"application start failed",
extra={
"attributes": {
"application_id": self.context.application_id,
"error_type": type(error).__name__,
}
},
)
raise
self._lifecycle = (
ApplicationLifecycle.RUNNING
if workers.successful
else ApplicationLifecycle.FAILED
)
self._logger.log(
logging.INFO if workers.successful else logging.ERROR,
"application start completed",
extra={
"attributes": {
"application_id": self.context.application_id,
"lifecycle": self._lifecycle.value,
"started_workers": list(workers.started),
"failed_workers": list(workers.failures),
}
},
)
return ApplicationStartResult(self._lifecycle, workers)
async def stop(
self, force: bool = False, timeout_seconds: float | None = None
) -> ApplicationStopResult:
return await self._stop_coordinator.stop(force, timeout_seconds)
def _set_lifecycle(self, lifecycle: ApplicationLifecycle) -> None:
self._lifecycle = lifecycle
+54
View File
@@ -0,0 +1,54 @@
from collections.abc import Callable
from pathlib import Path
import tomllib
from typing import Protocol
from uuid import UUID, uuid4
from plba.runtime.context import RuntimeContext
class ProjectNameReader(Protocol):
def read_project_name(self) -> str | None: ...
class TomlProjectNameReader:
def __init__(self, metadata_path: Path) -> None:
self._metadata_path = metadata_path
def read_project_name(self) -> str | None:
try:
with self._metadata_path.open("rb") as metadata_file:
metadata = tomllib.load(metadata_file)
except (OSError, tomllib.TOMLDecodeError):
return None
project = metadata.get("project")
if not isinstance(project, dict):
return None
name = project.get("name")
return name if isinstance(name, str) and name else None
class ApplicationIdentityResolver:
def __init__(
self,
project_name_reader: ProjectNameReader,
uuid_factory: Callable[[], UUID] = uuid4,
) -> None:
self._project_name_reader = project_name_reader
self._uuid_factory = uuid_factory
def resolve(self, configured_application_id: str | None = None) -> RuntimeContext:
if configured_application_id is not None and not isinstance(
configured_application_id, str
):
raise TypeError("configured_application_id must be a string or None")
if configured_application_id:
application_id = configured_application_id
else:
application_id = self._project_name_reader.read_project_name()
if not application_id:
application_id = str(self._uuid_factory())
return RuntimeContext(application_id=application_id)
+8
View File
@@ -0,0 +1,8 @@
from dataclasses import dataclass
@dataclass(frozen=True, slots=True)
class RuntimeContext:
"""Process-wide identity shared with runtime components."""
application_id: str
+32
View File
@@ -0,0 +1,32 @@
from dataclasses import dataclass
from enum import StrEnum
from plba.runtime.worker.supervisor import (
WorkerGroupStartResult,
WorkerGroupStopResult,
)
class ApplicationLifecycle(StrEnum):
NEW = "new"
STARTING = "starting"
RUNNING = "running"
STOPPING = "stopping"
STOPPED = "stopped"
FAILED = "failed"
class RuntimeLifecycleError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class ApplicationStartResult:
lifecycle: ApplicationLifecycle
workers: WorkerGroupStartResult
@dataclass(frozen=True, slots=True)
class ApplicationStopResult:
lifecycle: ApplicationLifecycle
workers: WorkerGroupStopResult
+126
View File
@@ -0,0 +1,126 @@
import asyncio
from collections.abc import Callable
import logging
import math
from plba.runtime.lifecycle import (
ApplicationLifecycle,
ApplicationStopResult,
RuntimeLifecycleError,
)
from plba.runtime.worker.supervisor import WorkerGroupStopResult, WorkerSupervisor
class ApplicationStopCoordinator:
def __init__(
self,
worker_supervisor: WorkerSupervisor,
lifecycle_reader: Callable[[], ApplicationLifecycle],
lifecycle_writer: Callable[[ApplicationLifecycle], None],
application_id: str,
logger: logging.Logger,
) -> None:
self._worker_supervisor = worker_supervisor
self._lifecycle_reader = lifecycle_reader
self._lifecycle_writer = lifecycle_writer
self._application_id = application_id
self._logger = logger
self._stop_task: asyncio.Task[WorkerGroupStopResult] | None = None
async def stop(
self, force: bool = False, timeout_seconds: float | None = None
) -> ApplicationStopResult:
self._validate_options(force, timeout_seconds)
lifecycle = self._lifecycle_reader()
if lifecycle is ApplicationLifecycle.STOPPED:
return ApplicationStopResult(lifecycle, WorkerGroupStopResult((), ()))
if lifecycle is ApplicationLifecycle.STOPPING:
if force:
self._worker_supervisor.request_stop_all(force=True)
else:
self._lifecycle_writer(ApplicationLifecycle.STOPPING)
self._logger.info(
"application stop requested",
extra={"attributes": {"application_id": self._application_id, "force": force}},
)
self._stop_task = asyncio.create_task(
self._finish_stop(force), name="application:stop"
)
if self._stop_task is None:
raise RuntimeLifecycleError("application stop operation was not created")
if timeout_seconds is None:
workers = await asyncio.shield(self._stop_task)
return ApplicationStopResult(self._lifecycle_reader(), workers)
done, _ = await asyncio.wait({self._stop_task}, timeout=timeout_seconds)
if not done:
return self._stopping_result()
return ApplicationStopResult(self._lifecycle_reader(), self._stop_task.result())
async def _finish_stop(self, force: bool) -> WorkerGroupStopResult:
try:
workers = await self._worker_supervisor.stop_all(force=force)
except Exception as error:
self._lifecycle_writer(ApplicationLifecycle.FAILED)
self._logger.error(
"application stop failed",
extra={
"attributes": {
"application_id": self._application_id,
"error_type": type(error).__name__,
}
},
)
raise
lifecycle = (
ApplicationLifecycle.STOPPED
if workers.complete
else ApplicationLifecycle.STOPPING
)
self._lifecycle_writer(lifecycle)
self._logger.info(
"application stop completed",
extra={
"attributes": {
"application_id": self._application_id,
"lifecycle": lifecycle.value,
"stopped_workers": list(workers.stopped),
}
},
)
return workers
def _stopping_result(self) -> ApplicationStopResult:
stopping = tuple(
sorted(
snapshot.name
for snapshot in self._worker_supervisor.snapshots()
if snapshot.lifecycle.value in {"starting", "running", "stopping"}
)
)
self._logger.warning(
"application stop is continuing",
extra={
"attributes": {
"application_id": self._application_id,
"workers_stopping": list(stopping),
}
},
)
return ApplicationStopResult(
ApplicationLifecycle.STOPPING, WorkerGroupStopResult((), stopping)
)
@staticmethod
def _validate_options(force: bool, timeout_seconds: float | None) -> None:
if not isinstance(force, bool):
raise ValueError("force must be a boolean")
if timeout_seconds is None:
return
if (
isinstance(timeout_seconds, bool)
or not isinstance(timeout_seconds, (int, float))
or not math.isfinite(timeout_seconds)
or timeout_seconds <= 0
):
raise ValueError("timeout_seconds must be a positive finite number")
+29
View File
@@ -0,0 +1,29 @@
from plba.runtime.worker.config import WorkerConfig, WorkerConfigReader
from plba.runtime.worker.models import (
ComponentHealth,
HealthStatus,
WorkerDefinition,
WorkerLifecycle,
WorkerSnapshot,
)
from plba.runtime.worker.registry import DuplicateWorkerError, WorkerRegistry
from plba.runtime.worker.supervisor import (
WorkerGroupStartResult,
WorkerGroupStopResult,
WorkerSupervisor,
)
__all__ = [
"ComponentHealth",
"DuplicateWorkerError",
"HealthStatus",
"WorkerConfig",
"WorkerConfigReader",
"WorkerDefinition",
"WorkerGroupStartResult",
"WorkerGroupStopResult",
"WorkerLifecycle",
"WorkerRegistry",
"WorkerSnapshot",
"WorkerSupervisor",
]
+33
View File
@@ -0,0 +1,33 @@
from dataclasses import dataclass
import math
from typing import Protocol
class WorkerConfigurationError(ValueError):
pass
@dataclass(frozen=True, slots=True)
class WorkerConfig:
disabled: bool = False
interval_seconds: float | None = None
pause_seconds: float | None = None
def validate_schedule(self) -> None:
if self.disabled:
return
has_interval = self.interval_seconds is not None
has_pause = self.pause_seconds is not None
if has_interval == has_pause:
raise WorkerConfigurationError(
"exactly one of interval_seconds or pause_seconds is required"
)
duration = self.interval_seconds if has_interval else self.pause_seconds
if isinstance(duration, bool) or not isinstance(duration, (int, float)):
raise WorkerConfigurationError("worker schedule must be a number")
if not math.isfinite(duration) or duration <= 0:
raise WorkerConfigurationError("worker schedule must be positive")
class WorkerConfigReader(Protocol):
def get_worker_config(self, worker_name: str) -> WorkerConfig: ...
+87
View File
@@ -0,0 +1,87 @@
from plba.runtime.worker.models import (
ComponentHealth,
HealthStatus,
WorkerDefinition,
)
class WorkerHealthTracker:
def __init__(self, definition: WorkerDefinition) -> None:
self._critical = definition.critical
self._health_check = definition.health_check
self._automatic_status = HealthStatus.OK
self._application_status = HealthStatus.OK
self._application_detail: str | None = None
self._application_meta: dict[str, object] = {}
self._last_error: BaseException | None = None
@property
def last_error(self) -> BaseException | None:
return self._last_error
def cycle_succeeded(self) -> None:
self._last_error = None
self._automatic_status = HealthStatus.OK
def cycle_failed(self, error: BaseException) -> None:
self._last_error = error
self._automatic_status = HealthStatus.UNHEALTHY
def runtime_failed(self, error: BaseException) -> None:
self._last_error = error
self._automatic_status = HealthStatus.UNHEALTHY
def cycle_exceeded_interval(self) -> None:
self._automatic_status = HealthStatus.DEGRADED
async def refresh_application_health(self) -> None:
if self._health_check is None:
self._application_status = HealthStatus.OK
self._application_detail = None
self._application_meta = {}
return
try:
health = await self._health_check()
except Exception as error:
self._application_status = HealthStatus.UNHEALTHY
self._application_detail = f"health check failed: {error}"
self._application_meta = {}
return
if not isinstance(health, ComponentHealth) or not isinstance(
health.status, HealthStatus
):
self._application_status = HealthStatus.UNHEALTHY
self._application_detail = "health check returned an invalid status"
self._application_meta = {}
return
self._application_status = health.status
self._application_detail = health.detail
self._application_meta = dict(health.meta)
def snapshot(self) -> ComponentHealth:
status_order = {
HealthStatus.OK: 0,
HealthStatus.DEGRADED: 1,
HealthStatus.UNHEALTHY: 2,
}
status = max(
(self._automatic_status, self._application_status),
key=status_order.__getitem__,
)
details = self._details()
return ComponentHealth(
status=status,
critical=self._critical,
detail="; ".join(details) or None,
meta=self._application_meta,
)
def _details(self) -> list[str]:
details: list[str] = []
if self._automatic_status is HealthStatus.UNHEALTHY and self._last_error:
details.append(f"worker cycle failed: {self._last_error}")
elif self._automatic_status is HealthStatus.DEGRADED:
details.append("worker cycle exceeded interval_seconds")
if self._application_detail:
details.append(self._application_detail)
return details
+48
View File
@@ -0,0 +1,48 @@
from collections.abc import Awaitable, Callable, Mapping
from dataclasses import dataclass, field
from enum import StrEnum
from typing import Any
class HealthStatus(StrEnum):
OK = "ok"
DEGRADED = "degraded"
UNHEALTHY = "unhealthy"
class WorkerLifecycle(StrEnum):
NEW = "new"
STARTING = "starting"
RUNNING = "running"
STOPPING = "stopping"
STOPPED = "stopped"
FAILED = "failed"
@dataclass(frozen=True, slots=True)
class ComponentHealth:
status: HealthStatus = HealthStatus.OK
critical: bool = True
detail: str | None = None
meta: Mapping[str, Any] = field(default_factory=dict)
HealthCheck = Callable[[], Awaitable[ComponentHealth]]
WorkerProcess = Callable[[], Awaitable[None]]
@dataclass(frozen=True, slots=True)
class WorkerDefinition:
name: str
process: WorkerProcess
critical: bool = True
health_check: HealthCheck | None = None
@dataclass(frozen=True, slots=True)
class WorkerSnapshot:
name: str
lifecycle: WorkerLifecycle
health: ComponentHealth
last_error: BaseException | None = None
+23
View File
@@ -0,0 +1,23 @@
from plba.runtime.worker.models import WorkerDefinition
class DuplicateWorkerError(ValueError):
pass
class WorkerRegistry:
def __init__(self) -> None:
self._definitions: dict[str, WorkerDefinition] = {}
def register(self, definition: WorkerDefinition) -> None:
if not definition.name or definition.name.strip() != definition.name:
raise ValueError("worker name must be non-empty and have no outer spaces")
if definition.name in self._definitions:
raise DuplicateWorkerError(f"worker already registered: {definition.name}")
self._definitions[definition.name] = definition
def get(self, name: str) -> WorkerDefinition:
return self._definitions[name]
def all(self) -> tuple[WorkerDefinition, ...]:
return tuple(self._definitions.values())
+153
View File
@@ -0,0 +1,153 @@
import asyncio
import logging
import time
from collections.abc import Awaitable, Callable
from plba.runtime.worker.config import WorkerConfig, WorkerConfigReader
from plba.runtime.worker.health import WorkerHealthTracker
from plba.runtime.worker.models import (
WorkerDefinition,
WorkerLifecycle,
WorkerSnapshot,
)
from plba.runtime.worker.schedule import WorkerScheduleController
AsyncSleep = Callable[[float], Awaitable[None]]
Clock = Callable[[], float]
class WorkerRunner:
def __init__(
self,
definition: WorkerDefinition,
config_reader: WorkerConfigReader,
sleep: AsyncSleep = asyncio.sleep,
clock: Clock = time.monotonic,
logger: logging.Logger | None = None,
) -> None:
self._definition = definition
self._config_reader = config_reader
self._sleep = sleep
self._clock = clock
self._logger = logger or logging.getLogger(__name__)
self._schedule = WorkerScheduleController(definition.name, config_reader, sleep)
self._lifecycle = WorkerLifecycle.NEW
self._health = WorkerHealthTracker(definition)
self._task: asyncio.Task[None] | None = None
self._stop_requested = asyncio.Event()
@property
def name(self) -> str:
return self._definition.name
@property
def task(self) -> asyncio.Task[None] | None:
return self._task
def start(self, config: WorkerConfig) -> None:
self._schedule.start(config)
self._lifecycle = WorkerLifecycle.STARTING
self._task = asyncio.create_task(self._run(), name=f"worker:{self.name}")
self._logger.info(
"worker started", extra={"attributes": {"worker": self.name}}
)
def request_stop(self, force: bool = False) -> None:
if self._task is None or self._task.done():
if self._lifecycle is not WorkerLifecycle.FAILED:
self._lifecycle = WorkerLifecycle.STOPPED
return
self._lifecycle = WorkerLifecycle.STOPPING
self._stop_requested.set()
if force:
self._task.cancel()
async def wait_stopped(self) -> None:
if self._task is None:
self._lifecycle = WorkerLifecycle.STOPPED
return
try:
await self._task
except asyncio.CancelledError:
self._lifecycle = WorkerLifecycle.STOPPED
self._logger.info(
"worker force-stopped", extra={"attributes": {"worker": self.name}}
)
except Exception as error:
self._health.runtime_failed(error)
self._lifecycle = WorkerLifecycle.FAILED
else:
if self._lifecycle is not WorkerLifecycle.FAILED:
self._lifecycle = WorkerLifecycle.STOPPED
def snapshot(self) -> WorkerSnapshot:
return WorkerSnapshot(
name=self.name,
lifecycle=self._lifecycle,
health=self._health.snapshot(),
last_error=self._health.last_error,
)
async def _run(self) -> None:
self._lifecycle = WorkerLifecycle.RUNNING
try:
while not self._stop_requested.is_set():
cycle_config = self._schedule.current
started_at = self._clock()
self._logger.debug(
"worker cycle started", extra={"attributes": {"worker": self.name}}
)
try:
await self._definition.process()
self._health.cycle_succeeded()
self._logger.debug(
"worker cycle completed",
extra={"attributes": {"worker": self.name}},
)
except Exception as error:
self._health.cycle_failed(error)
self._logger.error(
"worker cycle failed",
extra={
"attributes": {
"worker": self.name,
"error_type": type(error).__name__,
}
},
)
elapsed = max(0.0, self._clock() - started_at)
await self._health.refresh_application_health()
config = self._schedule.next_schedule()
if self._health.last_error is None:
if (
cycle_config.interval_seconds is not None
and elapsed > cycle_config.interval_seconds
):
self._health.cycle_exceeded_interval()
if self._stop_requested.is_set():
break
await self._schedule.wait_for_next_cycle(
config, elapsed, self._stop_requested
)
except asyncio.CancelledError:
raise
except Exception as error:
self._health.runtime_failed(error)
self._lifecycle = WorkerLifecycle.FAILED
self._logger.error(
"worker runtime failed",
extra={
"attributes": {
"worker": self.name,
"error_type": type(error).__name__,
}
},
)
return
self._lifecycle = WorkerLifecycle.STOPPED
self._logger.info(
"worker stopped", extra={"attributes": {"worker": self.name}}
)
+64
View File
@@ -0,0 +1,64 @@
import asyncio
from collections.abc import Awaitable, Callable
from dataclasses import replace
from plba.runtime.worker.config import WorkerConfig, WorkerConfigReader
AsyncSleep = Callable[[float], Awaitable[None]]
class WorkerScheduleController:
def __init__(
self,
worker_name: str,
config_reader: WorkerConfigReader,
sleep: AsyncSleep,
) -> None:
self._worker_name = worker_name
self._config_reader = config_reader
self._sleep = sleep
self._current: WorkerConfig | None = None
@property
def current(self) -> WorkerConfig:
if self._current is None:
raise RuntimeError("worker schedule has not started")
return self._current
def start(self, config: WorkerConfig) -> None:
if config.disabled:
raise ValueError(f"worker is disabled: {self._worker_name}")
config.validate_schedule()
self._current = config
def next_schedule(self) -> WorkerConfig:
config = self._config_reader.get_worker_config(self._worker_name)
if config.disabled and config.interval_seconds is None and config.pause_seconds is None:
return self.current
config = replace(config, disabled=False)
config.validate_schedule()
self._current = config
return config
async def wait_for_next_cycle(
self, config: WorkerConfig, elapsed: float, stop_requested: asyncio.Event
) -> None:
delay = self._delay(config, elapsed)
if not delay or stop_requested.is_set():
return
sleep_task = asyncio.ensure_future(self._sleep(delay))
stop_task = asyncio.create_task(stop_requested.wait())
done, pending = await asyncio.wait(
(sleep_task, stop_task), return_when=asyncio.FIRST_COMPLETED
)
if sleep_task in done:
await sleep_task
for task in pending:
task.cancel()
await asyncio.gather(*pending, return_exceptions=True)
@staticmethod
def _delay(config: WorkerConfig, elapsed: float) -> float:
if config.pause_seconds is not None:
return config.pause_seconds
return max(0.0, (config.interval_seconds or 0.0) - elapsed)
+151
View File
@@ -0,0 +1,151 @@
import asyncio
import logging
from dataclasses import dataclass, field
from types import MappingProxyType
from typing import Mapping
from plba.runtime.worker.config import WorkerConfigReader
from plba.runtime.worker.models import (
ComponentHealth,
HealthStatus,
WorkerLifecycle,
WorkerSnapshot,
)
from plba.runtime.worker.registry import WorkerRegistry
from plba.runtime.worker.runner import AsyncSleep, Clock, WorkerRunner
@dataclass(frozen=True, slots=True)
class WorkerGroupStartResult:
started: tuple[str, ...]
failures: Mapping[str, BaseException] = field(default_factory=dict)
@property
def successful(self) -> bool:
return not self.failures
@dataclass(frozen=True, slots=True)
class WorkerGroupStopResult:
stopped: tuple[str, ...]
stopping: tuple[str, ...]
@property
def complete(self) -> bool:
return not self.stopping
class WorkerSupervisor:
def __init__(
self,
registry: WorkerRegistry,
config_reader: WorkerConfigReader,
sleep: AsyncSleep = asyncio.sleep,
clock: Clock | None = None,
logger: logging.Logger | None = None,
) -> None:
self._registry = registry
self._config_reader = config_reader
self._sleep = sleep
self._clock = clock
self._logger = logger or logging.getLogger(__name__)
self._runners: dict[str, WorkerRunner] = {}
self._start_failures: dict[str, BaseException] = {}
self._expected_names: set[str] = set()
self._started_once = False
async def start_all(self) -> WorkerGroupStartResult:
started: list[str] = []
self._start_failures = {}
self._expected_names = set()
self._started_once = True
for definition in self._registry.all():
try:
config = self._config_reader.get_worker_config(definition.name)
if config.disabled:
self._runners.pop(definition.name, None)
self._logger.info(
"worker disabled for application start",
extra={"attributes": {"worker": definition.name}},
)
continue
self._expected_names.add(definition.name)
runner = WorkerRunner(
definition,
self._config_reader,
sleep=self._sleep,
**({"clock": self._clock} if self._clock is not None else {}),
logger=self._logger,
)
runner.start(config)
self._runners[definition.name] = runner
started.append(definition.name)
except Exception as error:
self._expected_names.add(definition.name)
self._start_failures[definition.name] = error
self._logger.error(
"worker start failed",
extra={
"attributes": {
"worker": definition.name,
"error_type": type(error).__name__,
}
},
)
return WorkerGroupStartResult(
started=tuple(started),
failures=MappingProxyType(dict(self._start_failures)),
)
def snapshots(self) -> tuple[WorkerSnapshot, ...]:
snapshots = {name: runner.snapshot() for name, runner in self._runners.items()}
for name, error in self._start_failures.items():
definition = self._registry.get(name)
snapshots[name] = WorkerSnapshot(
name=name,
lifecycle=WorkerLifecycle.FAILED,
health=ComponentHealth(
HealthStatus.UNHEALTHY, critical=definition.critical
),
last_error=error,
)
return tuple(snapshots[name] for name in sorted(snapshots))
def expected_workers(self) -> tuple[tuple[str, bool], ...]:
if self._started_once:
names = self._expected_names
else:
names = {
definition.name
for definition in self._registry.all()
if not self._config_reader.get_worker_config(definition.name).disabled
}
return tuple(
(name, self._registry.get(name).critical) for name in sorted(names)
)
async def stop_all(
self, force: bool = False, timeout_seconds: float | None = None
) -> WorkerGroupStopResult:
if timeout_seconds is not None and timeout_seconds <= 0:
raise ValueError("timeout_seconds must be positive")
runners = tuple(self._runners.values())
self.request_stop_all(force=force)
tasks = {runner.task: runner for runner in runners if runner.task is not None}
if not tasks:
return WorkerGroupStopResult((), ())
done, pending = await asyncio.wait(
tasks,
timeout=timeout_seconds,
return_when=asyncio.ALL_COMPLETED,
)
for task in done:
await tasks[task].wait_stopped()
stopped = tuple(sorted(tasks[task].name for task in done))
stopping = tuple(sorted(tasks[task].name for task in pending))
return WorkerGroupStopResult(stopped, stopping)
def request_stop_all(self, force: bool = False) -> None:
for runner in tuple(self._runners.values()):
runner.request_stop(force=force)
+19
View File
@@ -0,0 +1,19 @@
from plba.settings.models import (
SettingInvalidError,
SettingKey,
SettingNotFoundError,
SettingsDatabaseUnavailableError,
)
from plba.settings.postgres import PostgresSettingsRepository
from plba.settings.reader import SettingsReader
from plba.settings.repository import SettingsRepository
__all__ = [
"PostgresSettingsRepository",
"SettingInvalidError",
"SettingKey",
"SettingNotFoundError",
"SettingsDatabaseUnavailableError",
"SettingsReader",
"SettingsRepository",
]
Binary file not shown.
Binary file not shown.
+36
View File
@@ -0,0 +1,36 @@
from collections.abc import Callable
from dataclasses import dataclass
from typing import Any, Generic, TypeVar
T = TypeVar("T")
_MISSING = object()
@dataclass(frozen=True, slots=True)
class SettingKey(Generic[T]):
name: str
decoder: Callable[[str], T]
required: bool = True
default: Any = _MISSING
def __post_init__(self) -> None:
if not self.name or self.name.strip() != self.name:
raise ValueError("setting key name must be non-empty")
if self.required and self.default is not _MISSING:
raise ValueError("a required setting cannot have a default")
@property
def has_default(self) -> bool:
return self.default is not _MISSING
class SettingNotFoundError(LookupError):
pass
class SettingInvalidError(ValueError):
pass
class SettingsDatabaseUnavailableError(RuntimeError):
pass
+43
View File
@@ -0,0 +1,43 @@
from collections.abc import Mapping, Sequence
import re
from typing import Any, AsyncContextManager, Protocol
from plba.settings.models import SettingsDatabaseUnavailableError
class PostgresConnection(Protocol):
async def fetch(self, query: str, *parameters: object) -> Sequence[Mapping[str, Any]]: ...
class PostgresPool(Protocol):
def acquire(self) -> AsyncContextManager[PostgresConnection]: ...
class PostgresSettingsRepository:
_SCHEMA = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
def __init__(self, pool: PostgresPool, schema: str) -> None:
if not self._SCHEMA.fullmatch(schema):
raise ValueError("schema must be a PostgreSQL identifier")
self._pool = pool
self._table = f'"{schema}".settings'
async def fetch_values(
self, setting_key: str, config_key: str | None = None
) -> list[str]:
query = (
f"SELECT setting_value FROM {self._table} "
"WHERE setting_key = $1 AND ($2 IS NULL OR config_key = $2) "
"AND is_enabled = TRUE ORDER BY sort_order ASC, id ASC"
)
try:
async with self._pool.acquire() as connection:
rows = await connection.fetch(query, setting_key, config_key)
values = [row["setting_value"] for row in rows]
if any(not isinstance(value, str) for value in values):
raise TypeError("settings.setting_value must be text")
return values
except Exception as error:
raise SettingsDatabaseUnavailableError(
"settings query failed"
) from error
+35
View File
@@ -0,0 +1,35 @@
from typing import TypeVar
from plba.settings.models import (
SettingInvalidError,
SettingKey,
SettingNotFoundError,
)
from plba.settings.repository import SettingsRepository
T = TypeVar("T")
class SettingsReader:
def __init__(self, repository: SettingsRepository) -> None:
self._repository = repository
async def read(
self, key: SettingKey[T], config_key: str | None = None
) -> list[T]:
raw_values = await self._repository.fetch_values(key.name, config_key)
if not raw_values:
if key.has_default:
return [key.default]
if key.required:
raise SettingNotFoundError(f"required setting not found: {key.name}")
return []
decoded: list[T] = []
for index, value in enumerate(raw_values):
try:
decoded.append(key.decoder(value))
except Exception as error:
raise SettingInvalidError(
f"setting {key.name} value at index {index} is invalid"
) from error
return decoded
+7
View File
@@ -0,0 +1,7 @@
from typing import Protocol
class SettingsRepository(Protocol):
async def fetch_values(
self, setting_key: str, config_key: str | None = None
) -> list[str]: ...
+19
View File
@@ -0,0 +1,19 @@
from plba.tracing.context import business_trace_context, current_business_trace_id
from plba.tracing.models import BusinessTraceContext, BusinessTraceMessage
from plba.tracing.postgres import (
BusinessTraceContractError,
BusinessTraceWriteError,
PostgresBusinessTraceWriter,
)
from plba.tracing.repository import BusinessTraceWriter
__all__ = [
"BusinessTraceContext",
"BusinessTraceContractError",
"BusinessTraceMessage",
"BusinessTraceWriteError",
"BusinessTraceWriter",
"PostgresBusinessTraceWriter",
"business_trace_context",
"current_business_trace_id",
]
Binary file not shown.
Binary file not shown.
+23
View File
@@ -0,0 +1,23 @@
from contextlib import contextmanager
from contextvars import ContextVar, Token
from collections.abc import Iterator
from uuid import UUID
_BUSINESS_TRACE_ID: ContextVar[str | None] = ContextVar(
"plba_business_trace_id", default=None
)
def current_business_trace_id() -> str | None:
return _BUSINESS_TRACE_ID.get()
@contextmanager
def business_trace_context(business_trace_id: UUID | str) -> Iterator[None]:
if not str(business_trace_id):
raise ValueError("business_trace_id must be non-empty")
token: Token[str | None] = _BUSINESS_TRACE_ID.set(str(business_trace_id))
try:
yield
finally:
_BUSINESS_TRACE_ID.reset(token)
+52
View File
@@ -0,0 +1,52 @@
from collections.abc import Mapping
from dataclasses import dataclass, field
from datetime import datetime
from types import MappingProxyType
from typing import Any
from uuid import UUID
def _validate_event_time(value: datetime) -> None:
if value.tzinfo is None or value.utcoffset() is None:
raise ValueError("event_time must include a timezone")
@dataclass(frozen=True, slots=True)
class BusinessTraceContext:
business_trace_id: UUID
alias: str
trace_type: str
event_time: datetime
attributes: Mapping[str, Any] = field(default_factory=dict)
parent_id: UUID | None = None
def __post_init__(self) -> None:
if not isinstance(self.business_trace_id, UUID):
raise ValueError("business_trace_id must be a UUID")
if self.parent_id is not None and not isinstance(self.parent_id, UUID):
raise ValueError("parent_id must be a UUID or None")
if not isinstance(self.alias, str) or not isinstance(self.trace_type, str):
raise ValueError("alias and trace_type must be strings")
if not self.alias or not self.trace_type:
raise ValueError("alias and trace_type must be non-empty")
_validate_event_time(self.event_time)
object.__setattr__(self, "attributes", MappingProxyType(dict(self.attributes)))
@dataclass(frozen=True, slots=True)
class BusinessTraceMessage:
business_trace_id: UUID
event_time: datetime
step: str
status: str
level: str
message: str
attributes: Mapping[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
if not isinstance(self.business_trace_id, UUID):
raise ValueError("business_trace_id must be a UUID")
_validate_event_time(self.event_time)
if any(not isinstance(value, str) for value in (self.step, self.status, self.level, self.message)):
raise ValueError("step, status, level and message must be strings")
object.__setattr__(self, "attributes", MappingProxyType(dict(self.attributes)))
+92
View File
@@ -0,0 +1,92 @@
from typing import AsyncContextManager
import json
import re
from typing import Any, Protocol
from plba.tracing.models import BusinessTraceContext, BusinessTraceMessage
class BusinessTraceContractError(ValueError):
pass
class BusinessTraceWriteError(RuntimeError):
pass
class TraceConnection(Protocol):
async def execute(self, query: str, *parameters: object) -> str: ...
class TracePool(Protocol):
def acquire(self) -> AsyncContextManager[TraceConnection]: ...
class PostgresBusinessTraceWriter:
_SCHEMA = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
def __init__(self, pool: TracePool, schema: str, application_id: str) -> None:
if not self._SCHEMA.fullmatch(schema):
raise ValueError("schema must be a PostgreSQL identifier")
if not isinstance(application_id, str) or not application_id:
raise ValueError("application_id must be non-empty")
self._pool = pool
self._application_id = application_id
self._contexts = f'"{schema}".trace_contexts'
self._messages = f'"{schema}".trace_messages'
async def write_context(self, context: BusinessTraceContext) -> None:
attributes = self._json_attributes(context.attributes)
query = (
f"INSERT INTO {self._contexts} "
"(trace_id, parent_id, alias, trace_type, event_time, attrs_json) "
"VALUES ($1, $2, $3, $4, $5, $6::jsonb)"
)
await self._execute(
query,
context.business_trace_id,
context.parent_id,
context.alias,
context.trace_type,
context.event_time,
attributes,
)
async def write_message(self, message: BusinessTraceMessage) -> None:
attributes = self._json_attributes(message.attributes)
query = (
f"INSERT INTO {self._messages} "
"(trace_id, event_time, step, status, level, message, attrs_json) "
"VALUES ($1, $2, $3, $4, $5, $6, $7::jsonb)"
)
await self._execute(
query,
message.business_trace_id,
message.event_time,
message.step,
message.status,
message.level,
message.message,
attributes,
)
async def _execute(self, query: str, *parameters: object) -> None:
try:
async with self._pool.acquire() as connection:
await connection.execute(query, *parameters)
except Exception as error:
raise BusinessTraceWriteError("business trace write failed") from error
def _json_attributes(self, attributes: Any) -> str:
if not hasattr(attributes, "items"):
raise BusinessTraceContractError("attributes must be a JSON object")
values = dict(attributes)
if any(not isinstance(key, str) for key in values):
raise BusinessTraceContractError("attribute keys must be strings")
payload = {**values, "application_id": self._application_id}
try:
return json.dumps(payload, allow_nan=False)
except (TypeError, ValueError) as error:
raise BusinessTraceContractError(
"attributes must contain JSON-safe values"
) from error
+9
View File
@@ -0,0 +1,9 @@
from typing import Protocol
from plba.tracing.models import BusinessTraceContext, BusinessTraceMessage
class BusinessTraceWriter(Protocol):
async def write_context(self, context: BusinessTraceContext) -> None: ...
async def write_message(self, message: BusinessTraceMessage) -> None: ...
+23
View File
@@ -0,0 +1,23 @@
from plba.workflow.engine import WorkflowEngine
from plba.workflow.failure_recorder import WorkflowFailureRecorder
from plba.workflow.models import (
StepResult,
WorkflowContext,
WorkflowDefinition,
WorkflowNode,
WorkflowRunResult,
WorkflowTransitionError,
)
from plba.workflow.persistence import WorkflowPersistence
__all__ = [
"StepResult",
"WorkflowContext",
"WorkflowDefinition",
"WorkflowEngine",
"WorkflowFailureRecorder",
"WorkflowNode",
"WorkflowPersistence",
"WorkflowRunResult",
"WorkflowTransitionError",
]
Binary file not shown.
Binary file not shown.
+131
View File
@@ -0,0 +1,131 @@
from collections.abc import Callable, Mapping
import logging
from uuid import UUID, uuid4
from plba.workflow.models import (
StepResult,
WorkflowContext,
WorkflowDefinition,
WorkflowRunResult,
WorkflowTransitionError,
)
from plba.workflow.persistence import WorkflowPersistence
from plba.workflow.failure_recorder import WorkflowFailureRecorder
class WorkflowEngine:
def __init__(
self,
persistence: WorkflowPersistence,
id_factory: Callable[[], UUID] = uuid4,
logger: logging.Logger | None = None,
) -> None:
self._persistence = persistence
self._id_factory = id_factory
self._logger = logger or logging.getLogger(__name__)
self._failure_recorder = WorkflowFailureRecorder(
persistence, self._logger
)
async def run(
self, definition: WorkflowDefinition, context: WorkflowContext
) -> WorkflowRunResult:
definition.validate()
run_id = str(self._id_factory())
current_name = definition.initial_step
current_context = context
await self._persistence.record_run_started(
run_id, definition.name, self._snapshot(current_context)
)
self._logger.info(
"workflow run started",
extra={
"attributes": {"workflow": definition.name, "run_id": run_id}
},
)
try:
while current_name is not None:
node = definition.nodes[current_name]
self._logger.debug(
"workflow step started",
extra={
"attributes": {
"workflow": definition.name,
"run_id": run_id,
"step": current_name,
}
},
)
await self._persistence.record_step_started(
run_id, current_name, self._snapshot(current_context)
)
try:
result = await node.step(current_context)
next_name = self._next_node(node.transitions, result)
except Exception as error:
await self._failure_recorder.record_step_failure(
run_id, current_name, current_context, error
)
raise
next_context = current_context.with_updates(result.updates)
await self._persistence.record_step_completed(
run_id, current_name, self._snapshot(next_context), result
)
self._logger.debug(
"workflow step completed",
extra={
"attributes": {
"workflow": definition.name,
"run_id": run_id,
"step": current_name,
"status": result.status,
}
},
)
current_name = next_name
current_context = next_context
await self._persistence.record_run_completed(
run_id, definition.name, self._snapshot(current_context), "completed"
)
self._logger.info(
"workflow run completed",
extra={
"attributes": {
"workflow": definition.name,
"run_id": run_id,
"status": "completed",
}
},
)
return WorkflowRunResult(run_id, current_context, "completed")
except Exception as error:
await self._failure_recorder.record_run_failure(
run_id, definition.name, self._snapshot(current_context), error
)
raise
@staticmethod
def _next_node(
transitions: Mapping[str, str], result: StepResult
) -> str | None:
if not isinstance(result, StepResult):
raise TypeError("workflow step must return StepResult")
if not isinstance(result.updates, Mapping) or not isinstance(result.status, str):
raise TypeError("workflow step returned an invalid StepResult")
if result.transition is None:
if transitions:
raise WorkflowTransitionError(
"step must select one of its declared transitions"
)
return None
if result.transition not in transitions:
raise WorkflowTransitionError(
f"transition is not declared: {result.transition}"
)
return transitions[result.transition]
@staticmethod
def _snapshot(context: WorkflowContext) -> WorkflowContext:
return WorkflowContext(context.state)
+64
View File
@@ -0,0 +1,64 @@
import logging
from plba.workflow.models import WorkflowContext
from plba.workflow.persistence import WorkflowPersistence
class WorkflowFailureRecorder:
def __init__(
self, persistence: WorkflowPersistence, logger: logging.Logger
) -> None:
self._persistence = persistence
self._logger = logger
async def record_step_failure(
self,
run_id: str,
step_name: str,
context: WorkflowContext,
error: Exception,
) -> None:
try:
await self._persistence.record_step_failed(
run_id, step_name, self._snapshot(context), error
)
except Exception as persistence_error:
error.add_note(f"could not persist workflow step failure: {persistence_error}")
self._logger.error(
"workflow step failed",
extra={
"attributes": {
"run_id": run_id,
"step": step_name,
"error_type": type(error).__name__,
}
},
)
async def record_run_failure(
self,
run_id: str,
workflow_name: str,
context: WorkflowContext,
error: Exception,
) -> None:
try:
await self._persistence.record_run_failed(
run_id, workflow_name, self._snapshot(context), error
)
except Exception as persistence_error:
error.add_note(f"could not persist workflow failure: {persistence_error}")
self._logger.error(
"workflow run failed",
extra={
"attributes": {
"workflow": workflow_name,
"run_id": run_id,
"error_type": type(error).__name__,
}
},
)
@staticmethod
def _snapshot(context: WorkflowContext) -> WorkflowContext:
return WorkflowContext(context.state)
+87
View File
@@ -0,0 +1,87 @@
from collections.abc import Awaitable, Callable, Mapping
from dataclasses import dataclass, field
from copy import deepcopy
from types import MappingProxyType
from typing import Any
@dataclass(frozen=True, slots=True)
class WorkflowContext:
state: Mapping[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
state = deepcopy(dict(self.state))
if any(not isinstance(key, str) for key in state):
raise ValueError("workflow context keys must be strings")
object.__setattr__(self, "state", MappingProxyType(state))
def with_updates(self, updates: Mapping[str, Any]) -> "WorkflowContext":
return WorkflowContext({**self.state, **updates})
@dataclass(frozen=True, slots=True)
class StepResult:
transition: str | None = None
updates: Mapping[str, Any] = field(default_factory=dict)
status: str = "completed"
def __post_init__(self) -> None:
object.__setattr__(self, "updates", MappingProxyType(deepcopy(dict(self.updates)))
)
if not isinstance(self.status, str) or not self.status:
raise ValueError("step status must be a non-empty string")
if self.transition is not None and not isinstance(self.transition, str):
raise ValueError("step transition must be a string or None")
if any(not isinstance(key, str) for key in self.updates):
raise ValueError("step update keys must be strings")
WorkflowStep = Callable[[WorkflowContext], Awaitable[StepResult]]
@dataclass(frozen=True, slots=True)
class WorkflowNode:
step: WorkflowStep
transitions: Mapping[str, str] = field(default_factory=dict)
def __post_init__(self) -> None:
object.__setattr__(self, "transitions", MappingProxyType(dict(self.transitions)))
@dataclass(frozen=True, slots=True)
class WorkflowDefinition:
name: str
initial_step: str
nodes: Mapping[str, WorkflowNode]
def __post_init__(self) -> None:
object.__setattr__(self, "nodes", MappingProxyType(dict(self.nodes)))
self.validate()
def validate(self) -> None:
if not self.name or self.name.strip() != self.name:
raise ValueError("workflow name must be non-empty")
if not self.nodes or self.initial_step not in self.nodes:
raise ValueError("workflow initial_step must name a declared node")
for node_name, node in self.nodes.items():
if not node_name or node_name.strip() != node_name:
raise ValueError("workflow node names must be non-empty")
for transition, target in node.transitions.items():
if (
not isinstance(transition, str)
or not transition
or not isinstance(target, str)
or target not in self.nodes
):
raise ValueError(f"invalid transition declared by node {node_name}")
@dataclass(frozen=True, slots=True)
class WorkflowRunResult:
run_id: str
context: WorkflowContext
status: str
class WorkflowTransitionError(ValueError):
pass
+45
View File
@@ -0,0 +1,45 @@
from typing import Protocol
from plba.workflow.models import StepResult, WorkflowContext
class WorkflowPersistence(Protocol):
async def record_run_started(
self, run_id: str, workflow_name: str, context: WorkflowContext
) -> None: ...
async def record_step_started(
self, run_id: str, step_name: str, context: WorkflowContext
) -> None: ...
async def record_step_completed(
self,
run_id: str,
step_name: str,
context: WorkflowContext,
result: StepResult,
) -> None: ...
async def record_step_failed(
self,
run_id: str,
step_name: str,
context: WorkflowContext,
error: BaseException,
) -> None: ...
async def record_run_completed(
self,
run_id: str,
workflow_name: str,
context: WorkflowContext,
status: str,
) -> None: ...
async def record_run_failed(
self,
run_id: str,
workflow_name: str,
context: WorkflowContext,
error: BaseException,
) -> None: ...

Some files were not shown because too many files have changed in this diff Show More