alex/plba
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Правки описания

Browse Source
This commit is contained in:
zosimovaa
2026-03-04 10:38:21 +03:00
parent de787ce7ee
commit c5a78d80d4
6 changed files with 890 additions and 446 deletions
Download Patch File Download Diff File Expand all files Collapse all files

111
requirements/Architectural constraints.md Normal file
View File

@@ -0,0 +1,111 @@
**`docs/adr/0001-new-runtime.md`**
```md
# ADR 0001: Create a new runtime project instead of evolving the legacy ConfigManager model
## Status
Accepted
## Context
The previous generation of the application runtime was centered around a timer-driven execution model:
- a manager loaded configuration
- a worker loop periodically called `execute()`
- applications placed most operational logic behind that entry point
That model is adequate for simple periodic jobs, but it does not match the direction of the new platform.
The new platform must support:
- task sources
- queue-driven processing
- multiple parallel workers
- trace propagation across producer/consumer boundaries
- richer lifecycle and status management
- health aggregation
- future admin web interface
- future authentication and user management
These are platform concerns, not business concerns.
We also want business applications to describe only business functionality and rely on the runtime for infrastructure behavior.
## Decision
We will create a new runtime project instead of implementing a `V3` directly inside the current legacy ConfigManager codebase.
The new runtime will be built around a platform-oriented model with explicit concepts such as:
- `RuntimeManager`
- `ApplicationModule`
- `TaskSource`
- `TaskQueue`
- `WorkerSupervisor`
- `TaskHandler`
- `TraceService`
- `HealthRegistry`
The old execute-centered model is treated as a previous-generation design and is not the architectural basis of the new runtime.
## Rationale
Creating a new runtime project gives us:
- freedom to design the correct abstractions from the start
- no pressure to preserve legacy contracts
- cleaner boundaries between platform and business logic
- simpler documentation and tests
- lower long-term complexity than mixing old and new models in one codebase
If we built this as `V3` inside the old project, we would likely inherit:
- compatibility constraints
- mixed abstractions
- transitional adapters
- conceptual confusion between periodic execution and event/queue processing
The expected long-term cost of such coupling is higher than creating a clean new runtime.
## Consequences
### Positive
- the platform can be modeled cleanly
- business applications can integrate through explicit contracts
- new runtime capabilities can be added without legacy pressure
- mail_order_bot can become the first pilot application on the new runtime
### Negative
- some existing capabilities will need to be reintroduced in the new project
- there will be a temporary period with both legacy and new runtime lines
- migration requires explicit planning
## Initial migration target
The first application on the new runtime will be `mail_order_bot`.
Initial runtime design for that application:
- IMAP polling source
- in-memory queue
- parallel workers
- business handler for email processing
- message marked as read only after successful processing
Later:
- swap IMAP polling source for IMAP IDLE source
- keep the queue and worker model unchanged
## What we intentionally do not carry over
We do not keep the old architecture as the central organizing principle:
- no `execute()`-centric application model
- no timer-loop as the main abstraction
- no implicit mixing of lifecycle and business processing
## Follow-up
Next design work should define:
- core platform contracts
- package structure
- runtime lifecycle
- queue and worker interfaces
- config model split between platform and application
- pilot integration for `mail_order_bot`

116
requirements/Mail Order Bot Migration Plan.md Normal file
View File

@@ -0,0 +1,116 @@
# Mail Order Bot Migration Plan
## Purpose
This document describes how `mail_order_bot` will be adapted to the new runtime as the first pilot business application.
## Scope
This is not a full migration specification for all future applications.
It is a practical first use case to validate the runtime architecture.
## Current model
The current application flow is tightly coupled:
- the manager checks IMAP
- unread emails are fetched
- emails are processed synchronously
- messages are marked as read after processing
Polling and processing happen in one execution path.
## Target model
The new runtime-based flow should be:
1. a mail source detects new tasks
2. tasks are published to a queue
3. workers consume tasks in parallel
4. a domain handler processes each email
5. successful tasks lead to `mark_as_read`
6. failed tasks remain retriable
## Phase 1
### Source
- IMAP polling source
### Queue
- in-memory task queue
### Workers
- 2 to 4 parallel workers initially
### Handler
- domain email processing handler built around the current processing logic
### Delivery semantics
- email is marked as read only after successful processing
- unread state acts as the first safety mechanism against message loss
## Why in-memory queue is acceptable at first
For the first phase:
- infrastructure complexity stays low
- the runtime contracts can be tested quickly
- unread emails in IMAP provide a simple recovery path after crashes
This allows us to validate the runtime architecture before adopting an external broker.
## Phase 2
Replace:
- IMAP polling source
With:
- IMAP IDLE source
The queue, workers, and handler should remain unchanged.
This is an explicit architectural goal:
source replacement without redesigning the processing pipeline.
## Domain responsibilities that remain inside mail_order_bot
The runtime should not own:
- email parsing rules
- client resolution logic
- attachment processing rules
- order creation logic
- client-specific behavior
These remain in the business application.
## Platform responsibilities used by mail_order_bot
The new runtime should provide:
- lifecycle
- configuration
- queue abstraction
- worker orchestration
- tracing
- health checks
- status/control APIs
## Migration boundaries
### Move into runtime
- source orchestration
- worker supervision
- queue management
- trace provisioning
- health aggregation
### Keep in mail_order_bot
- email business handler
- mail domain services
- business pipeline
- business-specific config validation beyond platform-level schema
## Success criteria
The migration is successful when:
- mail polling is no longer tightly coupled to processing
- workers can process emails in parallel
- business logic is not moved into the runtime
- replacing polling with IDLE is localized to the source layer

469
requirements/README.md Normal file
View File

@@ -0,0 +1,469 @@
# PLBA
`PLBA` is a reusable platform runtime for business applications.
It solves platform concerns that should not live inside domain code:
- application lifecycle
- worker orchestration
- configuration loading from YAML
- tracing
- health aggregation
- runtime status reporting
- HTTP control endpoints
- logging configuration
Business applications depend on `plba` as a package and implement only their own business behavior.
## Architecture
Current PLBA architecture is built around one core idea:
- the runtime manages a set of application workers
A worker is any runtime-managed active component with a unified lifecycle:
- `start()`
- `stop(force=False)`
- `health()`
- `status()`
This means PLBA does not require separate platform categories like `source` and `consumer`.
If an application needs polling, queue processing, listening, scheduled work, or another active loop, it is implemented as a worker.
### Main runtime model
1. application creates `RuntimeManager`
2. runtime loads configuration
3. runtime applies logging configuration
4. application module registers workers and supporting services
5. runtime starts all workers
6. workers execute business-related loops or processing
7. runtime aggregates health and status
8. runtime stops workers gracefully or forcefully
## Core concepts
### `ApplicationModule`
File: [application.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/contracts/application.py)
Describes a business application to the runtime.
Responsibilities:
- provide module name
- register workers
- register queues if needed
- register handlers if needed
- register health contributors
- compose application-specific objects
`ApplicationModule` does not run the application itself.
It only declares how the application is assembled.
### `Worker`
File: [worker.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/contracts/worker.py)
The main runtime-managed contract.
Responsibilities:
- start its own execution
- stop gracefully or forcefully
- report health
- report runtime status
This is the main extension point for business applications.
### `TaskQueue`
File: [queue.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/contracts/queue.py)
Optional queue abstraction.
Use it when application workers need buffered or decoupled processing.
PLBA does not force every application to use a queue.
Queue is one supported pattern, not the foundation of the whole platform.
### `TaskHandler`
File: [tasks.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/contracts/tasks.py)
Optional unit of business processing for one task.
Useful when a worker follows queue-driven logic:
- worker takes a task
- handler executes business logic
### `TraceService`
File: [service.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/tracing/service.py)
Platform trace service.
Responsibilities:
- create trace contexts
- resume trace from task metadata
- write context records
- write trace messages
Business code should use it as a platform service and should not implement its own tracing infrastructure.
### `HealthRegistry`
File: [registry.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/health/registry.py)
Aggregates application health.
PLBA uses three health states:
- `ok` — all critical parts work
- `degraded` — application still works, but there is a problem
- `unhealthy` — application should not be considered operational
### Runtime status
File: [types.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/core/types.py)
Status is separate from health.
Current runtime states:
- `starting`
- `idle`
- `busy`
- `stopping`
- `stopped`
Status is used for operational lifecycle decisions such as graceful shutdown.
### `ControlPlaneService`
Files:
- [service.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/control/service.py)
- [http_channel.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/control/http_channel.py)
Provides control and observability endpoints.
Currently supported:
- health access
- runtime start action
- runtime stop action
- runtime status action
### `ConfigurationManager`
Files:
- [configuration.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/core/configuration.py)
- [file_loader.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/config/file_loader.py)
- [providers.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/config/providers.py)
Loads and merges configuration.
Current built-in source:
- YAML file provider
### `LogManager`
File: [manager.py](/Users/alex/Dev_projects_v2/apps/plba/src/app_runtime/logging/manager.py)
Applies logging configuration from config.
Current expectation:
- logging config lives in the `log` section of YAML
## Available platform services
PLBA currently provides these reusable services.
### 1. Runtime lifecycle
Service:
- `RuntimeManager`
What it gives:
- startup orchestration
- worker registration and startup
- graceful stop with timeout
- force stop
- status snapshot
Example use:
- start `mail_order_bot`
- stop it after active email processing is drained
### 2. Worker supervision
Service:
- `WorkerSupervisor`
What it gives:
- unified worker orchestration
- aggregated worker statuses
- aggregated worker health
- stop coordination
Example use:
- run one polling worker and three processing workers in the same application
### 3. Queue support
Services:
- `TaskQueue`
- `InMemoryTaskQueue`
- `QueueWorker`
What it gives:
- buffered processing
- decoupling between task production and task consumption
- worker concurrency for task handling
Example use:
- worker A polls IMAP and pushes tasks to queue
- worker B processes queued email tasks with concurrency `3`
### 4. Configuration
Services:
- `ConfigurationManager`
- `FileConfigProvider`
- `ConfigFileLoader`
What it gives:
- YAML config loading
- config merging
- access to platform and application config
Example use:
- load `platform` section for runtime
- load `mail_order_bot` section for app-specific config
### 5. Tracing
Services:
- `TraceService`
- `TraceTransport`
- `NoOpTraceTransport`
What it gives:
- trace context creation
- trace propagation through task metadata
- trace messages for processing steps
Example use:
- polling worker creates trace when it discovers a mail
- processing worker resumes trace and writes business steps
### 6. Health
Services:
- `HealthRegistry`
- `WorkerHealth`
What it gives:
- per-worker health
- aggregated application health
- critical vs non-critical component handling
Example use:
- email processing workers are critical
- optional diagnostic worker may be non-critical
### 7. Status
Services:
- `WorkerStatus`
- runtime aggregated state
What it gives:
- current activity visibility
- ability to stop application only after in-flight work is completed
Example use:
- stop application only after processing workers become `idle` or `stopped`
### 8. HTTP control
Services:
- `ControlPlaneService`
- `HttpControlChannel`
What it gives:
- HTTP health/status/actions
- operational integration point
Example use:
- inspect current health from orchestration
- request graceful stop remotely
## Public package API
Public namespace is `plba`.
Main imports for external applications:
```python
from plba import ApplicationModule, QueueWorker, RuntimeManager, create_runtime
from plba.contracts import Task, TaskHandler, TaskQueue, Worker, WorkerHealth, WorkerStatus
from plba.queue import InMemoryTaskQueue
from plba.tracing import TraceService
```
## Example application pattern
Minimal queue-based application:
```python
from plba import ApplicationModule, QueueWorker, Task, TaskHandler, create_runtime
from plba.queue import InMemoryTaskQueue
class ExampleHandler(TaskHandler):
def handle(self, task: Task) -> None:
print(task.payload)
class ExampleModule(ApplicationModule):
@property
def name(self) -> str:
return "example"
def register(self, registry) -> None:
queue = InMemoryTaskQueue()
traces = registry.services.get("traces")
queue.publish(Task(name="incoming", payload={"hello": "world"}))
registry.add_queue("incoming", queue)
registry.add_worker(QueueWorker("example-worker", queue, ExampleHandler(), traces))
runtime = create_runtime(
ExampleModule(),
config_path="config.yml",
enable_http_control=False,
)
runtime.start()
```
## Building business applications on PLBA
These are the current rules for building business applications correctly.
### 1. Keep platform and business concerns separate
PLBA owns:
- lifecycle
- worker management
- logging
- trace infrastructure
- health aggregation
- HTTP control
- config loading
Business application owns:
- business workflows
- domain services
- application-specific config schema
- business task payloads
- business error semantics
### 2. Build app behavior from workers
A business application should be described as a small set of workers.
Typical examples:
- polling worker
- processing worker
- reconciliation worker
Do not introduce new worker types at platform level unless there is clear need for custom runtime behavior.
### 3. Use queues only when they help
Queue is optional.
Use queue when:
- one worker discovers work
- another worker processes it
- buffering or decoupling helps
- concurrency is needed
Do not force queue into applications that do not need it.
### 4. Keep business logic out of worker lifecycle code
Worker should orchestrate execution.
Business rules should live in dedicated services and handlers.
Good:
- worker gets config
- worker calls domain service
- worker reports trace and status
Bad:
- worker contains all parsing, decision logic, integration rules, and persistence rules in one class
### 5. Use trace as a platform service
Business application should:
- create meaningful trace steps
- propagate trace through task metadata if queue is used
- record business-relevant processing milestones
Business application should not:
- implement its own trace store
- control trace transport directly unless explicitly needed
### 6. Read config through PLBA
Business application should not read YAML directly.
Recommended flow:
- PLBA loads config
- application reads only its own config section
- application converts it to typed app config object
- services receive typed config object
### 7. Distinguish health from status
Use `health` for:
- is application operational?
Use `status` for:
- what is application doing right now?
This is important for graceful stop:
- health may still be `ok`
- status may be `busy`
### 8. Design workers for graceful stop
Workers should support:
- stop accepting new work
- finish current in-flight work when possible
- report `busy`, `idle`, `stopping`, `stopped`
This allows runtime to stop application safely.
## Recommended repository model
PLBA is intended to live in its own repository as a reusable package.
Recommended setup:
- repository `plba`: platform package only
- repository `mail_order_bot`: business application depending on `plba`
- repository `service_b`: business application depending on `plba`
## Example: `mail_order_bot`
Simple first version of `mail_order_bot` on PLBA:
- `MailPollingWorker`, concurrency `1`
- `EmailProcessingWorker`, concurrency `3`
- shared `InMemoryTaskQueue`
- domain services for mail parsing and order processing
Flow:
1. polling worker checks IMAP
2. polling worker pushes email tasks into queue
3. processing workers consume tasks
4. processing workers execute domain logic
5. runtime aggregates health and status
This keeps `mail_order_bot` small, explicit, and aligned with current PLBA architecture.

248
requirements/architecture.md Normal file
View File

@@ -0,0 +1,248 @@
# Architecture
## Overview
The runtime is built as a platform layer for business applications.
It consists of four logical layers:
- platform core
- platform contracts
- infrastructure adapters
- business applications
## Layers
### Platform core
The core contains long-lived runtime services:
- `RuntimeManager`
- `ConfigurationManager`
- `WorkerSupervisor`
- `TraceService`
- `HealthRegistry`
- `ControlPlaneService`
- `ServiceContainer`
The core is responsible for orchestration, not domain behavior.
### Platform contracts
Contracts define how business applications integrate with the runtime.
Main contracts:
- `ApplicationModule`
- `TaskSource`
- `TaskQueue`
- `Worker`
- `TaskHandler`
- `ConfigProvider`
- `HealthContributor`
- `TraceFactory`
These contracts must remain domain-agnostic.
### Infrastructure adapters
Adapters implement concrete runtime capabilities:
- in-memory queue
- Redis queue
- file config loader
- database config loader
- polling source
- IMAP IDLE source
- HTTP control plane
- trace transport adapters
Adapters may change between applications and deployments.
### Business applications
Applications are built on top of the contracts and adapters.
Examples:
- `mail_order_bot`
- future event-driven business services
Applications contain:
- domain models
- domain handlers
- application-specific configuration schema
- source/handler composition
## Core runtime components
### RuntimeManager
The main platform facade.
Responsibilities:
- bootstrap runtime
- initialize services
- register application modules
- start and stop all runtime-managed components
- expose status
- coordinate graceful shutdown
### ConfigurationManager
Responsibilities:
- load configuration
- validate configuration
- publish config updates
- provide typed config access
- notify subscribers on reload
Configuration should be divided into:
- platform config
- application config
- environment/runtime overrides
### WorkerSupervisor
Responsibilities:
- register worker definitions
- start worker pools
- monitor worker health
- restart failed workers when appropriate
- manage parallelism and backpressure
- expose worker-level status
### TraceService
Responsibilities:
- create traces for operations
- propagate trace context across source -> queue -> worker -> handler boundaries
- provide trace factories to applications
- remain transport-agnostic
### HealthRegistry
Responsibilities:
- collect health from registered contributors
- aggregate health into liveness/readiness/status views
- expose structured runtime health
### ControlPlaneService
Responsibilities:
- control endpoints
- runtime state visibility
- administrative actions
- later authentication and user/session-aware access
## Main runtime model
The runtime should operate on this conceptual flow:
1. runtime starts
2. configuration is loaded
3. services are initialized
4. application modules register sources, queues, handlers, and workers
5. task sources start producing tasks
6. tasks are published into queues
7. workers consume tasks
8. handlers execute business logic
9. traces and health are updated throughout the flow
10. runtime stops gracefully on request
## Contracts
### ApplicationModule
Describes a business application to the runtime.
Responsibilities:
- register domain services
- register task sources
- register queues
- register worker pools
- register handlers
- declare config requirements
- optionally register health contributors
### TaskSource
Produces tasks into queues.
Examples:
- IMAP polling source
- IMAP IDLE source
- webhook source
- scheduled source
Responsibilities:
- start
- stop
- publish tasks
- expose source status
### TaskQueue
A queue abstraction.
Expected operations:
- `publish(task)`
- `consume()`
- `ack(task)`
- `nack(task, retry_delay=None)`
- `stats()`
The first implementation may be in-memory, but the interface should support future backends.
### Worker
Consumes tasks from a queue and passes them to a handler.
Responsibilities:
- obtain task from queue
- open or resume trace context
- call business handler
- ack or nack the task
- expose worker state
### TaskHandler
Executes business logic for one task.
The runtime should not know what the handler does.
It only knows that a task is processed.
## Mail Order Bot as first application
### Phase 1
- source: IMAP polling
- queue: in-memory queue
- workers: parallel email processing workers
- handler: domain email processing handler
- mark message as read only after successful processing
### Phase 2
- source changes from polling to IMAP IDLE
- queue and workers remain the same
This demonstrates one of the architectural goals:
the source can change without redesigning the rest of the processing pipeline.
## Suggested package structure
```text
src/
app_runtime/
core/
contracts/
config/
workers/
queue/
tracing/
health/
control/
container/
adapters/
mail_order_bot_app/
module/
sources/
handlers/
services/
domain/

119
requirements/vision.md Normal file
View File

@@ -0,0 +1,119 @@
# Vision
## Purpose
This project provides a reusable runtime platform for business applications.
The runtime exists to solve service and infrastructure concerns that are needed by many applications but do not belong to business logic:
- start and stop
- status and lifecycle
- configuration
- worker execution
- trace propagation
- health checks
- control and administration
- later authentication and user management
The runtime should allow a business application to focus only on domain behavior.
## Vision statement
We build a platform where business applications are assembled from small domain modules, while the runtime consistently provides lifecycle, workers, tracing, configuration, and control-plane capabilities.
## Problem
In the previous generation, the execution model was centered around a single `execute()` entry point called by a timer loop.
That model works for simple periodic jobs, but it becomes too narrow when we need:
- queue-driven processing
- multiple concurrent workers
- independent task sources
- richer health semantics
- trace propagation across producer and consumer boundaries
- reusable runtime patterns across different applications
- future admin and authentication capabilities
The old model couples orchestration and execution too tightly.
## Desired future state
The new runtime should provide:
- a reusable lifecycle and orchestration core
- a clean contract for business applications
- support for sources, queues, workers, and handlers
- explicit separation between platform and domain
- trace and health as first-class platform services
- the ability to evolve into an admin platform
## Design principles
### 1. Platform vs domain separation
The runtime owns platform concerns.
Applications own domain concerns.
The runtime must not contain business rules such as:
- email parsing policies
- order creation logic
- invoice decisions
- client-specific rules
### 2. Composition over inheritance
Applications should be composed by registering modules and services in the runtime, not by inheriting a timer-driven class and overriding one method.
### 3. Explicit task model
Applications should model processing as:
- task source
- task queue
- worker
- handler
This is more scalable than one monolithic execute loop.
### 4. Parallelism as a first-class concern
The runtime should supervise worker pools and concurrency safely instead of leaving this to ad hoc application code.
### 5. Observability by default
Trace, logs, metrics, and health should be available as platform services from the start.
### 6. Evolvability
The runtime should be able to support:
- different queue backends
- different task sources
- different control planes
- different admin capabilities
without forcing business applications to change their domain code.
## First target use case
The first business application is `mail_order_bot`.
Its business domain is:
- fetching incoming mail
- processing attachments
- executing order-related pipelines
Its platform/runtime needs are:
- lifecycle management
- polling or IMAP IDLE source
- queueing
- worker pools
- tracing
- health checks
- future admin API
This makes it a good pilot for the new runtime.
## Success criteria
We consider the runtime direction successful if:
- `mail_order_bot` business logic can run on top of it without leaking infrastructure details into domain code
- the runtime can manage concurrent workers
- the runtime can support a queue-based flow
- the runtime can expose status and health
- the runtime can later host admin/auth features without redesigning the core