pypz.core.specs.plugin module

class pypz.core.specs.plugin.ExtendedPlugin(name: str = None, *args, **kwargs)

Bases: Plugin, RegisteredInterface, ABC

This interface extends the normal plugin’s lifecycle. The methods defined span outside the execution context, hence it can be used, if you need to perform some action before and after execution.

Parameters:

name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

_abc_impl = <_abc._abc_data object>

abstract _post_execution() → None

This method is called after the executor and its state machine exited, but before the program exits. It can be used to perform finalization/shutdown logic that is outside the execution context.

abstract _pre_execution() → None

This method will be called before the executor state machine starts. It can be used to perform initialization that is required before the execution.

class pypz.core.specs.plugin.InputPortPlugin(name: str = None, *args, **kwargs)

Bases: PortPlugin, RegisteredInterface, ABC

This plugin interface allows to implement data transfer input port for an operator. Operators can communicate via ports. Different technologies can be implemented allowing operators to talk through them.

Parameters:

• name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

• schema – the schema of the port plugin, which will be used to send/retrieve data

• group_mode – if set to True, the all the input ports in the group shall receive all messages

_abc_impl = <_abc._abc_data object>

_group_mode: bool

If True, the InputPortPlugin shall receive all records sent to the group

abstract can_retrieve() → bool

This method shall implement the logic to signalize, whether the InputPort is still able to retrieve. Unable can mean for example that the OutputPort finished writing. This can be then used to terminate reading.

Returns:

True if port can retrieve, False if not

abstract commit_current_read_offset() → None

This method shall implement the logic of committing the current read offset based on the technology used.

is_in_group_mode()

abstract retrieve() → Any

This method shall implement the logic to retrieve data through the port.

Returns:

tbd by the implementation

class pypz.core.specs.plugin.LoggerPlugin(name: str = None, *args, **kwargs)

Bases: Plugin, ContextLoggerInterface, RegisteredInterface, ABC

This addon interface allows to implement different logging technologies to be used during the execution. Notice that the logger methods are coming from the ContextLogger class.

Parameters:

name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

_abc_impl = <_abc._abc_data object>

class pypz.core.specs.plugin.OutputPortPlugin(name: str = None, *args, **kwargs)

Bases: PortPlugin, RegisteredInterface, ABC

This plugin interface allows to implement data transfer output port for an operator. Operators can communicate via ports. Different technologies can be implemented allowing operators to talk through them.

Parameters:

• name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

• schema – the schema of the port plugin, which will be used to send/retrieve data

_abc_impl = <_abc._abc_data object>

abstract send(data: Any) → Any

This method shall implement the logic to send data provided as argument. The implementation shall specify the type of the data and the return value.

Parameters:

data – data to be sent

Returns:

tbd by the implementation

class pypz.core.specs.plugin.Plugin(name: str = None, *args, **kwargs)

Bases: Instance[None], InstanceGroup, RegisteredInterface, ABC

This interface has the only purpose to separate the plugin interfaces from other interfaces like the Operator. It is necessary to avoid the case, where an Operator could be nested into other Operators. All plugin interfaces shall extend this one.

Parameters:

name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

__logger: ContextLogger | None

Context logger, which is the Operator’s logger if Operator context existing, otherwise it defaults back to the DefaultContextLogger. Note that Plugin without Operator context makes only sense in test cases, hence the default log level is set to DEBUG.

_abc_impl = <_abc._abc_data object>

static create_from_dto(instance_dto: PluginInstanceDTO, *args, **kwargs) → Plugin

Creates an instance object from the DTO representation. It is capable to retrieve and load specified classes and to update created instances according to the DTO.

Parameters:

instance_dto – instance DTO

Returns:

instance object specified by the DTO

static create_from_string(source, *args, **kwargs) → Plugin

Helper method to provide the functionality to create an instance from a json model specified either as string or as dict.

Parameters:

source – model as string

Returns:

instance object specified by the DTO

get_context() → Operator

get_dto() → PluginInstanceDTO

Converts the instance information into the corresponding Data Transfer Object (DTO)

Returns:

DTO from instance

get_group_index() → int

get_group_name() → str | None

get_group_principal() → Instance | None

get_group_size() → int

get_logger() → ContextLogger

is_principal() → bool

class pypz.core.specs.plugin.PortPlugin(name: str = None, *args, **kwargs)

Bases: Plugin, RegisteredInterface, ABC

This plugin interface allows to implement common methods for port both input and output port plugins.

Parameters:

• name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

• schema – the schema of the port plugin, which will be used to send/retrieve data

__connected_ports: set[PortPlugin]

This member holds the information about the connected ports, where the key is the replication group names and the value is a list of connected ports.

__schema: Any

The port’s schema, which is used to identify the format of the data sent through the port.

_abc_impl = <_abc._abc_data object>

abstract _on_port_close() → bool

This method shall implement the logic to shut down the i/o port functionalities.

Returns:

True succeeded, False if more iteration required (to not block the execution)

abstract _on_port_open() → bool

This method shall implement the logic to initialize the i/o port functionalities.

Returns:

True succeeded, False if more iteration required (to not block the execution)

connect(other_port: PortPlugin) → None

get_connected_ports() → set[PortPlugin]

get_schema() → Any

set_schema(schema: Any) → None

class pypz.core.specs.plugin.ResourceHandlerPlugin(name: str = None, *args, **kwargs)

Bases: Plugin, RegisteredInterface, ABC

This plugin interface allows to implement resource management related functionalities. The respective methods will be called at specific times during the execution. Check Executor for more information.

Parameters:

name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

_abc_impl = <_abc._abc_data object>

abstract _on_resource_creation() → bool

This method shall implement the logic to create an arbitrary resource of any type.

Returns:

True succeeded, False if more iteration required (to not block the execution)

abstract _on_resource_deletion() → bool

This method shall implement the logic to destroy the created resource.

Returns:

True succeeded, False if more iteration required (to not block the execution)

class pypz.core.specs.plugin.ServicePlugin(name: str = None, *args, **kwargs)

Bases: Plugin, RegisteredInterface, ABC

This plugin interface allows to implement a service. Services are special entities in the execution, since those are decoupled from the execution life-cycle, hence can run in the background. Examples: - mounting service, which mounts and unmounts a remote location - listener service, which starts a background thread to listen for something - etc.

Parameters:

name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

__context: Instance

Reference to the context aka parent object. Derived automatically in the override implementation of __setattr__.

__depends_on: set

Set of other instances that is this instance depending on. Note however that the type of the dependencies are checked dynamically in runtime, since dependencies can only be defined on the same type of instance.

__expected_parameters: dict[str, ExpectedParameter]

Map of expected parameters defined as descriptor of the class. Key is the name of the parameter, value is the parameter descriptor. Used to check, if an expected (described) parameter value shall be set upon instance parameter setting.

__full_name: str

Full name of the instance which uniquely identifies it up to its topmost context. For example, if an instance A has a parent context B, which has a parent context C, then the full name of A is ‘C.B.A’. This value is calculated and the result is stored after the first calculation to avoid recalculation every time.

__logger: ContextLogger | None

Context logger, which is the Operator’s logger if Operator context existing, otherwise it defaults back to the DefaultContextLogger. Note that Plugin without Operator context makes only sense in test cases, hence the default log level is set to DEBUG.

__nested_instance_type: Type[NestedInstanceType] | None

Stores the specified type of the nested instances. This is required to be able to discover those instances

__nested_instances: dict[str, NestedInstanceType]

This dictionary holds all the instances that are nested in the context of this instance object

__parameters: InstanceParameters

The interpreted instance parameters i.e., cascading and templates are interpreted

__simple_name: str

Name of the instance, which is represented by the object created from the implementation class. If not provided and there is a parent context, then the parent context will use the name of the variable.

__spec_classes: set

Set of specs classes that are in the class hierarchy i.e., which specs classes are contributing to the implementation of this class

__spec_name: str

The name of the spec constructed of module and qualified class name. Notice that we separate the module name and the class name so that we can identify them at loading by name.

_abc_impl = <_abc._abc_data object>

abstract _on_service_shutdown() → bool

This method shall implement the logic of stopping the service and clean up the residuals. E.g., a mounting service could unmount or a background thread could be stopped. VERY IMPORTANT NOTE - you must always check, if your service start method has been called, because it can be that it is never called, if there was an exception raised from other entity. However, the shutdown will be called anyway.

Returns:

True if logic finished, False if it needs more iteration

abstract _on_service_start() → bool

This method shall implement the starting logic of the service. You can consider services as example like mounting service, which only mounts a folder to the system or a background service, which starts a background thread.

Returns:

True if logic finished, False if it needs more iteration