pypz.core.specs.plugin module

class pypz.core.specs.plugin.ExtendedPlugin(name: str = None, *args, **kwargs)

Bases: Plugin, RegisteredInterface, ABC

This interface extends the normal plugin’s lifecycle. The methods defined span outside the execution context, hence it can be used, if you need to perform some action before and after execution.

Parameters:

name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

_abc_impl = <_abc._abc_data object>

_internal_access = {'post_execution', 'pre_execution'}

abstractmethod _post_execution() → None

This method is called after the executor and its state machine exited, but before the program exits. It can be used to perform finalization/shutdown logic that is outside the execution context.

abstractmethod _pre_execution() → None

This method will be called before the executor state machine starts. It can be used to perform initialization that is required before the execution.

class pypz.core.specs.plugin.InputPortPlugin(name: str = None, *args, **kwargs)

Bases: PortPlugin, RegisteredInterface, ABC

This plugin interface allows to implement data transfer input port for an operator. Operators can communicate via ports. Different technologies can be implemented allowing operators to talk through them.

Parameters:

• name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

• schema – the schema of the port plugin, which will be used to send/retrieve data

• group_mode – if set to True, the all the input ports in the group shall receive all messages

__group_mode: bool

If True, the InputPortPlugin shall receive all records sent to the group

_abc_impl = <_abc._abc_data object>

abstractmethod can_retrieve() → bool

This method shall implement the logic to signalize, whether the InputPort is still able to retrieve. Unable can mean for example that the OutputPort finished writing. This can be then used to terminate reading.

Returns:

True if port can retrieve, False if not

abstractmethod commit_current_read_offset() → None

This method shall implement the logic of committing the current read offset based on the technology used.

is_in_group_mode()

abstractmethod retrieve() → Any

This method shall implement the logic to retrieve data through the port.

Returns:

tbd by the implementation

class pypz.core.specs.plugin.LoggerPlugin(name: str = None, *args, **kwargs)

Bases: Plugin, ContextLoggerInterface, RegisteredInterface, ABC

This addon interface allows to implement different logging technologies to be used during the execution. Notice that the logger methods are coming from the ContextLogger class.

Parameters:

name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

__logger: ContextLogger | None

Context logger, which is the Operator’s logger if Operator context existing, otherwise it defaults back to the DefaultContextLogger. Note that Plugin without Operator context makes only sense in test cases, hence the default log level is set to DEBUG.

_abc_impl = <_abc._abc_data object>

class pypz.core.specs.plugin.OutputPortPlugin(name: str = None, *args, **kwargs)

Bases: PortPlugin, RegisteredInterface, ABC

This plugin interface allows to implement data transfer output port for an operator. Operators can communicate via ports. Different technologies can be implemented allowing operators to talk through them.

Parameters:

• name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

• schema – the schema of the port plugin, which will be used to send/retrieve data

_abc_impl = <_abc._abc_data object>

abstractmethod send(data: Any) → Any

This method shall implement the logic to send data provided as argument. The implementation shall specify the type of the data and the return value.

Parameters:

data – data to be sent

Returns:

tbd by the implementation

class pypz.core.specs.plugin.Plugin(name: str = None, *args, **kwargs)

Bases: Instance[None], InstanceGroup, RegisteredInterface, ABC

This interface has the only purpose to separate the plugin interfaces from other interfaces like the Operator. It is necessary to avoid the case, where an Operator could be nested into other Operators. All plugin interfaces shall extend this one.

Parameters:

name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

__logger: ContextLogger | None

Context logger, which is the Operator’s logger if Operator context existing, otherwise it defaults back to the DefaultContextLogger. Note that Plugin without Operator context makes only sense in test cases, hence the default log level is set to DEBUG.

_abc_impl = <_abc._abc_data object>

static create_from_dto(instance_dto: PluginInstanceDTO, *args, **kwargs) → Plugin

Creates an instance object from the DTO representation. It is capable to retrieve and load specified classes and to update created instances according to the DTO.

Parameters:

instance_dto – instance DTO

Returns:

instance object specified by the DTO

static create_from_string(source, *args, **kwargs) → Plugin

Helper method to provide the functionality to create an instance from a json model specified either as string or as dict.

Parameters:

source – model as string

Returns:

instance object specified by the DTO

get_context() → Operator

get_dto() → PluginInstanceDTO

Converts the instance information into the corresponding Data Transfer Object (DTO)

Returns:

DTO from instance

get_group_index() → int

get_group_name() → str | None

get_group_principal() → Instance | None

get_group_size() → int

get_logger() → ContextLogger

is_principal() → bool

class pypz.core.specs.plugin.PortPlugin(name: str = None, *args, **kwargs)

Bases: Plugin, RegisteredInterface, ABC

This plugin interface allows to implement common methods for port both input and output port plugins.

Parameters:

• name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

• schema – the schema of the port plugin, which will be used to send/retrieve data

__connected_ports: set[PortPlugin]

This member holds the information about the connected ports, where the key is the replication group names and the value is a list of connected ports.

__schema: Any

The port’s schema, which is used to identify the format of the data sent through the port.

_abc_impl = <_abc._abc_data object>

abstractmethod _on_port_close() → bool

This method shall implement the logic to shut down the i/o port functionalities.

Returns:

True succeeded, False if more iteration required (to not block the execution)

abstractmethod _on_port_open() → bool

This method shall implement the logic to initialize the i/o port functionalities.

Returns:

True succeeded, False if more iteration required (to not block the execution)

connect(other_port: PortPlugin) → None

get_connected_ports() → set[PortPlugin]

get_schema() → Any

set_schema(schema: Any) → None

class pypz.core.specs.plugin.ResourceHandlerPlugin(name: str = None, *args, **kwargs)

Bases: Plugin, RegisteredInterface, ABC

This plugin interface allows to implement resource management related functionalities. The respective methods will be called at specific times during the execution. Check Executor for more information.

Parameters:

name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

_abc_impl = <_abc._abc_data object>

abstractmethod _on_resource_creation() → bool

This method shall implement the logic to create an arbitrary resource of any type.

Returns:

True succeeded, False if more iteration required (to not block the execution)

abstractmethod _on_resource_deletion() → bool

This method shall implement the logic to destroy the created resource.

Returns:

True succeeded, False if more iteration required (to not block the execution)

class pypz.core.specs.plugin.ServicePlugin(name: str = None, *args, **kwargs)

Bases: Plugin, RegisteredInterface, ABC

This plugin interface allows to implement a service. Services are special entities in the execution, since those are decoupled from the execution life-cycle, hence can run in the background. Examples: - mounting service, which mounts and unmounts a remote location - listener service, which starts a background thread to listen for something - etc.

Parameters:

name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name

__logger: ContextLogger | None

Context logger, which is the Operator’s logger if Operator context existing, otherwise it defaults back to the DefaultContextLogger. Note that Plugin without Operator context makes only sense in test cases, hence the default log level is set to DEBUG.

_abc_impl = <_abc._abc_data object>

abstractmethod _on_service_shutdown() → bool

This method shall implement the logic of stopping the service and clean up the residuals. E.g., a mounting service could unmount or a background thread could be stopped. VERY IMPORTANT NOTE - you must always check, if your service start method has been called, because it can be that it is never called, if there was an exception raised from other entity. However, the shutdown will be called anyway.

Returns:

True if logic finished, False if it needs more iteration

abstractmethod _on_service_start() → bool

This method shall implement the starting logic of the service. You can consider services as example like mounting service, which only mounts a folder to the system or a background service, which starts a background thread.

Returns:

True if logic finished, False if it needs more iteration