pypz.plugins.kafka_io.ports module

class pypz.plugins.kafka_io.ports.KafkaChannelInputPort(name: str = None, *args, **kwargs)

Bases: ChannelInputPort

__connected_ports: set[PortPlugin]

This member holds the information about the connected ports, where the key is the replication group names and the value is a list of connected ports.

__context: Instance

Reference to the context aka parent object. Derived automatically in the override implementation of __setattr__.

__depends_on: set

Set of other instances that is this instance depending on. Note however that the type of the dependencies are checked dynamically in runtime, since dependencies can only be defined on the same type of instance.

__expected_parameters: dict[str, ExpectedParameter]

Map of expected parameters defined as descriptor of the class. Key is the name of the parameter, value is the parameter descriptor. Used to check, if an expected (described) parameter value shall be set upon instance parameter setting.

__full_name: str

Full name of the instance which uniquely identifies it up to its topmost context. For example, if an instance A has a parent context B, which has a parent context C, then the full name of A is ‘C.B.A’. This value is calculated and the result is stored after the first calculation to avoid recalculation every time.

__logger: ContextLogger | None

Context logger, which is the Operator’s logger if Operator context existing, otherwise it defaults back to the DefaultContextLogger. Note that Plugin without Operator context makes only sense in test cases, hence the default log level is set to DEBUG.

__nested_instance_type: Type[NestedInstanceType] | None

Stores the specified type of the nested instances. This is required to be able to discover those instances

__nested_instances: dict[str, NestedInstanceType]

This dictionary holds all the instances that are nested in the context of this instance object

__parameters: InstanceParameters

The interpreted instance parameters i.e., cascading and templates are interpreted

__schema: Any

The port’s schema, which is used to identify the format of the data sent through the port.

__simple_name: str

Name of the instance, which is represented by the object created from the implementation class. If not provided and there is a parent context, then the parent context will use the name of the variable.

__spec_classes: set

Set of specs classes that are in the class hierarchy i.e., which specs classes are contributing to the implementation of this class

__spec_name: str

The name of the spec constructed of module and qualified class name. Notice that we separate the module name and the class name so that we can identify them at loading by name.

_abc_impl = <_abc._abc_data object>

_channel_reader: ChannelReader | None

The only channel reader maintained by this plugin.

_delete_resources: bool

This flag signalizes, if resource deletion can be performed. It will be cleared, if there is an error on OperationRunning state to allow process restart.

_expected_output_count: int | None

During _on_port_open the expected number of connected outputs will be calculated. It then will be used in cases, where we need to wait for all outputs.

_group_mode: bool

If True, the InputPortPlugin shall receive all records sent to the group

_interrupted: bool

Helper flag to register, if the execution has been interrupted.

_need_to_check_connections_opened: bool

Helper flag to signalize, if the channel shall be opened. This is necessary to maintain state in _on_port_open().

channel_reader_type: Type[ChannelReader]

Type of the channel reader to be created by this plugin.

class pypz.plugins.kafka_io.ports.KafkaChannelOutputPort(name: str = None, *args, **kwargs)

Bases: ChannelOutputPort

__connected_ports: set[PortPlugin]

This member holds the information about the connected ports, where the key is the replication group names and the value is a list of connected ports.

__context: Instance

Reference to the context aka parent object. Derived automatically in the override implementation of __setattr__.

__depends_on: set

Set of other instances that is this instance depending on. Note however that the type of the dependencies are checked dynamically in runtime, since dependencies can only be defined on the same type of instance.

__expected_parameters: dict[str, ExpectedParameter]

Map of expected parameters defined as descriptor of the class. Key is the name of the parameter, value is the parameter descriptor. Used to check, if an expected (described) parameter value shall be set upon instance parameter setting.

__full_name: str

Full name of the instance which uniquely identifies it up to its topmost context. For example, if an instance A has a parent context B, which has a parent context C, then the full name of A is ‘C.B.A’. This value is calculated and the result is stored after the first calculation to avoid recalculation every time.

__logger: ContextLogger | None

Context logger, which is the Operator’s logger if Operator context existing, otherwise it defaults back to the DefaultContextLogger. Note that Plugin without Operator context makes only sense in test cases, hence the default log level is set to DEBUG.

__nested_instance_type: Type[NestedInstanceType] | None

Stores the specified type of the nested instances. This is required to be able to discover those instances

__nested_instances: dict[str, NestedInstanceType]

This dictionary holds all the instances that are nested in the context of this instance object

__parameters: InstanceParameters

The interpreted instance parameters i.e., cascading and templates are interpreted

__schema: Any

The port’s schema, which is used to identify the format of the data sent through the port.

__simple_name: str

Name of the instance, which is represented by the object created from the implementation class. If not provided and there is a parent context, then the parent context will use the name of the variable.

__spec_classes: set

Set of specs classes that are in the class hierarchy i.e., which specs classes are contributing to the implementation of this class

__spec_name: str

The name of the spec constructed of module and qualified class name. Notice that we separate the module name and the class name so that we can identify them at loading by name.

_abc_impl = <_abc._abc_data object>

_channel_writers: set[ChannelWriter] | None

The channel writers to be maintained by this plugin. Note that a channel writer will be created for each connection from this output port plugin.

_delete_resources: bool

This flag signalizes, if resource deletion can be performed. It will be cleared, if there is an error on OperationRunning state to allow process restart.

_interrupted: bool

Helper flag to register, if the execution has been interrupted.

_port_close_errors: set[ChannelWriter]

Set to store the channels that had errors on port closure. It is necessary to be able not to reschedule those channels in re-execution of the state.

_resource_deletion_errors: set[ChannelWriter]

Set to store the channels that had errors on resource deletion. It is necessary to be able not to reschedule those channels in re-execution of the state.

channel_writer_type: Type[ChannelWriter]

Type of the channel writer to be created by this plugin.