pypz.core.specs.pipeline module
- class pypz.core.specs.pipeline.Pipeline(name: str = None, *args, **kwargs)
Bases:
Instance[Operator],RegisteredInterfaceThis class represents the pipeline instance specs. A pipeline is actually a virtual organization of operators. It has a meaning only on pipeline level actions like deployment and execution. A pipeline spec can contain operators as nested instance.
- Parameters:
name – name of the instance, if not provided, it will be attempted to deduce from the variable’s name
- __context: Instance
Reference to the context aka parent object. Derived automatically in the override implementation of __setattr__.
- __depends_on: set
Set of other instances that is this instance depending on. Note however that the type of the dependencies are checked dynamically in runtime, since dependencies can only be defined on the same type of instance.
- __expected_parameters: dict[str, ExpectedParameter]
Map of expected parameters defined as descriptor of the class. Key is the name of the parameter, value is the parameter descriptor. Used to check, if an expected (described) parameter value shall be set upon instance parameter setting.
- __full_name: str
Full name of the instance which uniquely identifies it up to its topmost context. For example, if an instance A has a parent context B, which has a parent context C, then the full name of A is ‘C.B.A’. This value is calculated and the result is stored after the first calculation to avoid recalculation every time.
- __nested_instance_type: Type[NestedInstanceType] | None
Stores the specified type of the nested instances. This is required to be able to discover those instances
- __nested_instances: dict[str, NestedInstanceType]
This dictionary holds all the instances that are nested in the context of this instance object
- __parameters: InstanceParameters
The interpreted instance parameters i.e., cascading and templates are interpreted
- __simple_name: str
Name of the instance, which is represented by the object created from the implementation class. If not provided and there is a parent context, then the parent context will use the name of the variable.
- __spec_classes: set
Set of specs classes that are in the class hierarchy i.e., which specs classes are contributing to the implementation of this class
- __spec_name: str
The name of the spec constructed of module and qualified class name. Notice that we separate the module name and the class name so that we can identify them at loading by name.
- _abc_impl = <_abc._abc_data object>
- _on_error(source: Any, exception: Exception) None
This method can be implemented to react to error events during execution. The error itself may come from arbitrary sources.
- _on_interrupt(system_signal: int = None) None
This method can be implemented to react to interrupt signals like SIGINT, SIGTERM etc. The specs implementation can then execute interrupt logic e.g., early termination of loops.
- Parameters:
system_signal – id of the system signal that causes interrupt
- static create_from_dto(instance_dto: PipelineInstanceDTO, *args, **kwargs) Pipeline
Creates an instance object from the DTO representation. It is capable to retrieve and load specified classes and to update created instances according to the DTO.
- Parameters:
instance_dto – instance DTO
- Returns:
instance object specified by the DTO
- static create_from_string(source, *args, **kwargs) Pipeline
Helper method to provide the functionality to create an instance from a json model specified either as string or as dict.
- Parameters:
source – model as string
- Returns:
instance object specified by the DTO
- get_dto() PipelineInstanceDTO
Converts the instance information into the corresponding Data Transfer Object (DTO)
- Returns:
DTO from instance