pypz.deployers.base module
- class pypz.deployers.base.Deployer
Bases:
ABCThis is the base class for every deployer implementations. If you want to develop your own deployer with your own choice of technology, then you need to implement this interface.
- _abc_impl = <_abc._abc_data object>
- attach(pipeline_name: str, on_operator_state_change: Callable[[Operator, DeploymentState], None] = None) None
This method attaches itself to a deployed pipeline and remains attached until the pipeline is not finished. It is possible to specify callback functions to hook into certain state changes. If that state change happens, then the callback gets the Operator instance and the corresponding state provided.
- Parameters:
pipeline_name – name of the deployed pipeline entity
on_operator_state_change – callback to hook into state changes
- abstract deploy(pipeline: Pipeline, execution_mode: ExecutionMode = ExecutionMode.Standard, ignore_operators: list[Operator] = None, wait: bool = True) None
Shall implement the logic to deploy a pipeline by its instance.
- Parameters:
pipeline – pipeline instance to be deployed
execution_mode – execution mode of the operators (check ExecutionMode for details)
ignore_operators – list of operator instance to be excluded from the deployment
wait – True - block until completion; False - don’t block until completion
- abstract destroy(pipeline_name: str, force: bool = False, wait: bool = True) None
Shall implement the logic to destroy a pipeline by its name.
- Parameters:
pipeline_name – name of the deployed pipeline entity
force – True - kill without grace period; False - with grace period
wait – True - block until completion; False - don’t block until completion
- abstract destroy_operator(operator_full_name: str, force: bool = False, wait: bool = True) None
Shall implement the logic to destroy a single operator by its name.
- Parameters:
operator_full_name – full name of the deployed operator
force – True - kill without grace period; False - with grace period
wait – True - block until completion; False - don’t block until completion
- is_all_operator_in_state(pipeline_name: str, *state: DeploymentState)
This method checks, if all the operators in the deployed pipeline is in any of the specified states.
- Parameters:
pipeline_name – name of the deployed pipeline entity
state – list of states in OR condition
- is_any_operator_in_state(pipeline_name: str, *state: DeploymentState)
This method checks, if any of the operators in the deployed pipeline is in any of the specified states.
- Parameters:
pipeline_name – name of the deployed pipeline entity
state – list of states in OR condition
- abstract is_deployed(pipeline_name: str) bool
Shall implement the logic to check, if a pipeline with the specified name has been deployed.
- Parameters:
pipeline_name – name of the pipeline to check
- Returns:
True, if deployed, False if not
- abstract restart_operator(operator_full_name: str, force: bool = False, wait: bool = True) None
Shall implement the logic to restart a single operator in a pipeline by its name. If the operator does not exist, it shall rather create it without throwing an exception.
- Parameters:
operator_full_name – full name of the deployed operator
force – True - kill without grace period; False - with grace period
wait – True - block until completion; False - don’t block until completion
- abstract retrieve_deployed_pipeline(pipeline_name: str) Pipeline | None
Shall implement the logic to retrieve and create the deployed pipeline instance by its name.
- Parameters:
pipeline_name – name of the pipeline
- Returns:
Pipeline object, if existing, None if not existing
- abstract retrieve_operator_logs(operator_full_name: str) str | None
Shall implement the logic to retrieve the logs from a deployed operator by its name.
- Parameters:
operator_full_name – full name of the deployed operator
- Returns:
operator logs as string
- abstract retrieve_operator_state(operator_full_name: str) DeploymentState
Shall implement the logic to retrieve the state of a single operator by its name.
- Parameters:
operator_full_name – full name of the deployed operator
- Returns:
check DeploymentState for details
- abstract retrieve_pipeline_deployments() set[str]
Shall implement the logic to get the names of all deployed pipelines.
- Returns:
set of names of the deployed pipelines
- retrieve_pipeline_state(pipeline_name: str) dict[str, DeploymentState]
This method retrieves and collects all the operators’ states in the deployed pipeline.
- Parameters:
pipeline_name – name of the deployed pipeline entity
- Returns:
a dict, where key is the name of the operator and the value is the corresponding state object