Concurrent Workflows to Reduce Warehouse ETL Load Time
In large data integration projects, it is quite common to source data from multiple systems, sources, regions etc... As the number of data sources increases, the ETL load time also increases, because of the increasing data volume. One way to reduce the load time is by running different ETL process in parallel. Informatica PowerCenter's capability to run workflow concurrently can be used in such scenarios to reduce the ETL load time.
With that concurrent workflow configuration is done. Now to trigger the workflow, you can start the workflow using "Start Workflow Advanced" option as shown below.
Choose the workflow instance name from the pop up window and click OK to run the selected workflow instance.
Form the workflow monitor you can see the running instance of the workflow. As shown in below image you can see the workflow run instance, which is running concurrently.
Hope you enjoy this tutorial, Please let us know your comments and feedback.
What is Concurrent Workflows
Performance Improvement Features
A concurrent workflow is a workflow that can run as multiple instances concurrently. A workflow instance is a representation of a workflow. We can configure two types of concurrent workflows.
1. Allow concurrent workflows with the same instance name. Configure one workflow instance to run multiple times concurrently. Each instance has the same source, target, and variables parameters. The Integration Service identifies each instance by the run ID.
2. Configure unique workflow instances to run concurrently. Define each workflow instance name and configure a workflow parameter file for the instance. You can define different sources, targets, and variables in the parameter file.
Concurrent Workflows Configuration
For the demonstration, lets consider a scenario where we need to load daily transaction data from North America, Europe region. These two files are expected to be available around the same time.
Here we will create one workflow to load the sales transaction data and the same work will be used to load both the file, which can execute concurrently.
Once the workflow is created, enable concurrent execution as shown in below image.
Now Click on “Configure Concurrent Execution” and given the properties as in below image. Provide two different parameter files, which contains the source file information of corresponding region.
With that concurrent workflow configuration is done. Now to trigger the workflow, you can start the workflow using "Start Workflow Advanced" option as shown below.
Choose the workflow instance name from the pop up window and click OK to run the selected workflow instance.Form the workflow monitor you can see the running instance of the workflow. As shown in below image you can see the workflow run instance, which is running concurrently.
Hope you enjoy this tutorial, Please let us know your comments and feedback.