Dylan's BI Study Notes

My notes about Business Intelligence, Data Warehousing, OLAP, and Master Data Management

Incremental ETL

What is Incremental ETL?

Incremental ETL is a typical approach in data warehousing to extract and load the data from the source to the target system.

It refers to the daily or regular ETL process.  These processes are scheduled in certain intervals.  The key is that everytime when the process is being executed, only the delta, the changes since the last process, is being processed.

Why do we need to use Incremental ETL?

Incremental ETL means that smaller volume of the data are being processed each time.

The load to both the source system and the target system can be reduced.

Criteria for evaluating a data warehousing architecture

Incremental ETL should be supported.

If it does not support incremental ETL, you should question if the time span for each ETL process is acceptable.

Also, you should watch out the resources used in extracting data from the source system.

 

 

 

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s