Reduce .NET ETL Code Size by 23 Times with actionETL

In a fully documented example, actionETL required only 9kB of code to create a high performance and reusable custom Slowly Changing Dimension (SCD) worker, 23 times less than the 209kB used by a SQL Server® Integration Services (SSIS) implementation with similar functionality. What caused this stark difference?

Modern AppDev Techniques

The actionETL library is designed from the ground up to make ETL development very productive. By using well-known application development techniques, it provides excellent:

  • Reusability
  • Composability
  • Encapsulation
  • Testability
  • Extensibility
  • Refactoring
  • Source control and Continuous Integration/Continuous Delivery

In the SCD example, actionETL composability pays a huge dividend, where existing ‘control flow’ and dataflow workers are easily combined to create a new high performance and reusable custom worker:

Unlike with SSIS, actionETL ‘control flow’ and dataflow workers can be freely mixed and matched, and can also be used to create new custom workers.

Visual Designer

In contrast, SSIS cannot use existing control flow tasks or dataflow components when creating new tasks or components, not even via C++, and must therefore implement all required functionality from scratch. Most SSIS custom tasks and components also require significant UI code. Both aspects heavily inflate the amount of code that must be written and maintained.

Virtually all traditional ETL tools have the same heavy focus on their drag&drop visual designer as SSIS has. While this certainly helps in some ways, like initially getting up to speed on the tool, they pay a very heavy price in terms of poor support for some or all of the above modern AppDev traits.


If you want to learn more about how actionETL compares to SSIS and traditional ETL tools, check out the Thirteen Factors Crippling ETL Productivity whitepaper.

Previously on Kristian Wedberg: Data Warehouse and BI articles

Just to have things nicely gathered in a single place, I’ll kick my new blog off with a few links to my earlier blog & articles, where the content is still relevant:

SQL Server Central articles

Conchango / EMC Consulting blog

Update: With EMC subsumed by Dell, these articles are no more:

  • Database Snapshots and SSIS
  • SSIS lookup transform gotcha (verified on SQL Server 2005)
  • SSIS and “Cursor Operation Conflict” (verified on SQL Server 2005)

To be continued…