SQLite is the most used database engine in the world and is now supported by actionETL. With SQLite being a fast and installation free local database in the public domain, it can be a great tool to combine with actionETL.
You might use SQLite as:
Your main (local) database
A complement to a traditional database server, e.g. to offload work from an expensive host, and to avoid the network performance overhead
Temporary SQL processing, e.g. to:
Reduce memory consumption by performing a large sort using the disk backed SQLite
Execute queries where the SQL set-based approach is a better fit (and using LINQ is not appropriate) than the dataflow row-by-row approach
In a fully documented example, actionETL required only 9kB of code to create a high performance and reusable custom Slowly Changing Dimension (SCD) worker, 23 times less than the 209kB used by a SQL Server® Integration Services (SSIS) implementation with similar functionality. What caused this stark difference?
Modern AppDev Techniques
The actionETL library is designed from the ground up to make ETL development very productive. By using well-known application development techniques, it provides excellent:
Source control and Continuous Integration/Continuous Delivery
In the SCD example, actionETLcomposability pays a huge dividend, where existing ‘control flow’ and dataflow workers are easily combined to create a new high performance and reusable custom worker:
In contrast, SSIS cannot use existing control flow tasks or dataflow components when creating new tasks or components, not even via C++, and must therefore implement all required functionality from scratch. Most SSIS custom tasks and components also require significant UI code. Both aspects heavily inflate the amount of code that must be written and maintained.
Virtually all traditional ETL tools have the same heavy focus on their drag&drop visual designer as SSIS has. While this certainly helps in some ways, like initially getting up to speed on the tool, they pay a very heavy price in terms of poor support for some or all of the above modern AppDev traits.
actionETL is a .NET library and NuGet package for easily creating high-performance ETL applications.
It combines familiar ETL tool features such as ‘control flow’ and dataflow with modern application development techniques, which together provide a highly productive environment for tackling both small and large ETL projects.
The combination is easy to learn and powerful to use, targeting users ranging from ETL developers with limited .NET experience, to seasoned .NET developers with limited ETL experience.