A public release is now available that adds .NET Standard 2.0 and Linux, with overall support for:
Windows
Linux
.NET Framework 4.6.1+
.NET Standard 2.0+
.NET Core 2.1+
It also adds nuget.org distribution and dotnet new project templates, making it very easy to get up and running – it only takes these three commands to install templates, create and then run your first actionETL application – try it now!
dotnet new --install actionETL.templates.dotnet
dotnet new actionetl.console --output my-app
dotnet run --project my-app
The actionetl.console template created this trivial but useful starting point, where you can replace and add your own ETL logic:
using actionETL;
using System.Threading.Tasks;
namespace actionETL.console.csharp
{
static class Program
{
static async Task Main()
{
// Test if file exists
var outcomeStatus = await new WorkerSystem()
.Root(ws =>
{
// Check filename loaded from "actionetl.aconfig.json"
new FileExistsWorker(ws, "File exists", ws.Config["TriggerFile"]);
})
.StartAsync();
// Exit with success or failure code
outcomeStatus.Exit();
}
}
}
I needed to ensure good CPU and memory performance in a VirtualBox virtual machine running on a 4-core desktop, and googling didn’t provide any clear guidance. After some benchmarking, the surprise came in the shape of consistently getting the best result when ignoring VirtualBox’ warning on oversubscribing processors:
VirtualBox warns if the guest is configured with more processors than there are physical cores on the host
As always, these results are only valid for my particular configuration and on my chosen benchmarks, including the assumptions that the physical host is idling while the single virtual machine is running flat out – do test with your own systems and tasks.
Physical Host
Desktop system where the CPU and motherboard were released in AD 2013:
CPU
Single Intel Core i7 4770K, 3.5GHz, 4 physical cores, 8 logical cores (when Hyper-Threading is enabled)
Note: All cores run at 3.7GHz during multi-threaded benchmarks; for single-threaded benchmarks, the core runs between 3.7 and 3.9GHz
Memory
G Skill F3-2400C10D-16GTX, Trident X Series, 2x8GB, PC3-19200, DDR3 2400MHz
Note: The results in this article are likely notapplicable to NUMA systems with physical processors in multiple sockets, since these have very different memory, cache, and thread scheduling characteristics.
For my particular requirements, I chose mainly multi-threaded CPU and memory bound benchmarks, with some disk benchmarks to guard against IO regressions – do follow the links for specifics on the individual benchmarks:
“Preferences > Number of processes” set to number of logical processors on host (i.e. 4 or 8)
All CPU and memory benchmarks have been included
Disk benchmarks were also executed, but detailed results are not included due to the variability in IO systems:
Enabling/disabling Hyper-Threading did as expected nothave any impact on disk performance
With Samsung RAPID mode disabled, disk performance in the VirtualBox guest ranged from 12% slower to 4% faster than the physical host
Enabling RAPID mode (which uses main memory as cache for SSD) improved VirtualBox guest disk performance with about 40%, and improved physical host disk performance with a whopping 9.5x – real life mileage will of course vary wildly
Results
To aid digestion, I’m presenting the data as speed-up or slowdown of different configurations vs. the on averagefastest configuration, which was to run on the physical host with Hyper-Threading enabled.
The “Overall Average” section at the top of the chart is the average slowdown of all the actual benchmarks further down. Comparing to physical host with Hyper-Threading enabled, we see that running on:
VB 4 NoHT (VirtualBox with 4 processors, host has Hyper-Threading disabled) is on average the slowest at -22%, with individual benchmarks ranging from -2% to -55% slower
VB 4 HT (VirtualBox with 4 processors, host has Hyper-Threading enabled) is on average -22% slower, with individual benchmarks ranging from 2% faster to -44% slower
Phys 4 NoHT (Physical host, Hyper-Threading disabled) is on average only -10% slower, with individual benchmarks ranging from 1% faster to -50% slower
VB 8 HT (VirtualBox with 8 processors, host has Hyper-Threading enabled) is on average only -9% slower, with individual benchmarks ranging from -2% to -27%
Conclusions
Given my set-up, requirements and assumptions, I find that:
Disabling Hyper-Threading makes both the physical host and the virtual machine on average quite a bit slower – I’ll leave it enabled.
Following VirtualBox’ recommendation of limiting virtual processors to number of physical cores brings a slowdown of -22% (VB 4 HT above). I’ll instead configure as many virtual VirtualBox processors as there are logical (Hyper-Threaded) cores (VB 8 HT above), giving only a -9% slowdown.
Finally, a small warning: if you configure VirtualBox to use more processors than there are logical (Hyper-Threaded) cores (e.g 16 virtual processors on my 4770K) , it can run an order of magnitude slower than normal – simply ensure that you have no more VirtualBox processors configured than there are logical (Hyper-Threaded) cores available.
