SteGriff

Blog

Next & Previous

What no-one tells you about getting started with BenchmarkDotNet

BenchmarkDotNet is a package for benchmarking (timing) .Net routines. It does correctly what you would do wrong if you were trying to time code yourself using System.Diagnostics.Stopwatch and gaffer tape. Like, it does timed rounds of volume testing, warmups, and JIT gymnastics all to make sure that the timings you get back are accurate. Pretty cool.

The samples are wilfully naive; they make it sound like all you do is throw in this [Benchmark] annotation but they don’t discuss what kind of project you’re supposed to use, what setup and config you need… so let me be honest with you:

It only works on optimized code (Release mode)

Set your Configuration to ‘Release’, or any other configuration which you have created which has ‘Optimize code’ turned on.

Without this, Benchmark will refuse to start.

It works best without the debugger attached

Ctrl+F5 starts your project without the debugger

You need a Program entry point which runs Benchmark

I had a Console project, for example, which kicks off like this:

using BenchmarkDotNet.Running;

...

static void Main(string[] args)
{
	var summary = BenchmarkRunner.Run<DynamicBenchmarks>();
	Console.WriteLine("End");
	Console.ReadLine();
}

…and that was my startup project, because all I was doing in it was benchmarking (there was no working program in the solution).

Instead, you could have multiple startup projects or just choose to run your Benchmark project ad-hoc using right-click, Debug, ‘Start new instance’.

You don’t have to do anything in your program with the returned result from .Run<>()

The output goes to the bin folder in three formats

The results files go to: {YourProject}\bin\Release\BenchmarkDotNet.Artifacts\results, and you will get, for each benchmarking project:

Don’t keep the report web page open in browser when running another attempt, or Benchmark will fail to write it and will throw a deathwailing exception.

Now you’re ready

Happy benchmarking! Hmu on twitter @SteGriff if you have more questions.