IoC Container Benchmark - Unity, Windsor, StructureMap and Spring.NET

There are a number of inversion of control containers out there so I thought it would be an interesting experiment to do a simple benchmark. There are different ways that one can instantiate a type in .NET, for example via the new operator, Activator, GetUninitializedObject and Dynamic Method. The performance difference between these methods are in some cases quite high, maybe the same is true for these IoC containers? Granted IoC containers do more than just create objects so other factors will probably play a big role in the results.

So here are the contestants:

• Castle Windsor
  • Part of the popular Castle Project.
  • First introduced in 2004
• StructureMap
  • Sourceforge project maintained and created by Jeremy D. Miller
• Spring.NET
  • Inspired by the spring java framework
  • First introduced in 2004, www.springframework.net
• Unity
  • Created by the Microsoft Patterns Practices team
  • First released in April 2008
  • Project homepage on codeplex

I have been using Castle Windsor since 2005 and I think it is the best of the bunch, so I guess I am unconsciously biased toward Windsor.  However I will try to make this benchmark as objective as I can.

The scenario for this test:

• Have each IoC container resolve a UserController 1000 000 times
• The UserController will have two constructor dependencies
• Run the test with transient (new instance for each resolve) and singleton components

The UserController looks like this:

public class UserController
{
    private IUserRepository repository;
    private IAuthentificationService authService;

    public UserController(IUserRepository repository, IAuthentificationService authService)
    {
        this.repository = repository;
        this.authService = authService;
    }
}

I have also a general container interface that the benchmark engine will use. Each container will implement this interface.

public interface IContainer
{
    string Name { get; }

    T Resolve<T>();
    
    void SetupForTransientTest();
    void SetupForSingletonTest();
}

All tests used the latest released version of each library. Before you interpret these charts please observe that the measurement is for one million component resolves which means the actual time difference between each container is actually very small.

Here are the results when all components were setup as singletons:

Here are the results when all components were setup as transient:

So what does these charts tell us? Lets take the biggest difference in the transient case, Spring.NET took 44.149 seconds and Unity took 8.164 seconds, what is the actual difference when resolving a single instance?

  Spring.NET : 44.149 / 1000000 = 0.000044149 seconds
  Unity      :  8.164 / 1000000 = 0.000008164 seconds

So the actual difference is only about 36 microseconds. Another way to put these values into perspective is to compare against the new operator. I created a NewOperatorContainer with a resolve method that looks like this:

public T Resolve<T>()
{
    object o = new UserController(new LdapUserRepository(), new DefaultAuthentificationService());
    return (T) o;
}

OK, comparing the above with an inversion of control container is like comparing apples to oranges, an IoC handles so much more than just object creation. Also an IoC cannot use the new operator directly but must use one of the other methods. My guess is that all IoC containers in this test uses an approach which involve IL Generation which if cashed comes close to using the new operator directly. Anyway I think it will show just how small the difference between the real IoC containers are. In order to visualize this I needed to invert the values so that high means fast and low means slow.

Update: The above chart can be very misleading. The x-axis is not seconds but 1/s. I hope it shows that the difference between the containers are very small compared to instantiating the objects manually.

OK, can we draw any conclusion from the test? Well I think we can say that performance should not be an issue when choosing one of these IoC containers. The difference is too small. When you choose which container to use you should consider other aspects, like how invasive the container is to they way you want to work.

For the complete code: IoCBenchmark.zip