Performance comparison: IIS 7.5 and IIS 8 vs. self-hosted mvc4 web api

Introduction

There are several ways to host your services with the new MVC4 Web Api framework. Traditionally, most people would use IIS which has several advantages such as application pools, recycling, monitoring and on-demand loading.

This post on stackoverflow has a more of in-depth discussion about the pros and cons of self-hosting so I won't re-iterate that here - this post assumes you only care about the raw performance comparison which I haven't been able to find so far. It will compare IIS7 and IIS8 (Windows 8 Consumer Preview) to see if there are any differences between different versions of IIS.

Test Environment

This test was run on an Intel i5 with 8GB ram using Apache Bench 2.3 - Rev. 655654 (from the XAMPP for Windows installation).

In these tests, the web-server and benchmark client were located on the same physical machine.

Windows 7 Ultimate Edition with IIS7 will be used and Windows 8 Consumer Preview with IIS 8 will be used for the Web server performance comparisons. Do not try this with the version of IIS Express that ships with Visual Studio - the performance was a fraction of "real" IIS 7.5 in my tests.

The Test Code

The nightly build of MVC4 from nuget (as of 11th June 2012) was used. For Self-Host, the host was a console application (I would expect a Windows Service to yield comparable results).

In both cases, the controller itself was located in the same, external assembly and contained the following code:

public class TestApiController : ApiController
{
    public IList GetAll()
    {
        return new List<Foo>
                   {
                       new Foo {Id = 1, Name = "Foo 1", CreatedOn = DateTime.Now},
                       new Foo {Id = 2, Name = "Foo 2", CreatedOn = DateTime.Now},
                       new Foo {Id = 3, Name = "Foo 3", CreatedOn = DateTime.Now}
                   };
    }
}

The console application was configured as follows to try and get as close to the same functionality as the Web equivalent:

static void Main(string[] args)
{
    const string serviceAddress = "http://localhost:4444";

    var config = new HttpSelfHostConfiguration(serviceAddress);
    config.Routes.MapHttpRoute("default-api",
                               "api/{controller}/{id}",
                               new
                                   {
                                       id = RouteParameter.Optional,
                                       namespaces = new[] { typeof(TestApiController).Namespace }
                                   });

    config.Formatters.XmlFormatter.SupportedMediaTypes.Clear();
    var server = new HttpSelfHostServer(config);
    server.OpenAsync();

    Console.WriteLine("Waiting on " + serviceAddress);

    Console.ReadLine();
}

The web application had the following code in the global.asax.cs file:

protected void Application_Start()
{
    var config = GlobalConfiguration.Configuration;
    config.Filters.Clear();
    ViewEngines.Engines.Clear();
    config.Routes.MapHttpRoute("default-api",
        "api/{controller}/{id}",
        new { id = RouteParameter.Optional, namespaces = new[] { typeof(TestApiController).Namespace } });
    config.Formatters.XmlFormatter.SupportedMediaTypes.Clear();
}

In the web.config, debug was set to "false", authentication mode was set to "none", runAllManagedModulesForAllRequests was set to "false" and sessions and profiles were disabled.

Although these options are not the defaults, I assumed they would give the best raw performance (although in practice they seemed to make little or no difference).

The Results

Note: all tests were run 3 times and the "best" time was taken.

All tests were run with:

ab -n 100000 -c 100 http://myip:port/api/testapi

	Requests (#/sec)	Time per request (ms)
Windows 7 (IIS 7.5)	3025.73	33.050
Windows 7 (self-host)	4624.44	21.624
Windows 8 (IIS 8)	4778.23	20.928
Windows 8 (self-host)	5612.23	17.818

As can be seen above, in my environment, self-hosting can serve approximately 50% more requests per second under Windows 7 and about 17.5% on Windows 8.

Some of the overall performance difference between Windows 7 and Windows 8 is likely to be as a result of my Windows 7 installation having more services running (as the Windows 8 installation is brand new).

Conclusion

It is hardly surprising that self-hosting is more performant in terms of raw requests than IIS considering the rich feature-set of IIS. In addition, there might be settings in IIS that can bring it closer (or surpass) that of self-hosting by disabling various features. I would be happy to re-run the tests if anyone has any ideas.

As a basic recommendation, I would suggest that if you are creating a web api project, IIS is the safer and simpler choice by default.

Deployment using something like webdeploy means that the service can be updated without interruption. In the case of self-hosting with a Windows Service, there might be some juggling required to make sure that the stop -> deploy -> restart cycle minimizes down-time. A thin wrapper with auto-reloading external assemblies could go some way to resolve this but would require you to roll your own piece of infrastructure - something that is already solved in IIS.

Use-cases I can see for self-hosting include:

• You are already shipping a Windows Service for a different purpose and would like to expose some kind of managment interface
• You want very fine-grained control over the hosting stack and don't need any of the features of IIS
• Squeezing out the maximum requests/second is critical to your application