Load Testing SharePoint (Lessons Learned) (Part 1)

I want to share some experiences from a recent load testing exercise I performed with SharePoint.

I am working on a web content management project that requires A LOT of customizations to SharePoint. The team needed to understand how these customizations would perform and scale. So I dusted off my limited knowledge of load testing with Visual Studio Team Test and went to work.

First I established my testing environment. I started by setting up my test agent. I was able to commandeer an old R&D server (HP 385, Dual Pentium III, 2 GB RAM, 1 GB NIC) for my cause. I wasn't sure, but suspected that this would be more than enough machine for my needs (oh yeah, its business time). Setup was a breeze, I just installed Visual Studio Team Test 2008 and my load agent was ready.

Next I needed to prepare my SharePoint environment. I already had a virtualized (is that a word) environment for SharePoint. Actually I had two to choose from, one dedicated to integration testing for the development team and another for user acceptance testing. I decided to go with the integration test environment because it was located inside of the corporate network (no messy firewalls).

Next I needed to create my web tests. This step required me to put on a thinking cap. There are many different approaches for load testing and I am not going to discuss them in this article. I chose to create tests that would isolate specific pieces of functionality (rather than trying to mimic usage patterns).  By using this approach I could clearly understand how our customizations would compare to out of the box SharePoint.

Here is a list of my tests

ID	Test	Description
1	HTML page with Hello World text	This is not a SharePoint item.  I created it so I could have an overall for the test environment. The results from this test represent the fastest the server can go.  This will act as a high water mark for me to compare my results to.
2	SharePoint Publishing Article Page	This is an out of the box SharePoint item. The results from this test represent a baseline for me to compare my test results against.  My tests results should be in the same neighborhood as these test results.
3	Our Customized SharePoint Page Layout with one customized control.	This represents a very lightweight version of our customizations.
4	Our Customized SharePoint Page Layout with two customized controls.	The idea is to add a small piece of functionality so I can understand how the change impacts the test results.

I ran each one of those tests independently.  The idea was to take a single test and stress the system.

I used goal based load tests in VSTT (a new feature for VS.Net 2008).  My goal was to stress the Web Server so the CPU would run between 70 - 90%.  Goal based testing is why I decided to avoid the firewalls (Goal based testing requires that the load agent communicate with the performance counters on the target machines). 

VSTT will continue to load transactions until the goal is reached.  If the goal is never reached then VSTT will load to the maximum number of requests/users specified in the load test.

Now I had everything ready to go... or so I thought.  In Part 2 I will go into detail about tweaks I had to make to my SharePoint environment so I could get the load tests to work correctly.