The holidays are not typically on our minds this early in the year given that it is currently early July and over 100 degrees Fahrenheit in portions of the Eastern USA. On the other hand, if you have an ecommerce site that has changed since the Holidays last year, then perhaps you should be thinking about gauging your traffic and applying a peak load to the site. Let’s not forget that even the biggest logos have had issues during peak traffic. Many large sites including HP’s outage in 2009, Wal-Mart’s outage in 2006, in 2008 Bloomingdale’s and J. Crew went down, and many others have made the news in recent years because their sites were note ready for the traffic. Oh, by the way, you have to do your testing early enough to take corrective action before the holidays!
Now that I’ve startled you out of your summer doldrums, let’s talk about how you can perform your tests most accurately. First, let’s talk about an all-too-common pitfall of not taking network latency into account when building the load test. Most teams will load test their site to ensure it will hold up to capacity, but all too often the team fails to take into account the implications that network latency will have on their load tests. As a result, testing teams will run load tests from a single location, often the same location that the Website is hosted or from within the lab itself.
If you just asked yourself, “What’s the problem with doing it that way?”, then you seriously need to read the next two paragraphs!
Unfortunately, our web sites have finite resources resulting in limited scalability. Yes, even cloud sites occationally suffer from this affliction, here are a few examples from this year http://www.crn.com/software/225701829. Load testing without taking latency into account gives the site an unfair advantage during load tests, which does not exist in the real world. This is especially true when your internet traffic hits on Cyber Monday. Think about it this way, when your load test generators are on the same network or physically very close to the Website, then packet round trip time is less than 1 millisecond. Now contrast that with users across the internet with packet round trip times of 30 to 50 milliseconds (that’s if your user is in the same country as the web server) or even 100 milliseconds or higher going to different continents. That translates into network connections staying open for 30, 50, or 100 times longer with internet user traffic compared to your lab setting. This is a significant difference which results in longer HTTP sessions, longer page downloads, higher simultaneous HTTP and application server sessions. This can even impact database activity. Ultimately, means Internet user traffic consumes more site resources compared to testing without the correct network latency.
I’ve conducted numerous load tests in a previous role, and I have literally seen websites crumble under Internet traffic that was less than half the load tested traffic where no network emulation was used during the load testing. Now think about all the other parameters that you should be thinking about, like packet loss, jitter, and the impact that the end user’s bandwidth has on the test dynamics.
Now, let’s re-run your load test and add in end user network impairment (latency, packet loss, bandwidth, and jitter) into the mix using Shunra. Because the network latency that we just discussed now matches that of the end user, you can expect an accurate estimation of the site’s true scalability. With Shunra, you can replicate users coming from a variety of locations so you’ll be correctly replicating the latency of users from multiple geographies for even higher accuracy.
To bolster the accuracy of the network emulation, Shunra has a Network Catcher technology that records network latency and other impairments, and then allows you to use those conditions in products such as Shunra for HP Software while testing your systems. This is important because network latency changes throughout the day depending on internet traffic. Thus internet latency at noon is not the same as it might be at 2AM, when many sites conduct their off hours load test. With Shunra all the network guess-work is taken out of the load test process and you are left with a more accurate load test. That’s powerful! And it’s one of the reasons that I joined the team here at Shunra.
Leave a Reply