Comments on: Replacing Remote Load Generators

By: Accurate Load Testing with VE Desktop for HP Performance Center 9.5 | Application Performance Management Blog - Shunra Software

Thu, 04 Jun 2009 14:27:06 +0000

[...] Past attempts to address this inaccuracy problem included anything from deploying remote load generators across the production network to limiting the scope of the load tests to focus on back-end server scalability while omitting the important aspect of end to end performance testing. Each of these approaches has serious downfalls, some of which are discussed in this post. [...]

By: Emomiaassinty

Emomiaassinty — Wed, 20 May 2009 12:01:29 +0000

Great article / hope to definitely come back again:)

By: Amichai Lesser

Amichai Lesser — Thu, 01 Jan 2009 17:48:44 +0000

Jack,
I have been involved in several comparisons of a remote load generator and one that is running in a WAN Emulation lab. The response time accuracy varies between 92% – 97%, meaning for example that a transaction that took 7 seconds in a remote generator took 6.8 seconds in an emulated lab.
Most of these studies were done with customer scripts so they are proprietary, but one study was done with SAP as part of their evaluation of Shunra for the Co-Innovation lab. This one involved a load generator in Germany compared to a load generator in the lab. I don’t have the data from the comparison but judging by the fact that Shunra was selected for the Co-Innovation lab speaks for itself.

You can read more about the SAP relationship here: http://www.shunra.com/news.aspx?newsType=1&newsId=397

Hope this helps,

Best,

Amichai

By: Jack Scheer

Jack Scheer — Wed, 17 Dec 2008 16:12:21 +0000

Has anyone ever done a comparison? Is there any studies out there that show what the accuracy between WAN emulation and sending remote load generators out? I think WAN emulation sounds great and make sense but I need to prove it to my boss and counterparts.
Any link would be helpful..

By: Les Murphy

Les Murphy — Wed, 03 Dec 2008 18:46:22 +0000

Dave, I have one other comment. Load tests should be repeatable, so that if any variable is changed the comparison between “A” and “B” will be valid. When using the actual production network for testing, any number of changes can invalidate the ability to compare against a baseline.

By: A C

A C — Tue, 02 Dec 2008 19:19:21 +0000

I’ve worked in performance testing my entire career, and this seems like a no-brainer. I’m amazed at how many companies still rely on remote generator deployment for their testing efforts when those generators are logistically challenging to deploy, cumbersone to administer, and produce questonable results (because testing is typically ocurring in off-, non-peak hours, when the network is lightly utilized and “available” for testing). Particularly these days, when teams are being asked to do more with less, Shunra’s approach makes a lot of sense to me.