Application Performance Engineering Blog – Shunra Software

Recently I was brainstorming with an insurance company for ways we could help them reduce their costs for business process (application) performance monitoring by introducing WAN emulation into their environment.

The problem was that they had thousands of locations accessing a wide variety (250+) applications.  Being an insurance company, most of the applications really were critical to the end users at every location.  The thought of installing and maintaining probes at every location was daunting.  Further, the reoccurring costs for probes, PCs, and their maintenance was staggering.  If we could find a way to minimize those costs by significantly minimizing the number of probes needed, then the ROI for the company could be enormous.

The first thought was to connect a few probes to Shunra’s WAN emulation appliance and then have the appliance emulate whatever network conditions they wanted.  To understand what network conditions to use, they would us VE Network Catcher to record the latency, packet loss, and available bandwidth to the live locations and then have that data automatically fed into the appliance for emulation.

From a high level this idea could work quite well.  You would have a small number of probes connected to the appliance, running through loops of scripts being subjected to recent network conditions for each location of interest.  The problems started to surface when we dug deeper.  The main problem was that the probes were never designed for such a use.  They were not easy to configure in real-time.  They generally ran in loops, on set intervals, and assumed that they are always in the same location.  This made reporting results for several emulated locations very difficult.

At this point, we took a step back and re-evaluated the goals of what we were trying to accomplish.  Our first thought was to reduce the number of probes by using WAN emulation to dynamically alter network conditions.  Fundamentally, the company wanted to know what their business process performance was at each location and to have a way to generate alerts when performance degraded beyond set thresholds.

With a higher understanding of the goals in mind, we came up with a much simpler solution.  The VE Network Catcher would still be used to monitor all network links 24/7.  Instead of trying to run probes together with WAN emulation, we would use VE Profiler to understand the network thresholds where the business process SLAs would be violated.  As inputs to VE Profiler, we would use the network metrics collected from VE Network Catcher and reuse the VUGen scripts from the probes in LoadRunner.  With a complete profile of the network sensitivity of the business processes, we would set thresholds in VE Network Catcher for latency, packet loss, and bandwidth utilization.  When network conditions degrade to levels that are known to cause poor business process performance, SNMP alerts will be triggered and corrective action can be taken.

This solution is not a complete replacement for maintaining probes in the data center and production network.  Probes are still important to ensure that the application servers are available and performing reasonably at a few, key locations.  For the many hundreds of other locations within the company, running periodic (monthly or quarterly) VE Profiler tests, updating the thresholds in VE Network Catcher, and waiting for any alerts is a perfect solution for this company’s needs.

Do you have any questions or comments?  I would love to hear your thoughts on this topic.