If you’re not familiar with LoadRunner in the Cloud, it’s one of HP’s latest offerings where they partner with Amazon AWS to offer HP LoadRunner in Amazon’s EC2.  The product is currently in beta and is doing very well.  To read my post about it and to get links to join the HP beta, go here.

As a long-time partner of HP, Shunra was invited to augment the HP/Amazon partnership by including Shunra for HP Software.  In our recent white paper with SAP and HP, we already showed how testing with WAN emulation is valuable for anyone testing distributed applications AND how it all works swimmingly in virtual environments, so the only hurdle was to package everything together.

That packaging is now done.  We worked with HP to build an Amazon Machine Image (AMI) that is based off of HP’s 32 bit Load Generator AMI and includes Shunra’s WAN emulation component by default, and we have defined a very fast and easy process to install our component on the Controller instance.  The Controller component installation for Shunra requires a 15MB download and approximately 90s to install.

Once the new Load Generator is approved internally by HP (in the next few days), it will be added to the list of AMIs available for the HP beta program (again, to request inclusion in the beta program, go here).  In the mean time, you can still take advantage of this offer by joining the HP beta program and contacting me directly.  After confirming your participation in the HP program, I’ll send you download and installation instructions for the Load Generator as well as the Controller components from Shunra for HP Software – both are required to use WAN emulation in your LoadRunner in the Cloud tests.

As I mentioned in my previous post, HP is waiving their license fees for this beta program.  In that light, Shunra has agreed to do the same.  The only fees you will be responsible for are from Amazon AWS.  From personal experience, I can say that the AWS usage fees are quite reasonable.

If you’re not planning to participate in this beta program, but would still like to learn more about application performance testing in the cloud, I encourage you to comment below or send me a message directly.  My contact information is listed at the bottom of this post.

By fiscal year 2013, the policy will require agencies to provide details and road maps on their plans for adopting cloud-based technologies. This story has been corroborated at a post at the Federal News Radio website.

This is the latest indication that the Government is getting very serious about the implementation of a Federal Cloud, starting by all the civilian agencies. During 2009, several groups have been established to test feasibility and learn how much it can be saved hardware and infrastructure.

The drive, accordingly to David Lithicum is obviously cost, the OMB believe that moving to a Federal “Cloud” may save significant amount of money and effort on multiple layers of data centers, equipment, personnel,  facilities costs and power consumption.   A Federal “Cloud” from sea to shining sea my provide coverage for many agencies and projects, however, this is the place to emphasize that performance will be impacted by the location of the user and the distance to the server.  In a “Cloud” environment, testing for latency impact is paramount.

The agencies implementing “Cloud” will have to spend significant efforts testing for latency impact on the application performance, as well as mitigation strategies, like acceleration, mirroring and also distributed cashing.

Shunra technologies can provide the perfect test bed for a Cloud environment, by recreating the operational conditions before and after the move to the “Cloud” and providing empirical evidence of the latency impact on the applications.

PCoIP is a type of display protocol used by remote desktops when doing desktop virtualization.  It delivers the desktop from the host server to the remote user, along with capturing mice and keyboard inputs.  PCoIP also delivers HD audio, USB and the user display as well. PCoIP will enable VMware View to deliver the true PC experience that end-users are seeking, capturing high-resolution graphics and rich multimedia, as well as streaming movies and high-definition audio capabilities. In most cases, the user cannot tell that their PC is not at their desktop anymore.

While PCoIP implementations today are all hardware-based, VMware’s implementation will be all software based (Soft PCoIP) and will not require any additional hardware on either end. VMware’s software implementation of PCoIP uses Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) over port 50002. The TCP port is used for session establishment and control.  The UDP port is used for optimal performance of media and streaming content. All traffic between the host server and remote user is encrypted with 128-bit Advanced Encryption Standard encryption.

VMware will use the software-only implementation of PC-over-IP to support users with 150-250ms latency. They’ll give them “basic” flash (think YouTube videos, not full-on Flash apps), voice over IP, remote printers, local storage, etc.

The implication is that you must plan and test for the deployment over a WAN environment that will produce realistic results, for example using Shunra Virtual Enterprise, to ensure user satisfaction and complete SLA compliance.

However, we may need to apply a sobriety test, where are those servers? And how far out is my virtual storage?  A recent article in Scientific Computing (http://www.scientificcomputing.com/articles-HPC-Cloud-Computing-Pie-in-the-sky-120109.aspx) reminds us that the flexibility of an elastic infrastructure comes with strings attached.

The author presents a case for testing adequate bandwidth, as well as measuring and testing network latency.  Virtual machines can introduce additional latency through the time-sharing nature of the underlying hardware. Since the service provider — and not the scientist — control the hardware, unanticipated sharing and reallocation of machines can significantly affect runtimes.

My conclusion from this article, and others, dealing with the latency issues induced by Cloud Computing, is to include a rigorous testing phase before committing to a migration. An end to end latency measurement and emulation are probably the best ways to ensure smooth adoption of this exciting new technology.

That sounds like a lot of clouds!  What we are simply trying to understand is where and how to deploy your application in the cloud, so it will perform well for your customers.

If you take the case of deploying an online store into the Amazon EC2 cloud, there are many things to consider that will impact your customer’s experience.  Two of the most important are where to deploy what, and how your store will function over the Internet.  To compound the issue, where you physically deploy machines within EC2 is not clear.  Amazon goes to great lengths to avoid you knowing where their zones are.

Fortunately, you cannot only find solutions to these problems, but with cloud computing the process gets a lot easier.  To break the problems down into a logical flow, you’ll first need to know the demographics of your customers.  Where are they?  How will they access the store?  What needs to happen for them to have a positive experience at your store?  If you can answer these questions, you’ll likely come up with some numbers that place percentages of your users in different regions around your target market (US, EMEA, global, etc.).  Further, you’ll know the top two browsers they’ll use and with what kind of connections they’ll access the internet (e.g. Firefox with a 768/256kbps DSL, or IE8 with dial-up – yes people still use dial-up!).  Finally, you’ll have some way to quantify a “positive user experience”.  That usually involves consistency (does it work?) and speed (how fast?).

Using the information above and HP LoadRunner with WAN emulation, you can quickly build a to-scale test in your cloud-based lab and understand what would happen if you hosted your application all in one place and serviced your entire customer base.  Odds are that the test will not meet all of your performance goals the first time you run it.  Most likely, you will need to adjust some things in your store (e.g. reduce chattiness and image resolution, optimize Web 2.0 use) and then begin the discussion about where to deploy what.  Can you keep all of your database servers in one place?  If you have to split your database servers, how will that affect performance?  Will you need to deploy web servers in every zone, will you be okay with one in each region, or will you need something in-between?

Again, not only can answers to these questions actually be answered, but by using cloud-based testing you can arrive at valid conclusions faster than ever.  To understand the network impact of deploying machines in different zones, network performance measurement tools like VE Network Catcher can run in each of the zones to measure network performance between zones and to customer representative endpoints.  Importing that data directly into HP LoadRunner with WAN emulation and using the flexible cloud environment, you can rapidly form and either dismiss or validate hypotheses about how your application should best be deployed.

The benefit of testing in this way is that you can experiment with many different configurations in rapid succession to find the one or few that are most appropriate for your needs.  Trial and error with a live application is not advisable.  And building a test lab with physical machines to simulate the same thing is not only significantly more expensive, but a lot slower process.  As I pointed out in my previous blog, the same ROI formula that explain and fuel the explosion of Cloud Computing for hosting applications are also relevant for using Cloud Computing to test your applications with.

Now you know what I think.  What do you think??

Performance Test Lab testing

Pros: Controlled, repeatable, relatively easy to debug observed problems

Cons: Expensive to build and maintain, commonly implemented without real world network considerations

Live Network testing

Pros: Cheap, good for periodic sanity checks of performance

Cons: Puts non-revenue generating load on the production network, very difficult to debug, high volume testing is not an accurate depiction of end user performance

The good news is that developments in cloud-based testing and in HP LoadRunner have been able to improve the accuracy of Performance Test Lab testing while also reducing the complexity and overall cost.  By moving HP LoadRunner load generators into a cloud-based testing environment, your testing can immediately benefit from the flexibility and scalability of the cloud.

One specific use case is for Peak Testing.  Peak testing is simply another name for load testing.  What “peak” implies is that it is at a scale that is not common.  If you are running, or want to run a test that is not commonly run in your environment, then it is fair to assume that you’ll need to plan ahead to gather the necessary resources.  Working in a cloud environment makes gathering and configuring extra resources on demand easy.  Need an extra five load generators?  No problem; just clone your existing one and wait for them to start.  Need more RAM?  No problem; just shut the machine down, configure more RAM, and restart the machine.  Working with the latest version of HP LoadRunner with WAN emulation makes the story even more compelling.  Need load generation from LA, London, and Tokyo?  No problem; just configure the load generators to emulate those locations and they will.  Sound easy?  It is.  Companies like Skytap already have thought this scenario through and have a great subscription-based model built for you to use today.

The real benefits of peak testing in the cloud are that you can quickly scale your lab up and down without the burden of maintaining a lab large enough to support your peak needs –saving time and money.  It is the end of costly lab build-ups, running tests during maintenance windows in the middle of the night, sending load generators to all corners of the world, and putting test traffic on your production network!

Of course, there are a lot of other ways Cloud Computing can be used in a QA environment; validating new versions of test products and proof of concepts are two that immediately come to mind.  In the end, the same ROI formulae that explain and fuel the explosion of Cloud Computing for hosting applications are also relevant for using Cloud Computing to test your applications with.

Now you know what I think.  What do you think??

P.S. Stay tuned for a coming blog on location-aware deployment testing in the cloud…

Which is why I was glad to stumble upon Shamus McGillicuddy article in www.searchenterpriseWAN.com titled “WAN engineers prepare networks as cloud computing adoption rises” that can be found here.

There are some good observations in the article, I especially appreciate the comment on how application architecture best practices such as minimizing application turns become key when developing applications for the Cloud as well as testing applications for network latency becomes critical as the enterprise is migrating more and more applications to the Cloud.

Many questions still remain though and the impact of Cloud computing on performance engineering is still to be determined. Some of the questions that are on my mind are: (and feel free to chime in with more questions or attempts at answers)

1. How do you manage the performance of an application in the Cloud?

2. What changes if any are needed to the service deployment model and version upgrades?

3. How do you conduct a performance test for an application or a service in the Cloud (internal clouds and external clouds)?

4. How do you troubleshoot a performance issue in the Cloud?

Something to think about…

Best,

Amichai Lesser

I thought when I opened the article that it was going to refer to the major players like Salesforce, Amazon, Google, etc. and their jockeying for position in the space.  Instead the article discusses 3 of the new ways companies are looking at cloud computing.

There is the traditional thoughts on cloud computing that anyone who’s followed the technology has read about.

It is a call to jettison traditional IT altogether, and focus efforts on leveraging the work of professional providers of IT applications, platforms, infrastructure, and services. By this definition, it is indeed a complete change in IT paradigm.

Now there are emerging thoughts of the ‘internal cloud’.

The starting point for any enterprise with existing IT infrastructure investment, according to this camp, is an “internal cloud.” An internal cloud applies the concepts of cloud computing (on-demand resources, pay-as-you-go pricing, and the appearance of infinite scalability) to resources wholly owned by the enterprise consuming the service.

And finally, the ‘private cloud’ concept.

A private cloud consists of IT resources under the control of the enterprise consuming it. Those resources may be owned by the enterprise, consumed from a public cloud provider, or some combination of the two. The only requirement is that the resources be under the direct control of the customer under a unified management system, as opposed to each separately consumed offering being individually managed through the interfaces provided by their respective owners.

Personally, I subscribe to the original thought process of cloud computing, but one cannot ignore the IT investment already made in house and the possibilities of leveraging that capital.  I feel a lot of companies already do the ‘private cloud’ approach, thought people are just starting to put a label on it.

Yet again, it will be interesting to see where this all goes.  Stay tuned.

While the challenges to implementation are significant, what about privacy, portability, accuracy, backup, access control and dozens of factors that must be taken into consideration, I believe the impact of adoption standard and acceptable methodology will be extremely positive, not only on the  cost control and fraud elimination but also on the clinical side as well. Our research centers will have statistical access to unprecedented wealth of empirical data, allowing medical decisions to be made based on huge segments of the population, ensuring diversity in research and treatment. The ability to create portable and interchangeable systems will allow patients to seek and get treatment across state lines; ensure complete medical history in case of accidents or medical emergencies while being away from home, etc.

As technologist, we see this trend as extremely interesting, since new products will have to be developed for authentication, encryption, safekeeping, backup, recovery, and storage in unprecedented capacity. At the same time, these may come in the form of portable storage devices, so you may be traveling with your complete medical history on a SIM card.

A complete new set of challenges will derive from the need to protect the data and block malevolent use of the information, as well as misuse by insurers, potential employers and anyone who may discriminate against a patient based on the context of the medical records.  Are we up for the challenge?  I think we are; this is the perfect opportunity to create new industries and services around this task.

I recently read an article that is probably one of the most thought provocative pieces written about IT in recent years. It is titled is “IT Doesn’t Matter” written by Nicolas Carr and can be found here. As you can guess from the title it lays out a strong argument that as IT becomes more ubiquitous, it becomes less of a competitive advantage and more of a risk factor for organizations.

I found myself taking issues with many of the points in the article, as I think Mr. Carr intended the readers to do just that. One key issue I had is with the view of IT as a single transport layer responsible for processing, storing and transporting information.  IT is more complex and multi layered than that, as I am sure most of the readers of this blog will agree.

My biggest issue with the article is coming from my personal experience, having worked with IT organizations of Fortune 2000 companies for the past 8 years, I’ve seen clear evidence that the implementation of  IT, and how integrated IT is with the business,  is a source for competitive advantage for organizations.

One example I would like to focus on in this post is the performance behind IT services. Performance is a good example to how un-ubiquitous IT still is. The attitude towards performance and the implementation of performance best practices varies from one organization to another and sometimes even between departments within the same organization. I have worked with a big Boston based bank in which performance is treated as a product, where every transaction has clear SLAs in the few second range and performance is an integral part of the product requirement specification. On the other side of the spectrum, I have also worked with companies where performance is that thing you take care of when end users complain. Needless to say that the performance oriented bank is still (even in these times) a very successful enterprise while some of the performance lacking organizations that I mentioned are no longer in business.

One implication of the paper that I do agree with is that more and more IT services will be delivered as web services and hosted in the Cloud. Again in this scenario, performance is not only critical, but can also be the source of competitive advantage, if all software based business processes are going to become commodities as the paper suggests, then the speed at which these services are delivered is going to be a key differentiator between one service provider (let’s say SAP for example ) to another (Salesforce.com as another example).

Performance can also provide cost saving advantages as improvements are made to response time, server scalability and processing efficiencies, the same infrastructure can support more users, increasing the [number of supported users]/[dollars spent] ratio.

I would like to hear about examples you have about competitive advantages delivered through IT and specifically from implementing a performance engineering process? Do you agree with Carr that IT doesn’t matter?

Amichai