I’m going to admit one or two things right upfront: I am today not a big fan of HP LoadRunner. I was fan of the Mercury product back in the 90s but it was more by lack of other option to load test client/server application than anything else. I’ve never been a big fan of HP either, their hardware or software. They always felt clumsy or aging compare to the competition. So yes, this article is a bit biased but I’m a bit angry at HP right now and I’ve got to share my feelings with my fellow colleagues in the software and performance testing community.
Call me stupid but I have acquired for my organization some (very) expensive licenses of HP LoadRunner last year. It hurt, and still hurts. So why in hell did I buy LoadRunner if I don’t believe in either the software nor HP? Back in the 90s (First version of LoadRunner was shipped in 1989, 21 years ago !) and early 2000, Mercury has been really good at market and sell their products. I have to give them that. They’ve been precursor in the performance testing tool arena (a niche at the time) and extended their reach in a lot of verticals, including financial services for Tier 1 customers. Well, guess what? Most products my organization is responsible for are targeted at the financial services sector for Tier 1 customers ie. Bank of America, HSBC, Barclays, etc. Desktop based or web base solution. For these customers, you can expect software quality and performance to be really high on their requirement. They expect the Software Testing team organization responsible to test the products they will buy to use “Professional” tools. Tools that they’ve heard of. Tools that their own organization use maybe. They want to see “LoadRunner” as a tool listed in the quality section of any RFP (Request for Proposal). Because they don’t know anything else. Because Loadrunner has been there forever and they assume that it is the best in the industry. They feel reassured. They don’t ask question on how the tool is used, they just want to make sure it exists.
So yes, I’ve bought some licences to make sure I could reassure customers asking about performance testing tools. Most of our other tools are open source ie. Jmeter, Selenium, http-load, grinder, home-grown harness etc. but unfortunately for some industry, open-source is still scary.
Our purchase of LoadRunner (some 12 months ago), coincided with our first interest for Cloud Computing environment Amazon EC2. As I’ve written before, cloud testing is an area which should solve many of our scalability testing problem we’re facing today by offering us at low cost a very flexible and elastic environment. So I had bought a very expensive software, you can bet that I wanted that thing to be able to perform load test in the cloud! It was actually fairly easy to put in place. Your LoadRunner controller is stuck in its own local server thanks to the “let’s suck as much money as possible from our customer” License agreement. But it’s ok, the most important piece are the LoadRunner Agent. This is what you want to get up there in the cloud and this is what we did. We’ve installed our Agent in a couple of Amazon Images, declared them to the local controller and installed our application to load in a separate Amazon Image. And ran our scenario ! No problem there, it does work as expected. LoadRunner in the cloud !
So, why am I mad at HP today? For 2 specific reasons:
Cloud environment have been broadly available for 4 years now (Amazon launched its service in 2006) and there have been an increasing interest for these environment for Software Testing organization ever since. New ventures have quickly grasp the opportunity to offer tools which can take advantage of the cloud: Soasta (started in 2006 !), SauceLabs, BrowserMob, LoadLabs … All taking a different approach ranging from open-source based tooling (Selenium primarily) or proprietary technology such as Soasta and its interesting Performance Intelligence approach to drill down results using a home grown OLTP engine. All offering slick web based UI close to the application they target (desktop is dead soon, remember?). All this time, HP has been really quiet and I’ve been expecting a big announce from them in 2010. Something revolutionary, a real breakthrough which could justify their high cost software. Well folks, I think HP has finally put the first nail in LoadRunner coffin. I’ve read the announce, watched the video, read the white paper. I really tried to get my head around what they’ve been working on lately. All I’m seeing is the possibility to install the LR controller in an Amazon Image (probably a change of a few lines in their licensing code) and some Amazon API integration in the (still desktop based) UI. BIG FREAKING DEAL ! This is at most a marketing dance. And a bad one if I may say. I guess ever since the acquisition of Mercury, HP has cut funding for the LoadRunner organization and maybe these guys are struggling to develop anything meaningful which could keep them up with the competition. They better get their act together soon, or maybe the best thing for the tool is to get acquired by a more innovative company? Frankly, when I watch the presentation of the new offer all I can think about is: HP doesn’t give a damn.
The second reason I’m mad at HP is their condescending tone toward open source tool.
This is what I’m referring to:
And let’s not forget: it takes money to make money, guys. When you have almost no budget to invest in testing tools, you might be convinced to rely on inexpensive or open source testing solutions that provide inadequate testing capabilities.
Come on Mr Tomlinson, this is not 1995! Things have changed. You can’t just have a blanket statement bashing open-source. At least, if it’s your communication strategy, do it properly. Give us facts. Be specific. Or maybe you don’t want to because you’re afraid it would actually give credibility to some tools? If that’s the case, just ignore the whole open-source community. All you’re doing with such statement is alienate decision maker such as myself as well as the GenY who will be soon making the call and who are all for open-source. What you’re doing is very shortsighted. How about you open-up a little bit? How about reaching out to this community you’re so afraid of? You could learn a thing or two and maybe share with them your challenge and initiate a discussion? For the sake of better software? Yes, this is a business, a very good business and you want to protect your reputation and $. I get it. But as I’ve said, your 1990 strategy won’t work forever. You better do something about it or I can see a lot of nails in that coffin.
I’ll be watching closely future releases of Loadrunner. Maybe HP will prove me wrong, I sincerely hope so. I’m not the only one who think that LoadRunner needs to reinvent itself (Here: Will Cloud Computing be the tipping point for change in the test industry: “Even a heavy hitter like HP LoadRunner is considered past its prime for today’s market needs“) and hopefully they will hear our concerns as constructive criticism. Bring us back innovation HP !
Note: I’ve tried to setup my blog to allow comments. Hopefully I will be able to manage the spam.