Why is HP killing LoadRunner?

loadrunner Why is HP killing LoadRunner?

I’m going to admit one or two things right upfront: I am today not a big fan of HP LoadRunner. I was fan of the Mercury product back in the 90s but it was more by lack of other option to load test client/server application than anything else. I’ve never been a big fan of HP either, their hardware or software. They always felt clumsy or aging compare to the competition. So yes, this article is a bit biased but I’m a bit angry at HP right now and I’ve got to share my feelings with my fellow colleagues in the software and performance testing community.

Call me stupid but I have acquired for my organization some (very) expensive licenses of HP LoadRunner last year. It hurt, and still hurts. So why in hell did I buy LoadRunner if I don’t believe in either the software nor HP? Back in the 90s (First version of LoadRunner was shipped in 1989, 21 years ago !) and early 2000, Mercury has been really good at market and sell their products. I have to give them that. They’ve been precursor in the performance testing tool arena (a niche at the time) and extended their reach in a lot of verticals, including financial services for Tier 1 customers. Well, guess what? Most products my organization is responsible for are targeted at the financial services sector for Tier 1 customers ie. Bank of America, HSBC, Barclays, etc. Desktop based or web base solution. For these customers, you can expect software quality and performance to be really high on their requirement. They expect the Software Testing team organization responsible to test the products they will buy to use “Professional” tools. Tools that they’ve heard of. Tools that their own organization use maybe. They want to see “LoadRunner” as a tool listed in the quality section of any RFP (Request for Proposal). Because they don’t know anything else. Because Loadrunner has been there forever and they assume that it is the best in the industry. They feel reassured. They don’t ask question on how the tool is used, they just want to make sure it exists.

So yes, I’ve bought some licences to make sure I could reassure customers asking about performance testing tools. Most of our other tools are open source ie. Jmeter, Selenium, http-load, grinder, home-grown harness etc. but unfortunately for some industry, open-source is still scary.

Our purchase of LoadRunner (some 12 months ago), coincided with our first interest for Cloud Computing environment Amazon EC2. As I’ve written before, cloud testing is an area which should solve many of our scalability testing problem we’re facing today by offering us at low cost a very flexible and elastic environment. So I had bought a very expensive software, you can bet that I wanted that thing to be able to perform load test in the cloud! It was actually fairly easy to put in place. Your LoadRunner controller is stuck in its own local server thanks to the “let’s suck as much money as possible from our customer” License agreement. But it’s ok, the most important piece are the LoadRunner Agent. This is what you want to get up there in the cloud and this is what we did. We’ve installed our Agent in a couple of Amazon Images, declared them to the local controller and installed our application to load in a separate Amazon Image. And ran our scenario ! No problem there, it does work as expected. LoadRunner in the cloud !

So, why am I mad at HP today? For 2 specific reasons:

Cloud environment have been broadly available for 4 years now (Amazon launched its service in 2006) and there have been an increasing interest for these environment for Software Testing organization ever since. New ventures have quickly grasp the opportunity to offer tools which can take advantage of the cloud: Soasta (started in 2006 !), SauceLabs, BrowserMob, LoadLabs … All taking a different approach ranging from open-source based tooling (Selenium primarily) or proprietary technology such as Soasta and its interesting Performance Intelligence approach to drill down results using a home grown OLTP engine. All offering slick web based UI close to the application they target (desktop is dead soon, remember?). All this time, HP has been really quiet and I’ve been expecting a big announce from them in 2010. Something revolutionary, a real breakthrough which could justify their high cost software. Well folks, I think HP has finally put the first nail in LoadRunner coffin. I’ve read the announce, watched the video, read the white paper. I really tried to get my head around what they’ve been working on lately. All I’m seeing is the possibility to install the LR controller in an Amazon Image (probably a change of a few lines in their licensing code) and some Amazon API integration in the (still desktop based) UI. BIG FREAKING DEAL ! This is at most a marketing dance. And a bad one if I may say. I guess ever since the acquisition of Mercury, HP has cut funding for the LoadRunner organization and maybe these guys are struggling to develop anything meaningful which could keep them up with the competition. They better get their act together soon, or maybe the best thing for the tool is to get acquired by a more innovative company? Frankly, when I watch the presentation of the new offer all I can think about is: HP doesn’t give a damn.

The second reason I’m mad at HP is their condescending tone toward open source tool.

This is what I’m referring to:

And let’s not forget: it takes money to make money, guys. When you have almost no budget to invest in testing tools, you might be convinced to rely on inexpensive or open source testing solutions that provide inadequate testing capabilities.

Come on Mr Tomlinson, this is not 1995! Things have changed. You can’t just have a blanket statement bashing open-source. At least, if it’s your communication strategy, do it properly. Give us facts. Be specific. Or maybe you don’t want to because you’re afraid it would actually give credibility to some tools? If that’s the case, just ignore the whole open-source community. All you’re doing with such statement is alienate decision maker such as myself as well as the GenY who will be soon making the call and who are all for open-source. What you’re doing is very shortsighted. How about you open-up a little bit?  How about reaching out to this community you’re so afraid of? You could learn a thing or two and maybe share with them your challenge and initiate a discussion? For the sake of better software? Yes, this is a business, a very good business and you want to protect your reputation and $. I get it. But as I’ve said, your 1990 strategy won’t work forever. You better do something about it or I can see a lot of nails in that coffin.

I’ll be watching closely future releases of Loadrunner. Maybe HP will prove me wrong, I sincerely hope so. I’m not the only one who think that LoadRunner needs to reinvent itself (Here: Will Cloud Computing be the tipping point for change in the test industry: “Even a heavy hitter like HP LoadRunner is considered past its prime for today’s market needs“) and hopefully they will hear our concerns as constructive criticism.  Bring us back innovation HP !

Note: I’ve tried to setup my blog to allow comments. Hopefully I will be able to manage the spam.

11 thoughts on “Why is HP killing LoadRunner?

  1. Mr. Beringer – this was a great blog post. I too used LoadRunner back in the day. I recall many a day in the 90′s working with the tool and having a great amount of success. But they have become stagnant. I have no idea what took them so long to leverage the power of a cloud environment. I have done the exact same thing HP/LoadRunner is offering now with JMeter for years. Nothing new an innovative there.
    I have said before I think we are living in the golden age of performance testing. Life is so much easier for a performance engineer now than it used to be. The options are incredible and power you have access to now via the cloud is just something you could never imagine 10 years ago. Keep up the great work on this blog!

    -Lawrence Nuanez

  2. Fred,

    All HP is doing is setting up a “Leasing” model for Loadrunner on the EC2 cloud. They are going to charge on an hourly rate is what the guy in the video alluded to, so they will make some big bucks.

    They have actually put a good chunk of money into the tool and technology over the last few years. Admittedly when HP bought out Mercury they moved the majority of the development work to Isreal (where QTP is also developed) from the Silicon Valley facilities.

    What HP is doing is like a lot of business partners would do for Load Test services where you would get a timelock license for the tool (either VUser packs or the whole thing with Controller and VU’s). This was a way to ‘reduce’ the cost of the tool itself via a 3rd party. HP is now just screwing (finishing off) its business/solution partners in this respect.

    I think the ‘demise’ of Loadrunner is a bit premature. It is going to be around for a long time in my opinion and experience. I guess the only thing you could say is that they finally woke up and smelled the coffee, and caught a clue that Cloud based testing is the upcoming thing.

  3. Fred,

    You make a few good points, but I think you underestimate the requirements we have for Enterprise-class software. It takes much more effort and consideration than the other startups you mention.

    I think we deserve at least some credit for taking a product like LoadRunner which was never designed to be working in/on the Cloud and actually make it work. I think some customers will really dig this.

    Cheers,

    -mt

  4. Hi Fred.

    Disclaimer: Fred knows, but many of you may not, I’m the Founder and CEO of SOASTA. And as Fred referenced in his post, we have been offering a Cloud based Test alternative to LoadRunner since 2007.

    Fred also knows, that we at SOASTA, strongly endorse his views on this subject. So much has changed in our Business and Technology requirements since 1989. Web sites today, have become the primary vehicle of communication between corporations and their global user communities. To improve user experiences, websites now must support video, chat, voice, even streaming video to an ever growing number of mobile users. Pretty advance stuff, from what we were all doing back in 1989. What is even scarier, is that the pace of innovation is dramatically on the rise. Websites, will become even more advance in terms of interaction with consumers in the years to come. Today’s testing challenges will become even more extreme in the next 12 -18 months…and candidly will not be solved by yesterday’s tools. This is not intended to be a shot at LoadRunner, just a fact. LoadRunner has had an amazing run..and has been a terrific solution for over 20 years. But we business is driving us towards agile & continuous testing of production sites that increasingly more visual and dynamic. LoadRunner’s time has come, testers of tomorrow will need the next generation of testing solutions.

  5. @Mark: Call me a demanding customer (I am :-) ) but I was expecting a bit more when I read the announce. As I mentioned, we already do Load Testing using LoadRunner in our EC2 environment. All we can’t do is have the Controller up there. I understand really well the needed requirements to build enterprise-class software. It is tough indeed but LoadRunner has been there for more than 20 years so I would expect that these requirement are already embedded within the software. Yes, the Cloud add some Security/Privacy/export classification requirement but with 20 years of experience I’m expecting that this is not really new to the LoadRunner team. It is however probably new to these startups getting in the field and they will have to address these requirement if they want to be successful.

    Overall I’m glad to see HP taking this direction (I guess you don’t have much choice …) but I’m really eager to see a lot more !

  6. Fred, All,

    I suggest you wait and see what HP’s LoadRunner are bringing to table in their coming release.
    I believe that all your talks about “innovation”, “eager to see more” and such will be positively answered.
    I’m glad to see that you are a demaning customer. This is a real driver for us.

  7. A good article… Loadrunner was mainly so successful because they (Mercury) had a very aggressive sales force – Rational at the time had a better product – but just didn’t sell it hard enough. Loadrunner became the de-facto standard (It was the best at the time). I find the Performance Center interface clunky & dated – and I think they have failed to develop it further because of a lack of competition and because the product is so firmly entrenched. HP as far as I can see are milking the product and sweating (aka milking) their customers. They will just buy a new product when they manage to kill this one. Loadrunner does have a time and a place – but they are now faced with increasing good competition. All the more so as people like me do not accept the product as the default solution (Some would treat that a blaspheme). I’ve written a piece on the different types of performance tools out there see (http://www.perftesting.co.uk/choosing-the-correct-load-testing-tool/2011/09/07/)

    Jason

  8. Fred, Nice analogy of Mecury and its dominance in the market. one good thing about LR was Mercury’s marketing and sales activity. at the same time, there were good tools in the market and where i was successful in most of the projects, such as, Silk Performer & PreVueX (later it became SQA Robot & then Rational Robot) but due their lack of marketing, it didnt gained the visibility as Load Runner. SP has been good and especially if you are programming geek – you have the power to write amazing intelligence….because of this mind set created, even the testing team members / testing business guys wants “load runner” as their first choice, even there are equivalent feature rich tools at a reasonable price. i too have a tough time in internally convincing them why not ‘load runner’….

    as far as the cloud env. is concerned, yes, still it needs to scale up and be more compatible on the different OS that cloud service providers offers!!! but LR and other tools are still locked on to Windows OS and making the whole work handicap. well there are other tools, which we tried one one project for a retail customer and was able to scale very smoothly. recorded the scripts on windows and were able to execute the scripts on Linux based machine in the EC2 cloud(Neoload).

    off topic: the most confusing part which HP did was to QC and PC is to introduce version control…this has put the automation team with an addition responsibility of becoming a configuration specialists rather than focusing as test automation experts.

    Hence, my 2 cents Worth….

  9. I have worked with LR, Silk Performer, JMeter, OpenSTA, VSTS and couple other tools.
    I simply loved Silk Performer – in my opinion (and many of my colleagues’) it is by far the best commercially available tool out there. Lack of marketing is killing it too, by the way.

    LR was a pain to work with. Fucntionalities that we could easily achieve with Silk needed you-know-what- kind of complex coding in LR.

    Among the free tools, JMeter worked best. There are a couple of freely available JMeter AMIs (machine images) too in ec2.

  10. Dear Fred,

    It was a great and significant post. In fact you are right. As my personal experience, HP relies on the number of VU licenses to sell load runner whereas with open source tools, you can just scale it up to add number of VUs like in the case of JMeter. Sometimes, it’s hard and difficult to change the perception on the software community including the end-users that proprietary tools are not good in everything and also not all open source tools are bad.

    Hope others especially those from software testing practitioners read this post and trigger their minds.

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>