29 January 2013 ~ 0 Comments

QTP, leave them mobile testers alone

the wall1 QTP, leave them mobile testers alone

I’m having more and more discussions with customers looking at investing into a mobile test automation solution. It’s a great way for me to listen to their problems and discuss potential solutions. One recurring requirement I’m hearing is the capability for the automation solution to integrate with HP QTP. And I have a big problem with the rationale that makes them request such integration.

I have nothing against HP products (or any testing products) as long as they’re used for what they’re designed for. As an example, Loadrunner is probably best fit for load and performance testing of client-server applications rather than large scale testing of web or mobile applications. Same goes with QTP in my opinion. It’s probably king for functional testing of desktop applications in a waterfall environment but is completely unfit for mobile applications which require dev and test high velocity. Do we still need a rudimentary script language (vbscript! Come on, this is 2013!) for fast test creation? Do we want to write code or rather record directly on an actual mobile device? Do we want to use a product that was built for mobile or a product built for desktop apps, developed 12 years ago? Do we want developers to use … QTP? Really?

I understand the reasons why a mobile test automation company would want to integrate with QTP. This product has a large user base as it’s been used quite extensively in the past to test desktop apps. A huge market opportunity! A lot of testers are (too) comfortable with QTP and as they transition to mobile testing, their learning curve would seem minimal. Makes sense right? This is unfortunately a very shortsighted view that would leave a lot of testers behind in a very tough job market.

Mobile apps are very different animals than desktop apps. Remember those 18 months releases for your typical desktop apps? It was quite comfortable wasn’t it? Well things changed and today release cycle for a typical mobile app can be measured in days (and sometime hours for the most popular one!) and the automation product you’re using needs to have this new paradigm as part of its DNA. QTP has a desktop DNA relying on a prehistoric scripting language requiring heavy volume of maintenance each time your mobile app is built (like every hours!). Additionally, have you ever tried to have a developers write tests using QTP? Mobile apps are all developed in an agile environment where everyone is testing and a lot more developers are writing functional tests (Could they be the future of Software Testing?). I have never seen QTP installed on a developers machine as it is a big fat memory hog and I can’t imagine a developers go back to … VBScript to write … anything.

I might get some traditional testers a bit angry at me for saying that they’re part of the problem. I understand that after working a long time with QTP, you master the tool and as you transition to mobile testing, you might get tempted to continue using it. Changes can be seen as scary, pushing us to get out of our comfort zone without guarantee that we’ll be successful. So quite a number of testers want to stay in the past and are dragging innovation down for the rest of us. Some vendors would do anything to grab that share of the market, knowing quite well that this is not the best technical solution, only driven by revenue. I feel software testing would benefit tremendously if we could eradicate this behavior and maybe adopt a more open and forward thinking mentality similar to what we find in the development world. I don’t forget that there is a new wave of testers riding the innovation wave (you know who you are) but throughout my discussions with customers I’m also a bit shocked to realize that a lot of them are still in the dark ages and are at risk in the long term.

Full disclaimer: Yes, I work for SOASTA and you might find my opinion a bit biased. More than 2 years ago, I’ve joined this incredible team because I was convinced that they wanted to change the status-quo and build the future of software testing. I’ve not looked back ever since and each and every day brings new reasons to think that this was the best decision of my career. Yes I’m responsible for our automated mobile solution and you bet I’m gonna evangelise it a whole lot! But those who know me would tell you that I’m able to keep my impartiality when it comes to software testing. I’m just too passionate to be able to BS on a subject so close to my heart. I would acknowledge innovation when I see it and there are other testing companies on the right path (they push us to be better!) but I feel it is my responsibility to point dysfunction in our industry when I see one.

QTP, leave them mobile testers alone.

Leave a Reply