4 reasons why Appium is not ready for prime time

appium 4 reasons why Appium is not ready for prime time

As someone responsible for a mobile automation product, I spend a reasonable amount of time looking at the competition. I particularly like when I see some innovation, something that forces us to do even better and stay ahead of everyone else. Usually, this innovation comes from small company or from the open source community. Appium has been on my radar ever since it came out, but I waited for Appium to get some maturity before trying it out. My conclusion is very simple: Appium is not ready for prime time. It reminds me of Selenium and webDriver in its infancy. Not something you want to invest in for serious work. Maybe fine for individual developers to build some smoke tests, but if you have an enterprise app for example, with complex gestures and a lot of tests to build and maintain, Appium is not a good candidate, yet.

These are the main shortcomings I’ve noted. Some are really showstoppers for me.

#1 – No support for intelligent waits. 

notimedelays.jpg 4 reasons why Appium is not ready for prime time

This is the biggest shortcoming and a big red flag. You can code some time delays (polling loops) and that’s it. A time delay is the WORST mechanism to manage your page flow or address back-end response time variability. Reasons being:

  • How long should I wait after a back-end call? Yeah, let’s put 10 seconds, it should be way enough. It might work for 90% of the time and you end up spending time investigating errors for 10% of your test cases. The goal of automation is to find regression issues in your app! Not problems with your environment.
  • How long should I wait to account for device performance? Transition between pages depends a lot on the performance of your device. Might be almost instantaneous on the latest ipad Air for example with their new CPU and could be crawling on older iOS devices. So should you add 2 secs each time you transition between screens in your app? Should be enough right? Sure, it should be fine. Multiply these 2 seconds by your number of transitions, number of tests, number of devices and you end up with a lot of hours spent WAITING! When you have 10000 tests to run during the night for example, that’s not something you can afford. Especially when developers need feedback FAST!

Without being able to wait on a UI element characteristics (visibility, value, etc.) to manage page flow or back-end call, an automation framework for mobile is pretty much worthless in my book.

#2 – On iOS, you can only execute one test at a time per Mac.

This is a limitation coming from Apple Instruments that Appium uses for execution. I’m working with customers running 5000+ tests per night, on 30+ devices, all in parallel. It would take them many days to run all their tests sequentially. This is a SERIOUS limitation and I don’t know how Appium is going to solve this since they have no control over Apple Instruments. You can always buy 1000 Mac Minis! Yeah right.

#3 – Limited support for gestures.

If your app only use simple gestures, like tap, you could be fine. But this is 2013 and gestures are the name of the game to improve the user experience (double-tap and pan for Google map for example. It allows you to zoom with your thumb for example. Pretty neat!). Appium allows you to write Javascript wrappers to support these gestures. But c’mon .. Do you really want to spend time writing support for gestures or would you rather write new tests for your app?

#4 – Limited support for Android < 4.1

androidversion 4 reasons why Appium is not ready for prime time

Appium only supports API Level >=17. If you look at the number of devices running the different versions of Android (via the Android Dashboard) you realize that Appium can only support a very limited share of the market. Do you want to invest seriously in a framework that only allows you to support 18.2% of your market? I don’t think so. I thought it was pretty crazy to have such limited support and I dug a bit. Apparently, you can support Android < 4.1 by using a Selendroid library but it’s not for the faint of heart by judging by the number of people struggling to make it work.

These are my 4 reasons why I think Appium is not ready for serious work. They’re showstoppers in my book. I’m also not a big fan of the approach Appium is taking, which is fairly similar to Selenium/Webdriver. If you like to write tons of code for your functional tests, be my guest. Of all the WebDriver implementations I’ve seen in my career, a lot required a big investment, especially on the maintenance side. But I haven’t seen all projects in the world.

Appium is young and it might grow to something fit for mobile. And honestly, I hope it will! I’m a big fan of open source and the innovation it brings. But mobile is a big business and the product you pick to test your app is an important choice and investment, and you want to make sure it covers all your requirements. I just don’t think Appium is there yet.

6 thoughts on “4 reasons why Appium is not ready for prime time

  1. While I have my complaints against appium. Let me counter you on your points
    1. No implicit waits – Appium has good support for implicit waits. Ruby_lib ->https://github.com/appium/ruby_lib/blob/master/lib/appium_lib/driver.rb is using them and my tests rarely have to wait on something. Let me know I can provide my logs of this works.

    2. No parallel runs – Appium/sauce labs philosophy is to use emulators in their cloud and the support parallel tests, its in their road map but not there yet

    3. Gestures – there is support for gestures, not in the most easiest of the ways but I am able to flick, swipe, long tap etc. Now tell me which other framework provides you even that?

    4. Limited support for android < 4.1. Now is it appium's fault that google decided to come up with new framework 17+? Also there are couple of projects underway to merge selendroid into appium, or using espresso. If you are motivated enough, you could create implement a page object design pattern and use selendroid and appium in the same test suite.

    Yes it is a significant time investment but what are the alternatives? Not do mobile test automation?

    • Thanks for your comment Satyajit! You bring up some important points I’d like to answer. I will probably write a new post to cover my answer. Thanks again!

  2. I think rather than coming to directly limitations of a tool you should also discuss about it’s pros. I am using appium since its early stages and it has come a long way.

    There is no other opensource tools which supports web, native and hybrid apps on ios and android. Commercial tools are pathetic when it comes to object recognition.

    I didnt find you mentioning about any other tool which is better in comparison and reasons for the same..

    Critizing is easy my friend, contributing is difficult.

    • Thanks for your comment Aniket! I will be watching the growth of Appium closely!

      As for commercial tools and their limitation with object recognition, you should maybe try to register for TouchTest Lite and let me know what you think. I think you’ll be surprised.

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>