03 March 2010 ~ 1 Comment

The Long Tail of Bugs: Should we revisit it ?



thelongtailofbugs4 The Long Tail of Bugs: Should we revisit it ?
I’m done reading the revised version of The Long Tail: Why the future of business is selling less of more by Chris Anderson. The long tail term was coined by Anderson in an article he wrote for Wired in 2004. In his book, he explains that we’ve been moving away from a hits market into a niche market. The increased popularity of the Internet has accelerated this transition. In the original article, Anderson shares 3 main observations:

  • The tail of available variety is far longer than we realize.
  • It’s now within reach economically.
  • All the niches, when aggregated, can make up a significant market.

In his book, Anderson backs up his statement with example taken from today’s various industry: iTunes, Ebay, NetFlix, Amazon, Google, etc. All these company have quickly grasped the concept of “market of multitudes” as Anderson calls it.

Following the article and the original book, a lot of people are seeing the Long Tail everywhere ! There is a long tail in TV ads, guitare tablature (!), college sports, sports in general etc. A hit market in one region of the world could be an unexplored  niche market in another area. The Long Tail is all about the economics of abundance and how to turn unprofitable customers, products, services and overall market into profitable ones.

Bellow a quick overview of the long tail for music.

anatomy of long tail The Long Tail of Bugs: Should we revisit it ?

I strongly recommend you to read the original article and of course the book !

As a little mind game, I’ve tried to apply the Long Tail concept to Software Testing and particularly bugs. Anderson demonstrates that the Pareto principle is not necessarily valid today in retail. Basically, the market of niches could be of greater value than the mainstream market. 20% of the products don’t necessarily account today for 80% of the revenue.

If you’ve been in Software Development and Software Testing, you have realized that the Pareto principle is very much alive. You quickly realize that roughly 80% of high priority issues comes from 20% key causes.

thelongtailofbugsgraph3 The Long Tail of Bugs: Should we revisit it ?
Because these high priority problems are usually highly visible or translate into high financial impact, a software development team will focus a lot of their attention on these. Software Testers will determine areas of high-risk to dictate their approach:

  • New functionalities.
  • Features with complex business logic.
  • Area with security or safety impact.
  • Area with financial impact.
  • Features identified as high priority for stakeholders.
  • Features developed under great time constraint, using new technologies or with lots of code refactoring.
  • etc.

Note that I consider high priority bugs and not high severity bugs. A crash is always severe (your software is on the floor damn it !) but depending on how you end up with the crash, it might not be the most important crash to fix. You might want to read the previous sentence a second time and start screaming. Done? Good icon smile The Long Tail of Bugs: Should we revisit it ? Feel better ? Seriously, would you rather fix a crash which occur when you start your software on any platform or fix a crash which only occur when you’re running your software on AIX 5.3 Technical Level 2 Patch Level 27 with a DB2 8.2 Service Pack 1 patch 35 and with character set ISO 8859-14 (Celtic Languages)? The answer is obvious but everything could change if your largest customer is actually running under the second environment (you still want to fix the first crash though).

What do you find in your long tail of bugs? Well, it really depends on what is considered a priority for your specific customers or market but generally:

  • Bugs occurring on very odd situation and what could be considered as unrealistic path of discovery.
  • Bugs occurring on certain combination of environment.
  • Bugs occurring on non-supported environment.
  • Bugs occurring when testing the software with limits beyond requirement.
  • Low priority usability bugs.
  • Low priority maintainability bugs.
  • Internationalization bugs.
  • etc.

Again, the content of your long tail of bugs varies depending on the type of software you’re involved in, your market, your type of customers. There is not one rule.

At this stage, you might ask yourself the following question:

What depth of the tail needs to be addressed? In other words: When do you stop fixing bugs? (In a sense it is related to the “when do we stop testing” question. I recommend you to read this excellent article from Michael Bolton). Again, there is not one single answer (it would be kind of boring wouldn’t it ?). It really depends on a number of factor:

  • You’re in startup mode and you want to show your software as soon as possible to get feedback.
  • You’re working on a service pack version of a software and you’ve addressed all high priority problems of your highly visible (or high revenue generator) customers.
  • You’re working on a brand new product and your capacity and timing allows you to fix as much bugs as possible (this strategy will compete with the possibility to implement new features of course).
  • You’ve reached code freeze and your product needs to enter its packaging cycle (if you’re shipping CDs for example)
  • You’re insane and you think there is an end to the long tail of bugs icon smile The Long Tail of Bugs: Should we revisit it ?
  • etc.

To answer the original question: Should the long tail of bugs be revisited? Of course ! Your niche bugs (yeah, you know that one with the Celtic character sets, running on this improbable environment) could become your hits bugs of tomorrow ! This is especially true if you’re working on global products where internationalization could redefine quickly your priority! If you have a product which can easily adapt to different target audience, your priority could move quickly as well !

Having visibility on your complete tail of (known) bugs should help you with the longer strategy of your products as you quickly assess if you can easily tackle a new market with the same product.

On top of this long tail of bugs, you can easily imagine a parallel long tail of test cases but maybe more importantly a long tail of features. Do you think your most popular features generate the most revenue or do you think the game is changing and you should instead focus on niche features to grab a potential larger audience?

 The Long Tail of Bugs: Should we revisit it ?

One Response to “The Long Tail of Bugs: Should we revisit it ?”


Trackbacks/Pingbacks

  1. [...] goes for bughunting as well. Fred Beringer has a good blogpost on The Long Tail of bugs (with a clear refference to the “Long tail” and the 80-20 Pareto principle). He asks [...]