This article is not about the silver bullet around quality metrics you’ve been looking for ! Whether it exists or not is a vast debate. I’ve been writing about metrics in the past. Especially for important metrics to track when you’re software is out the door. I’m still scratching my head to find key relevant metrics to track during the course of a given project. I’ve been tracking a lot during the course of my career but they’ve never satisfied me fully.
This article is around tools which could help you visualize these quality metrics, at least the one you think can help you assess the quality of your software and project. The most important factor is to find the right metrics. The second most important is to calculate and track them in an efficient way. I’ve started my career using Lotus 1-2-3 to build quality metric dashboard. Thank god, these times are over and they are plenty of tools out there which help you automate the whole process and integrate it in your development framework.
I’ve done a quick round-up of what was available out there especially for the combination Hudson/Maven/Jira which is the framework we’re using.
Code quality metrics
If you’re into Java development, you probably use a set of plugins which give developers quick feedback about their code and are part of the continuous integration. We’re using PMD, FindBugs and Checkstyle primarily (as discussed here) to detect high priority warnings in the code. We run Veracode static analysis with shippable products but these plugins give early warning. We also track unit tests and coverage as part of the continuous integration. The plugins themselves report valuable information but they don’t provide an integrated dashboard and it’s a bit difficult to agregate all these data. That’s what Sonar is supposed to do for you and from what I’ve seen it seems to be the best tool for the job. It integrate as a Hudson plugins but can also be used with Maven. It covers coding rules, unit tests, standard metrics (lines of code, cyclomatic complexity, duplicated code etc.) and integrate PMD and findbugs. One interesting metric they provide (it’s actually a plug-in which can be downloaded here) is technical debt. I’ve been talking about technical debt in one of my previous article and this is something I’m very keen to define and start tracking with this plug-in. Could it be the executive level silver bullet metric I’ve been looking for ? I’ll keep you updated on that one !
You can play with Sonar on their live instance and I’m attaching a screenshot to give you an idea of what you get.
These metrics are a statistician dream as there are so many you can capture, drill, flip upside-down, left and right etc. The debate around these and their relevance and usefulness will last until the end of time. However, if you define some as your primary KPI for your software projects, the goal is to be able to generate and aggregate them quickly, accurately and keep track of them overtime. You can export your Jira or Bugzilla data and create fancy excel spreadsheet or you can try to find a tool which automate the whole process (not that the combo export->spreadsheet->report can’t be automated. But this is beside the point). There are some commercial tool which intend to give you this luxury. But come on, this is 2009 during one of the worst economic crisis the world has experienced. Cheap is king ! I’ve done a quick roundup and it looks like the good people at Penthao (yup, the same one developing a complete business intelligence platform) provides a free Jira and Bugzilla plugins to help us gather our defect information in a nice and understandable way. Nothing fancy but right to the point. It provides quality metrics, reports and analysis. Of course you can track your progress and metrics historically. The Jira and Bugzilla plugin can be found here.