According to this article, it looks like testing is overrated !! Code review might be the way to go !
Most studies have found that inspections are cheaper than testing. A Study at the Software Engineering Laboratory found that code reading detected about 80 percent more faults per hour than testing (Basili and Selby 1987). Another organization found that it cost six times as much to detect design defects by using testing as by using inspections (Ackerman, Buchwald, and Lweski 1989). A later study at IBM found that only 3.5 staff hours were needed to find each error when using code inspections, whereas 15-25 hours were needed to find each error through testing (Kaplan 1995).
Well thanks for us in the software testing world, Luke refers to developers testing and I tend to agree with his opinion. Let’s face it, developers don’t like testing and for the majority don’t care about it. I’ve met a lot of developers in my career and I can only think about a handful who were really serious about testing, and one of them was me ! Having a formal testing team as part of the development organization doesn’t encourage them to do more either.
I push strongly to get all developers to write reasonable unit tests. It’s not so much about the code coverage these tests bring, but more about pushing them to step back and think about the code they’ve just written and make appropriate change if needed. Writing unit test is a good way for them to reflect back on their code.
Some of them do some amount of manual testing of their own code but usually do a sloppy job as they don’t find any fun in it and slow them down in their development. If you have a highly qualified team of testers beside them, it’s fine as long as they provide an adequate set of smoke tests which ensure the stability of any given build. That’s the minimum I request from them.
Code review amongst peers is a great investment as well. It’s not the holy grail to get error-free code but should be part your tool set. It does bring benefits which you shouldn’t ignore:
- Find defects: That’s the most obvious objective. Getting a second fresh opinion on your own code is invaluable and might spot some important problems which might have been overseen.
- Knowledge sharing: Having experts in some area is good but you never know when you’ll get a major ramp up of activities in this area or some downsize in your organization. You absolutely need to share code knowledge as much as possible to be able to manage unplanned situation in the best possible way.
- Coaching: Code review is a good way to mentor junior developers and also to expand experienced developers knowledge (when they review other experienced developers’ code)
- Enforcing standards: One of the key to a good product is its level of maintainability and supportability. By enforcing proper industry standards through code review, you will be half there. Additionally, the code review expand the number of developers being able to support this area of the product (it goes back to the knowledge sharing benefit)
There are numerous tools to give quick feedback to developers on the quality of their code. We use a few in our development organization fir our Java based products which I find reasonable:
FindBugs looks for bugs in Java programs. It is based on the concept of bug patterns. A bug pattern is a code idiom that is often an error. Bug patterns arise for a variety of reasons:
- Difficult language features
- Misunderstood API methods
- Misunderstood invariants when code is modified during maintenance
- Garden variety mistakes: typos, use of the wrong boolean operator
FindBugs uses static analysis to inspect Java bytecode for occurrences of bug patterns. Static analysis means that FindBugs can find bugs by simply inspecting a program’s code: executing the program is not necessary.
This makes FindBugs very easy to use: in general, you should be able to use it to look for bugs in your code within a few minutes of downloading it. FindBugs works by analyzing Java bytecode (compiled class files), so you don’t even need the program’s source code to use it. Because its analysis is sometimes imprecise, FindBugs can report false warnings, which are warnings that do not indicate real errors. In practice, the rate of false warnings reported by FindBugs is less than 50%.
PMD scans Java source code and looks for potential problems like:
- Possible bugs – empty try/catch/finally/switch statements
- Dead code – unused local variables, parameters and private methods
- Suboptimal code – wasteful String/StringBuffer usage
- Overcomplicated expressions – unnecessary if statements, for loops that could be while loops
- Duplicate code – copied/pasted code means copied/pasted bugs
These two tools are used on our nightly build and results are reported to developers every day. As a formal static analysis tools we’re using Veracode which is mandatory for us to be compliant with some of our customer’s requirement.