uTest.com – Can crowdsourced testing help my organization ?

crowdsourcing uTest.com   Can crowdsourced testing help my organization ?
For a company such as Experian, providing best of breed information,analytical tools and marketing services there is a real opportunity to match quickly new market requirements. The world is transforming itself and whoever is able to rapidly adapt, will emerge at the top of the pack. We need to act fast, deliver these new products and ensure the same level of quality our customers expect from us.

As my growing organization is picking up additional product responsibility, I need to be a bit creative to manage 2 important challenges:

  • An increasing need for a larger and flexible heterogeneous hardware environment primarily used for compliance and performance testing.
  • A need to ensure adequate and flexible testing capacity to be able to cope with aggressive release timeframe.

From a test perspective, these two challenges are particularly difficult to manage during the tough economic downturn we’re in. Hardware is not cheap and there is a justified scrutiny for additional budget to increase resource capacity.

On the hardware front, I’m exploring new services such as Amazon EC2 and probably the new IBM Cloud to see if what they have to offer allow me to find flexibility at a reasonable price. I’m having some proof of concept organized around these services to validate the model. I’m also trying to take the crisis we’re in as an opportunity to actually invest in hardware and negotiate firmly with hardware vendors. They’re dying to sell and I believe they are some bargains to be made.

On the testers capacity front, it’s a bit more problematic. My organization is still growing and I’m actually doing quite a bit of hiring. However, with what I currently have in my pipe, I need to be pro-active and explore new directions. I’ve never been a big fan of outsourcing. Been there, done that in my IBM years and I haven’t been too impressed. It might be really good for well established and stable products but for new development in a fast pace environment, it might be more of a burden than anything else.

I’m a big fan of the crowdsourcing concept. It fit really well for a whole lot of product. Companies, small or large, have very quickly understand ways to take advantage of a worlwide crowd for their products. Whether they need help on design, research, idea, actual code etc. In a way, they allow people to be part of a global team, being able to influence the product. You’d argue, quite rightly, that there is much more in in for the company  than for the “crowd”. But that’s a
topic for another article icon smile uTest.com   Can crowdsourced testing help my organization ?

Soon enough, crowdsourcing was going to come to software test and it looks like the smart people at uTest understood it quickly. The latest buzz they made recently around the testing of tweeter desktop applications reminded me about them and I’ve decided to check them out again a couple of days ago … My, they’ve grown and it looks like they’ve been smart enough to focus on the right area to build up their community of testers and target the right softwares to build confidence for potential customers.

  • Gaming is so obvious ! Whether it’s functional test or localization, the uTest model fit really, really well ! The EA’s of the world struggle so much to manage a very flexible and dynamic overall test capacity, building localization centers all over the world to be ready for prime time season (jingle bell, jingle bell …). There is a lot of money to be made here …
  • Mobile applications .. Bang ! spot on. So many mobile providers, carriers, models. The test matrix is huge and the opportunity for automation very slim ! I’m not very fond of throwing large numbers of testers on any given products but in that particular case, the more the merrier !
  • Web apps: Make sense for new services targeted at a large community. It could complement nicely a private beta. I’m a bit more skeptical for complex and business oriented web application where subject knowledge is critical. It looks like Googgle just joined their growing list of customers … Makes perfect sense for them in order to get better detailed report on problems. They’ve been relying on their community of users to report problem for ages. Looks like uTest will
    allow them to take the next step and complement nicely their team of software engineers in test.
  • Desktop apps: It really depend on the apps … The twitter application testing fit really well. Light installation (don’t need to provide a vritual environment for testers) and huge community of enthusiasts.

How would uTest fit for some of the products I’m responsible for ? Am I confident I would get any testers knowledgeable in origination, customer management, fraud detection or collection? Are they ( (both uTest and the testers) comfortable with very technical environment? From a security standpoint, am I comfortable giving access to confidential information to a large crowd of testers (Yes testers sign NDAs but we’re looking for trouble here especially to manage specific
international legislation …). If they move toward long term dedicated group of testers for a given project, how different are they from a typical outsourcing company?

Don’t get me wrong, I think the idea is brilliant and they’re taking all the right steps to be successful ! Sales and Marketing are very reactive (I’ve watched their video on Tuesday night, without giving my phone number. I’ve received a phone call from one of their sales yesterday afternoon at my office ! I’m usually not interested by such call, but I was actually interested by that one). They’re using all the right buzz to grow the interest of both the companies and the potential testers (the twitters apps war on bugs is a good example …).

I think while they’re working aggressively to get more customers on board, they also need to build up their testing community. Be part of it, probably
participate to test conference to build up a specialized test community for their next stage of expansion and tackle different vertical markets. Probably expand geographically and open up sales office to not only grab additional customers but also to build up the test community. This is going to be particularly important if they start getting involved in a lot of internationalization testing.

In any case, they must do something right since they’ve just won the “Best new company of the year 2009” at American Business Awards yesterday ! Pretty impressive for a 1.5 years company !

I’ll watch them closely ! Good luck to them !