Building high performing Software Testing team – 90 days plan (part 1)

team21 Building high performing Software Testing team   90 days plan (part 1)
You’ve just started your new job as a QA manager, QA Director, Senior VP of Software Testing etc. Your title doesn’t really matter as long as your responsibility implies some of amount of management and responsibility for one or multiple products. There is no secret recipes to become successful in your new job but the first 90 days are an incredible opportunity for you to set the tone for this next move in your career. This limited window of opportunity should be used to create sustainable advantage to pave a successful way for you and the products you’re responsible for. Optimally during these 90 first days, you should be able to:

  • Drive a 90 days tactical plan.
  • Run a deep organizational analysis and provide recommendation.
  • Build a strategic plan.

There are 4 key area to focus on during these first 90 days. In this first part, I’ll focus on the learning stage.

 Building high performing Software Testing team   90 days plan (part 1)

Homework before you start your new job
Generally, there is always a gap between the day you sign your hiring contract and your actual first day. You will still be involved with your previous job or responsibility but all information you can gather about your new company will prove a key advantage as you start.

  • Ask your new boss or HR about any documentation they might be able to give you about the products (user guide, case studies etc.), about the development process (covering testing of course).
  • Get some information about the company itself, revenue figures, key executives, activities (optimally, you should get these information BEFORE you sign the contract, but it might be easier to get additional details after you’ve been hired). Learn the history of the company, how they had grown ie. organically or through acquisition (this is important if you’re responsible for multiple line of products)
  • Try to do a self-assessment of your strengths and weaknesses. You’re starting with a clean sheet with this new company but people will judge you very quickly. Think about what went well and what could be improved based on your past experience. Jot down a few initiatives and improvement you can make in your new job. It can be anything from communication improvement, better listening skills, punctuality etc.
  • If you feel you have a technical gap, try to fix it. If the product is Java based, try to read as much as possible about the latest and greatest Java test tools, technique. You don’t have to become an expert (would be useful though) but at least be prepared so you can have informed discussion with your new team.
  • Try to understand what development process is used in the new company. If you’re not comfortable with the process they’re using (could be waterfall, SCRUM, XP, RUP etc.) document yourself as much as possible. Don’t wait until you start your new job.
  • If you’re not familiar about the business domain, document yourself as much as possible. I was personally not the expert in origination, customer management, collection or Fraud. I had no idea about scorecards and credit bureau. It’s not rocket science, but you have to make the effort to get comfortable with how the products are used by your customers and the related specific business jargon.

In summary, try to be as much informed as possible before day 1. A new job can be overwhelming, the more prepared you are, the better.

Product knowledge
This is true for any level of management. Hands-on experience will be more difficult as you’re higher in the hierarchy (and usually responsible for a growing number of products) but you should do everything possible to get a real experience of the product you’re going to care for.
If your product is well documented, you can do a lot yourself. If this is not the case, make sure you try to get as much information without impacting too much people around you.
Try to get a feel about the stability of the product, its usability, strengths and what you would consider weaknesses. If you find bugs (you will) try to report them using the proper process so you can experience it. Take note. Don’t necessarily communicate about your findings and avoid early judgment. Your note will get useful as you talk about the products to other people within the organization.
Before understanding how the product is tested, try to think about how you’d test it and what are the critical paths. From experience, early thoughts are usually the right ones.

This is one of the most difficult part of the learning curve. You will usually get as many opinion as there are stakeholders and they all have their own perception of the quality of the products and will try to convince you to focus on certain priority. Get information from:

  • Development team
    They will usually think the product is the greatest ever icon smile Building high performing Software Testing team   90 days plan (part 1) but if you start digging a bit you might discover some important shortfall on their end ie. Lack of unit test, manual build process (error prone by essence), no static analysis, no code review, no documented code (hardly maintainable), maybe lack of specific skills etc. A good team will cover these important activities which build quality upfront but discussing with developers (not necessarily the most senior one) will help you paint an objective picture which will give you a sense of the quality of what you’ll be responsible to test. You should also try to understand how they perceive the test team and how well the relationship is going.
  • Project managers
    Important to discuss the timings of releases and their history. What’s a major release? A hotfix ? A Service pack ? Try to understand the historical timing ie. How many hotfix a months? Fixed service pack timing? How acurate are the sizings? You want to understand how much your team has on its plate.
  • Support managers
    If there is a good support team, you should be able to dig into historical statistics around defects and most importantly their type. Understand the volume of defects (requiring change of code) going out to customer and their type ie. functional problem, usability, performance. Is there a trend? Are there obvious and critical problem which should get fixed right away. How many hot production problem? Try to assess the efficiency of the support team ie. Is support a good shield for development and QA? How much help can you expect from them ?
  • Service or delivery
    If your products are customized and delivered by a service team, you can gather quite a bit of important information by talking to delivery people. They’re using your product on a daily basis and should be able to provide useful feedback. They’re in direct contact with customers so they should be able to give you another angle about the perceived quality of the products.
  • Product manager
    They own the roadmap and have the best view about business requirement. They can give you a good view about the future of the product and the component which will become increasingly important. They’re usually in direct contact with customers and should be able to give you some good informed feedback.
  • Customers
    Hopefully (or not, it’s a double edge sword), you might have access to end users. Depending on the size of your customer base, it might be easy to identify clearly area of focus. If the base is large, it’s a bit more difficult as you can’t talk to each customers and they all will have different problem. Informal discussion with customers will help you identify trend and recurrent type of issue.
  • Senior management
    They’ve hired you and will have an opinion about the quality of the product. They might give you advice regarding priorities but most importantly, they expect you to be in charge and present them a strong strategy.
  • Sales
    They should be able to give you visibility on revenue figures per products and customers which should help you understand priority from a financial perspective. They should tell you how they think your organization can help them. Not only can your organization help build a product of better quality but there is a lot of information you can provide them to give confidence to the customers (visibility on process, test strategy, performance figures, benchmark etc.)
  • Existing testing team
    It’s your team. They should be honest with you and let you know where they feel the team should focus from a product perspective.

Talking to all important organization and stakeholders will help you understand how the quality of the product is perceived, what are the most important component of the product (critical path), relationship between development/QA/Support etc. Additionally, discussion should help you identifying high visibility issues, critical decisions which can’t be postponed, quick win that can be accomplished and gain management support.

Current level of testing and the associated process
That’s the fun part if you passionate about testing (and hopefully you are ! If this is not the case, start to wonder why you’ve accepted this position !). This is where you will be able to take a good look at what you’ve inherited if you start with an existing team in place. Depending on where you stand in the hierarchy, you might be able to experience the process and be involved in the actual testing. It’s the best way to get the most objective view. If you’re responsible for a set of products, make sure you still try to be as close as possible from the actual testing to make your own judgment. Having only discussion brings a large level of subjectivity which could be a derailing factor for your mission.

There are key take-away to get as quickly as possible:

  • Current responsibility
    You need to understand quickly the current testing responsibility of your team. If you identify any potential gap, is there another team taking care of it? Responsibility could includes: Functional testing, performance, benchmark, stress, localization, compliance, usability, accessibility, regression, installation, recovery, BVT, security etc. All these type of tests don’t necessarily apply to your product but you need to make sure that you identify any gap which could impact the quality of the software.
  • Current test approach
    How is the current software being tested? White Box, Black box, mostly through the UI (if any), API level, mostly ad-hoc, exploratory, keyword-driven, manually, automatically, risk based etc. You need to quickly decide if the current approach is appropriate or if another approach would be more efficient. Again, hands-on experience with the product and the process will help you make a good judgement.
  • Current functional coverage
    How much of the software the current approach cover? How deep does it go? Is it mostly at a unit level, component level? Are there any business scenario (end-to-end) being performed? Does the software integrate with other products? Is it being tested? This coverage is sometime a bit difficult as there is a large layer of subjectivity. Code coverage helps to some extent but doesn’t really assess the relevance of your tests.
  • Level of automation
    How much of the current testing is automated? How fast is the current test team being able to provide feedback to the development team on any given build? How much does it cost to maintain the automation framework, testcases and relevant data? What’s the ROI?
  • Tooling
    Are the current tools used the best one for the job? Automation tools, test case management, performance testing tools etc. What is the approach taken? Commercial, open source, home grown? Why? What is the associated cost? Can the current tools allow you to scale and pick up additional responsibility or scope?
  • Testing environment
    Is there a proper environment to perform required testing? Is it scalable? How well is it managed? Are you dependent on other team to maintain it? Is it efficient?
  • Data
    Is there a dependency on specific test data? Do you need to generate them? Do we need to get them from customers?
  • Information coming out of testing
    What type of reporting is expected from the test team? Who are the stakeholders? What value does this documentation currently provides? How is the information generated? Is it efficient?

Organizational analysis
If you end up with an existing team, this is a critical exercise. If you have the luxury to create a team from scratch, you can refer to the next part of this series (Plan and organize).

I’ve said it before and I say it again: People are key. They should be your priority #1. In the first 90 days, you should be able to:

  • Identify if there are any unclear roles and responsibilities leading to a dysfunctional structure. Identify if a clear career path framework is in place.
  • Identify high and low performers. Try to get an objective opinion through discussions, observations, past performance records etc.
  • Level of relationship with the other organization, especially how well developers and testers get along.
  • Understand what drives your people.
  • Assess the technical level of your people. Do you have people with development background, IT background, business etc.  Is there a technical gap with developers?
  • What is the training situation? Are there any formal training? Development plan for strong and low performer?
  • What is level of passion within the organization? Do people breathe testing?
  • Understand if there is a lack of QA alignment with business.

During this learning period, there is a lot of listening and observing to do. It’s not so much what you do but how you do it which will be important.

Words of wisdom

  • Avoid politics
    This is pretty much true even after your 90 first days but is particularly true at the beginning of your new job. Getting caught in internal politic or turf battles doesn’t bring you anywhere and your initial knowledge won’t allow you to get any win. Stay open. Don’t buddy up with everybody but treats everyone as equals with openness (not intimacy). Don’t align too quickly with certain individuals.
  • Be independent
    Your company might give you an orientation to the company and organize something formal for you. Don’t bet on it and design your orientation program yourself. Be visible, try to meet a lot of people, network, socialize to some extent.
  • Understand the corporate culture
    The more you understand the corporate culture, the more successful you’ll be. Don’t try to fight it but understand as much as possible to navigate smoothly. How does people communicate? Meeting, email? Who should be invited? Is hierarchy important? What are the office policies toward lunch, hours, time off, telecommuting? If you work in a foreign country, try to understand as much as possible the national culture, what are the do and don’t?
  • Be tactful
    Don’t rush to judgment. Try to understand how some area could be improved in the most constructive way. Involved others. Get their opinion.
  • Find the right work-life balance
    This is true in general but particularly in a new job where you have a lot on your plate and so much energy to do well. Try to balance your life and set the right precedent. If you work 18 hours a day during the first few months in your new job and if you scale down, it could be seen as a decrease of commitment or enthusiasm.

The learning phase in a new job is a critical one. It’s a good one since you learn so much and a lot is new. Doing it in the right way will help you build a strong tactical plan and strategic vision as well as organize your team to match your plan. We’ll see how to achieve this goal in the next part of this series.

 Building high performing Software Testing team   90 days plan (part 1)