Software testing has come a long way since 1960’s from debugging to defects prevention oriented approaches. However, one thing that hasn’t changed much is the way the testers are evaluated. Let us look at the history and patterns of different evaluation techniques.
Let us being with the waterfall era. It was very clear that testers job was to catch all the defects introduced during the development stage. Testers had pretty much no options other than keep showing different metrics to prove their existence.
Some of the popular metrics out of many that were relevant during the waterfall era are listed below:
- Testing effectiveness
- Defects injection rate during each stage of SDLC (Design, Coding and testing)
- Defect detection rate at each stage of SDLC
- Defect seepage rate at each stage of SDLC
Pretty much testing used to be a reactive approach at that time. Testers were in turn measured by the number of defects they identified during each stage of the SDLC. This measurement technique in turn encouraged non-collaborative behaviors in the team. For example, testers knowingly withholding information from the dev team allowing the developers to make mistakes and enabling the testers to catch the defects later!
How testers are evaluated in waterfall?
During the performance review discussions, the test lead would review the feedback from the project manager as well glance at the defect trends in the project. Positive trend always carried more points. Testers used to get some brownie points, if they had uncovered critical defects in production or pre-prod environments.
Testers role in Agile world
In the Agile world, thought process is entirely different, defect preventers are highly regarded. I won’t be covering different aspects of differences in Waterfall and Agile testing since several articles have been written already. However, filtering some key differentiators, in the Agile world:
- The entire team is responsible for the quality
- Role of testers is not to identify defects but to prevent them
- Continuous improvement is the key to succeed in Agile environment
- Every one collaborates to achieve the goals
- Automate testing
- Need for T-Shaped skills
As one could see from the picture above, the Agile testers form the center of the universe.
They are encouraged to work with:
- Product owners and business analysts in refining the requirements and building acceptance tests.
- Pairing with developers, encouraging them to be better testers and guiding them through the quality process.
- They also collaborate with operations team while sorting infrastructure requirements for automated testing and cloud based performance testing.
- Testers collaborate with architects/designers in building testable application.
How testers should be evaluated in Agile environment?
It is really a shame to see that many Agile organizations are still following the waterfall model of evaluation. The testers are still being evaluated on the number of defects detected and the rest of the metrics from the Waterfall world. Agilists don’t recommend the typical individual performance evaluation, however recommend team based approaches. Irrespective of evaluation techniques following parameters should be evaluated:
- Testers collaboration with rest of the team. Check this article to learn how to measure collaboration
- What is the new skill they have picked up towards building T-Shaped skill?
- As part of the development plan, the testers should be encouraged to master atleast one additional skill. It could be learning a programming language, a database admin skill or a config management. A SMART goal should be set and tracked.
- What is the contribution towards defect prevention?
- There are several techniques to prevent defects. Taking the Poka-Yoke idea from Toyota, one could measure the testers contribution towards Exploratory testing, ATDD and for helping the developers to build the quality in.
- What is the contribution towards automation ?
- The testers should carefully do the cost-benefit analysis of test automation of their system and should be measured on these benefits.
I have also observed that there are some sub-roles within testing team such as test analysts, test engineer and test managers. Depending on their position, leadership parameters could be added, as well.
To conclude, tell us, how are testers evaluated in your organization?