There was an interesting and bit controversial thread on the agile testing yahoo group over the last couple of days. Some people thought that all agile testers should have programming skills. Others thought that programmers cannot possibly be good testers because they don’t consider overall quality.
It raised a lot of thoughts in my mind, so I thought I’d put them down here. I started my career as a programmer and developed much of my attitude towards testing in my first job. My supervisor was diligent about making me walk through the tests I wrote, and my test results. We did all our own functional type testing, and the operations group did the installation and system testing.
Programmers can make excellent testers, and on agile teams that practice TDD, when asked, they say they spend about half their time testing. Teams that truly practice the ‘whole team’ approach and believe everyone on the team is responsible for quality will work together as a team to make sure all the necessary testing is done.
That said, I probably wouldn’t hire someone who’s main passion was programming and saw the tester role as a stepping stone to being a programmer – mostly because they wouldn’t be committed to the craft of testing. However I do prefer testers with a technical background because it makes it easier for them to collaborate with the programmers. That does not mean I wouldn’t hire testers who do not have programming or some kind of technical background. Depending on the tools the team uses, it is not an absolute necessity… as long as they are willing to learn. I look for attitude.
There are many testers who are “ex-programmers” – I am one, so I wouldn’t let the fact they are programmers stop me from hiring them as testers if that is what they want to do.
Testers do add value to an agile team – with or without programming experience. They can see the big picture and consider impacts to the system that programmers often miss. When programmers are in a team, they are programming at the code level. It is hard for them to switch contexts and be concerned about the big picture impacts. I think we sometimes do teams harm when we ask them to switch contexts often. I’ve seen “programmer only” teams be successful, but they do have to make a conscious effort to think big picture and often have one team member play the role.
To quote Lisa Crispin, “Diversity of skills and viewpoints is critical to a successful project. We need people with different backgrounds, different types of experience, different abilities working together.”
I do not believe an agile team can succeed in the long term without automation of some kind. We need to consider the agile testing quadrants, as well as the pyramid. Our regression suite includes automated unit tests, as well as service layer tests and yes, some GUI and workflow tests. We do need to remember that automation does the checking – it doesn’t find new bugs but exposes unexpected changes. Automation is necessary, but not sufficient, so the team needs the skill set to support all automation tasks.
The team also needs skills to perform exploratory testing, and most often those come from someone with a testing background. I have seen several teams be successful where “manual” testers create/design the tests using Fit, or a Fit-like type of interface, while the programmers create the fixtures in the code itself. This has been very successful, and when it was done using Ruby instead of Java, within 1 ½ years, those manual testers were supporting the whole automation framework. They learned how to code in Ruby, but still collaborated with the programmers on new ideas.
I don’t really care who does the automation – unit tests, API or service layer, GUI / scenario tests, load automation, etc….. , as long as it gets done in a timely way, and provides the feedback necessary.
13 comments on “Programmers as Testers?”
I think it is time to separate the programmer and developer roles. In an agile TDD world most of the verification effort is/should be done by programming tests rather than testing. And a verification engineeer that spends most of his/her time programming tests, is indeed a programmer (has to be, using programming skills), but not a developer.
Since many people mix up the programmer/developer roles, these kind of discussions can sometimes be rather confusing.
If your primary goal is automation, then programmers could contribute as much as professional testers.
But for me, testing about about thinking – thinking about what needs to be tested, how it might best be tested, how the results are evaluated, and how you will know if you are doing a good job of it.
If the programmers are primarily tasked with developing the product, testing-on-the-side may not be the best approach.
Here's some additional food for thought:
http://strazzere.blogspot.com/2010/05/slideshow-optimistic-developers.html
Excellent summary, Janet. If people are willing and conscientious, you can accomplish what you need with the people you have.
I no longer think in terms of "what testers do I need." Instead, I think in terms of representing all the mindsets. The "programmer's mindset" is one of figuring out how something could work. The "tester's mindset" is one of figuring out how something could fail. Both of these are important. They could be the mindsets of a single person, though that's difficult to accomplish in practice.
Put these two mindsets together with the "business mindset" which is one of figuring out what is desired, and you've got what I call the Three Amigos. These three mindsets are minimally sufficient for success, in most cases.
I deliberately used the word programmer to distinguish between the 2 roles. We did so in the book as well. I would like to see 'developer' applied to everyone on the team who works on the product. Unfortunately, most of the time it refers to the programmers (I do it myself in conversation).
Most of what we really talk about are activities – not roles. Some of us are better at certain activities than others. For example, I might be able to script tests, but I have been out of coding too long to be good at that anymore.
If everyone could think "What can we do to help deliver the product / service successfully", and then do what needed to be done, we'd be in a better world.
I like the comment that Joe left – "business mindset"; the 3 amigos – what I call "The Power of Three". not necessarily 3 roles, but 3 mindsets.
>>> Programmers can make excellent testers
Why do you think so? What skills that programmers typically possess to be excellent testers? Your post does not make any further reference to this. Can you elaborate?
Programmers writing automation (that detect code regression – change detectors)does not make programmers as excellent testers or does it?
Programmers in agile teams practicing TDD, spending half their time testing – does not make them excellent testers any more than testers spending equal amount of time in testing. We are talking about skills that make excellent testers right? Aren't we?
>>> Teams that truly practice the ‘whole team’ approach and believe everyone on the team is responsible for quality will work together as a team to make sure all the necessary testing is done.
I think this is problematic – this notion of "whole team", "every one is responsible for quality" makes key testing skills rather generic and something that everyone do. It is similar to everyone contributing to reduce "damage to our earth's climate/ecosystem".
I am struggling to understand the point that you are trying to make here "programmers as testers"?
Shrini
I often feel a bit as a programmer-turned-tester, it's a bit like poacher-turned-gamekeeper!
I've found my background often invaluable to "talk technical" with programmers, and sometimes to translate between the programmers and management.
That said, two of the finest testers I've worked with in the field of avionics turned out to have marine biologist backgrounds. Which goes to show you there are no absolutes!
Being able to program does not make you an excellent tester. Having a certificate does not make you an excellent tester. I did not say what skills a tester requires, although Lisa Crispin and I are writing a 2 part article for SQE on that particular subject. I did mention that they need to be able to do exploratory testing – that requires a great deal of 'thinking skills'. They need to see the big picture so they can look at impacts to the system. There are many skills that a good tester needs.
Every team I've worked with requires different skill sets. For example, some need more technical, some need more domain knowledge. I don't believe there needs to be a "one size fits all" solution. I am suggesting that we open our minds to possibilities.
I think two key skills for testers are "logical thinking" and "communication".
To me "logical thinking" because a tester needs to understand requirements as well as a developer (in my mind better). Testers need to have the same "working model" of the end product a developer should.
And "communication" because it's one thing to find a problem in software, but you have to be able to communicate that problem, so other members of the team can understand the issue.
That aside, I do kind of agree with Janet. It's better than a team has a diverse spread of skills than the same skills replicated in several team members – theres no opportunity for us to learn and develop from each otheer otherwise.
I like your post, and the underlying principles I see as correct. And I also think as you say that it is more important to talk about the activities/tasks rather than the roles, ie when talking about the power of three.
However, I also have to question this:
"Programmers can make excellent testers, and on agile teams that practice TDD, when asked, they say they spend about half their time testing."
Of course they say that they practice testing half the time. But from a testing perspective, I do want to state that they are talking about the good programming practice of writing unit checks. This is not the same as testing.
And I really dont think that /all/ or /any/ programmers would be able to turn testers, however much they would want.
This was really interesting- and as I look to hire people for my team, it is something I think about a lot while interviewing candidates.
Personally, I did not come from a programming background but have always been eager to learn, experiment with programming and interested in the technology my team uses. I don't think programming is a pre-req to good testing, however interest in and comfortable-ness with technology is a requirement. I like testers who are not afraid to dig in to more technical tests or get into discussions with the programmers.
As you said, the most important thing is attitude, interest and mindset.
If we think of testing as an activity, there are all kinds of tests to run. Programmers normally are responsible for their unit tests, but it doesn't mean they can't help out with other testing activities if the need is there. I agree most programmers don't want to be "testers". That is not where their strength is, or their passion so I wouldn't expect all to turn into testers … but, I know many who have. I'm living proof.
I've had a side conversation and there was some confusion. As clarification, I meant that good programmers are good at unit testing. They can be good at other types of testing as well, but they need to learn how as with any new skill. Good testers (QA) have learned those skills and that is their strength.
It really helps you form a better solution, and find and fix any potential problems, /before/ you’ve taken the time to write all of that code.