This quote was recently published with the question “Hmm😶 What’s your take on this?” You can read some of the comments here.
It made me realize that I should probably write a blog post so that people don’t take out of context what I mean. The quote is “almost” correct, but not quite. One word makes all the difference in the world sometimes.
The quote people most likely hear me say is:
“Throw away the ‘then’ in ‘code then test’ – replace it with ‘and’. And maybe, also reverse the order to say, ‘test AND code’. Put the test first.”
This came about because teams don’t think about the term ‘code then test’ as a problem. But every time someone says that phrase, it reinforces that testing comes after coding.
Testing and coding shouldn’t be separated as stand-alone activities. They are part of the same development process. Code is not complete without testing – at least some kind of testing, although some testing activities can be completed without writing any code. An example of this might be when we do an experiment or a simulation to test an idea and the team decides not to follow-up.
I was chatting with Paul Seaman about the Linked-In conversation because I saw his name in one of the comments. He was surprised with some of the comments from folks. He asked, “Why do people think BDD and TDD are about testing? Both could be activities which help and influence testing choices but are not testing.”
TDD – it is about design and building testability into the code. BDD (behavior-driven development) and ATDD (acceptance test-driven development) are about getting shared understanding about what we are going to build. That said, I look at as these practices as testing activities since we are testing ideas and assumptions. It is a testing activity that the whole team can participate in to help make sure we understand the feature or story.
This leads into the next point that Paul was concerned about. Sweeping statements such as “the best and most efficient way to avoid bugs in code is to test the requirements” may put emphasis on the wrong thing. For example, the assumption that “better requirements (whatever better might mean) automatically reduces coded bugs” doesn’t hold for him. I like his thought that “Better quality software requires a holistic goodness in systems of development not a singular focus.”
Paul and I have shared many conversations over the past year, and we’ve discussed many of these themes. We both agree the Linked-In thread could have been a great discussion about collaborating through the lifecycle of development – how we influence, assist, and try to make sure the product that build, ships successfully.
It’s not about testing specific aspects like requirement, or code. It’s about how can we give fast feedback. It is one of the reasons, I created this diagram trying to show continuous testing.
I was influenced by Dan Ashby’s continuous testing model, and I am not completely happy with it yet, but it’s a start. For example, it’s not obvious to everyone what testing activities are happening in the formulate and build section. We need to have fast feedback and practices like ‘show me’ to have the programmer show what he/she coded so bugs can be found quickly and fixed immediately.
In summary, testing is an integral part of the whole development and cannot be separated from the coding aspect of building a product. I think one of my new quotes will be:
6 comments on “Testing And Coding, Not Coding “Then” Testing”
I agree with the idea of continuous feedback; call it testing if you like. Like Weinberg, I consider compilation to be testing. And reviewing. And code reading. Most of what is important in testing has little to do with any notion of “tester.”
While I think the underlying sentiment and overall goal are laudible, I think this model trips over itself in a myopic test-centric view. That’s what a tester would say. Another would say that it’s all iteration. Another would way that it’s all theory-building (which, btw, strongly interacts with an experimentalist’s model of testing). I think that perspective has allowed some bugs to creep into the model, and I think they’re beyond just minor tweaks. Maybe I’m misunderstanding something and you can explain why I’m having these reactions:
“Test in production”? No. Why not mitigate that risk with prototypes? That is a technique to test concept alignment (as well as implementation details like screen layout.)
This is focused on testing. Aren’t there many other forms of feedback, of which testing actually forms only a small part? Words mean things. Testing, as usually understood by a software or manufacturing person, is one of the least efficient ways we know.
There is a spectrum from “waterfall,” to spiral, to one-by-one production. Isn’t this picture just spiral with a lot of words added? Process research, Scrum, and lots of others show that one-by-one production (everybody doing everything all the time) is much better for quality and innovation than the others. (That’s the whole story of “New New Product Development Game.”) That is, the shift is on the order of seconds or hours, not days or weeks.
In the plethora of terms, where are modeling, prototyping, instrumentation, measuring, auditing, observing, reviewing (no review before deploying?)… I could go on. These seem not to quite fit under the terms that are there.
Isn’t the whole picture is very “left:” “analyze,” not “reflect;” “monitor,” not “sense;” “formulate,” not “socialize” (note that this flags the left/right boundary as artificial); and so forth?
In summary: my reaction to the whole thing is this may be useful for hammers if you are a hammer and you want to focus only on your personal role and if you want to remain a hammer forever. I think modern design theory (since Thackara, etc.) moved beyond this some 40 years ago.
@James I have been working on a second iteration of this model and hope to have that blog post out shortly. Hopefully it addresses some of your concerns. One of the ideas I was trying to get across is that development is not linear. Also, because it is a ‘model’, it has flaws and doesn’t work for everyone. I tried to keep it simple, so it loses some of the detail. For example, in I would definitely fit prototypes into testing ideas.
Hi Janet,
I speak your opinion Janet and I go even further: for me, developers should spend more than 50% of their time understanding business needs without being imbued with both the words and the stories of the business (Event Storming – narratives composed of business events, User Story Mapping – narrative composed of epics and stories, Example Mapping – business rules illustrated by concrete examples, CRC Cards confirmed by Class Diagram Collaborative Sketching on a whiteboard). After the automation phase in Test First mode should be easy.
The longer the discovery phase, the less painful the coding phase is; it becomes a side effect of the discovery phase.
What do you think?
Bruno
Those are useful tools for sure, and I encourage them. However, I wouldn’t go so far to say the longer the discovery phase, the better it is. So much depends on context. For example, if we think about the Cynefin model, if it is simple – we’ve done it before, then perhaps we don’t need to spend any (or hardly any) in discovery. But, if it’s complicated then tools like example mapping are a great way to think about edge conditions and get a good understanding. If we are in the complex area, maybe we need to do many more spikes to understand.
So, I guess, I’d have to say “it depends”. sometimes we speed through one area, but need to slow down in others.
Nice one, Janet! Quality Analysts and QA managers must outline the anticipated level of quality, performance, and operative success of the code so that developers running tests know what bugs to look for. They must recognize what issues are non-negotiable and which can be forwarded to QAs (preferably with a few lines of context). Clarity on quality standards also aids tests in applying the appropriate level of rigor to find bugs in their own code. Here is an insightful article that talks about Shifting Left – Using Cost of Quality as a Driver Against Resistance – https://bit.ly/3BBDjcF I also saw an upcoming webinar – https://bit.ly/3zxqWfG Through a real-world corporate case study, learn how to make a case for Shift Left, build an end-to-end plan, anticipate challenges, and manage change in this webinar.
First of all, sorry about me so late with the approval of this comment. It escaped my notice :-(.
I’m glad you liked the post, but I disagree that the quality analysts and QA managers are the ones who define what quality looks like. There are so many different aspects to quality that I believe the whole team needs to be part of defining “process” quality. The product people are the ones representing the customers, and they can talk about “product” quality.