View into test planning at the release level
One of the common pitfalls of agile teams is “Forgetting the big picture”. Lisa Crispin and I wrote an article for informIT in May 2012 http://www.informit.com/articles/article.aspx?p=1905881. Teams tend to focus on individual stories, and often don’t think about the feature or the system until the end game when they get ready to release to the customer or put it into production.
Using a tool such as a test matrix at the release level gives the team a different viewpoint into the testing needed. After the release planning meeting, the team usually has a fairly good idea of what will be included in the release. Plan a workshop for the testers, domain experts, in fact anyone who is interested. The outcome of this collaboration workshop is a high level testing matrix. Start the workshop with a listing all the features down the left side of a spreadsheet. Then add possible test conditions across the top. Sometimes I suggest starting with a testing mind map to generate test ideas (see http://lisacrispin.com/wordpress/2011/02/28/using-mind-maps-for-test-planning ).
Leave open columns for adding extra test ideas, as well as rows for new features, and be prepared to cross off features that may be lowered in priority and moved to a later release. Once the spreadsheet has been created, gray out the cells that are non-applicable. It gives an easy reference for what we want to think about when testing. Here is an example with 3 features and some possible test ideas.
I find that this collaborative process is helpful in getting everyone involved and getting different ideas for testing. The value for me is the planning effort. At this point, the team could put it on the wall and use it as reference, reviewing it with programmers or other team members. However, I have found that there is another use for this test matrix.
Project managers find it a very useful tool to see progress at a high level if we color code the cells as the testing is completed. It is more of a “gut feel” or approximation than an absolute indicator. The colors I have used in the past are:
If we look at the example matrix again, it might look something like the following once development has begun.
You can see at a glance that most of the testing for “Save customer information” has been completed, but we may want to visit security a bit more. There is also a major issue in currency for “Update shipping charges”.
I have used this tool in many of the projects I have worked in and had success with all. I suggest to all the teams I work with to find something simple to show visibility of testing at the project release level. This is only one alternative.
8 comments on “View into test planning at the release level”
Any form of visual always helps. I can see this useful for the System level testing with in sprints. An update will be required as the System and its testing requirements grows.
Thanks for sharing the testing matrix. Are these tests about exploring / experimenting and finding bugs? Or are they verifying / checking the funtionality / feature with specifications?
Since we (all teams) like to use ATDD completely end-to-end and check the complete picture for every story, why should I use another release test plan meeting?
I think we should focus more on treating individual stories as a complete feauture according to acceptance criteria and tested / checked end-to-end.
Don't wait untill the end game to Check or Test. Test ASAP and often.
Kishen,
The test matrix at this level is not meant to replace ATDD. It is another way to look at testing the big picture. One of the problems I see in some teams, is the concentration of testing individual stories prevents teams from stepping back and looking at the bigger picture. The matrix is one way of doing that.
I completely agree that we need to test early, often and provide feedback as quick as we can.
Thank you so much for sharing this article..this is very useful
Hey, This is very impressive article. It helps a lot to every beginner can easily understand about testing of a software. Thanks for sharing this beautiful article keep moving on. I’m very impressed. Well I am Jain Addams, Certified QA Engineer From QA Test Training.
I am impressed by the information that you have on this blog. It shows how well you understand this subject. Best of luck.
Thank you so much for sharing this article..this is very useful
Thank you so much for sharing this article..this is very useful