Last night (Feb 1rsts) at the Scotsman’s pub in Oslo after the XP Meetup, I had an interesting philosophical discussion with Tom Gilb, Geir Amdal and Rune Sørensen.
My definition of quality is pretty simple – A product has a sufficient level of quality if it meets or exceeds customer’s expectations. I know it’s pretty vague, but so are most of the customer’s expectations.
Tom presented some of his ideas about defining attributes of quality with the values varying depending on the defined parameters given by the customer. For example, the quality attribute of availability might vary between 0% and 100%, but the acceptable level for the customer might be 99.5%.
Geir suggested that the most common interpretation of the term in an everyday context (as opposed to the requirements and value discussion) is ‘how well a thing is made’. He then talked about how we often have one conception of quality when we speak of quality in general, and another when we’re talking about specifics such as Tom outlined. What is its relationship to the cost-time-scope triangle of project management?
We discussed pros and cons of all three ideas and really didn’t solve the riddle. However, about 3:00 am this morning (I was suffering from a bit of jetlag), I realized that I could use a simple example to explain my position.
Let’s take 3 makes of cars. The Mini Cooper, Toyota Corolla, and Kia Rio, all small, all imported into Canada.
- Base Kia Rio – $14,000
- Base Corolla – $16,000
- Base Mini – $24,000
I bought a Mini Cooper for lots of different reasons. Ron Jeffries claims it is because of its cuteness factor and that may be part of it, but I also took into account its reputation (how well it is made). Another deciding factor was its acceleration and handling capabilities. I chose the upgraded Clubman S model so paid more than the base price because I wanted the extra room and the added acceleration.
I drove my daughter’s Corolla the other day, and found it lacking in so many ways. Basics I take for granted never even crossed her mind when she bought her car. I missed my heated seats, and automatic door locks – forget the other reasons. However, she bought it because of its resale value. She has a lease and wants to buy it out and to be able to recoup some of her money at the end of it all. She chose the Corolla, even if it was a bit more money at the time than something like the Kia Rio because of its reputation, but couldn’t afford any of the extras like automatic door locks.
Quality has a cost. If we make all cars to the standards of the Mini or a Mercedes, we eliminate a whole market of buyers. So, although I agree with both Tom and Geir, I still think quality is meeting or exceeding customer’s expectations. In my example, my expectations are much higher than my daughters, but I’m still not willing to pay for the perceived higher quality of a high end BMW or Mercedes. And I really didn’t care if it accelerated from 0 to 60 in less than xx seconds. I only cared that it went fast enough for me.
In agile projects, when the customer understands what the critical success factors are up front, it helps the team work towards those delivering to them. If quality is defined as one of those factors, and the customer(s) can articulate it and how much they are willing to pay (a constraint), we can work towards that level to meet their expectation.
6 comments on “Quality – What is it, and what’s the cost?”
The one "problem" can sometimes be perceived quality, and that can be hard for a customer to define. Take the cars again as an example: up until recently, Toyota's were synonymous with "quality" in the minds of many consumers. There are die-hard Toyota fans out there that will carry that with them, no matter how Toyota quality may be slipping by most tangible measurements. There are others that will, based on recent events, decide that Toyotas are not quality, no matter what the company does to address the current problems, will not associate Toyota with quality.
I remember a discussion with a Nortel executive who was describing customer reaction to beta versions of the original OC-series fibre switches. Their first few releases were, by Nortel standards, top quality because they met all of the usual metrics: no severity 1 problems, no more than XX severity 2, etc. However, the customers said that they didn't like the quality of the product. The next release went out with more features, but also with more defects than the usual rules allowed. The customers were thrilled, and thought that release was very high quality. What they were looking for at that time were features, not minimization of defects. That would come later as the product moved closer to release. Objectively, those releases where not high quality, but in the minds of the customers they were.
Perception can be a tough one to understand, and in some cases overcome. But I agree with you 100% that, at the end of the day, its about meeting the customer's expectations.
I still have problems with your definition of quality. For several reasons, it turns out.
But first and foremost because it reduces quality to perception: "A product has a sufficient level of quality if it meets or exceeds customer’s expectations."
Not to put too fine a point on it, this entails that through manipulating customer perception or expectation, I'd be creating quality. To me that does not make sense.
I could well agree with what I assume is the intention behind the definition, though; that the measure of a successful product can be found in meeting or exceeding the expectations of its current or prospective customers. But is that quality?
One might approach it from the other direction; focusing on the customer's reception of the product as a way of investigating its suitability for its intended purpose. Is that related to quality? Yes, definitely. But are the two equal? I'd say no.
You both claimed quality is subjective. And while I'll agree we have no objective way of measuring quality, I believe that when we speak of quality, it is at the very least an inter-subjective conception of aptness, handiwork and the characteristics of its materials we have in mind.
A conception of craftsmanship, then. In combination with its suitability for an intended purpose and the characteristics of the materials from which it is made.
Then there is the Semmelweiss argument, as I like to call it: Robert Martin's argument that a doctor should never agree to forego washing his hands, even though his customer, the patient, insists on it. There should be a minimum standard below which we could not go and continue calling ourselves professionals. This also has something to do with quality. And nothing to do with the customer's expectations. He or she might not even notice. Until something goes wrong.
Because poor quality in this sense does not necessarily mean a product will suffer constantly. But quite often it means that it has a much greater likelihood of failure. Until it does, the customer might be none the wiser. Does that mean the quality of the product drops when a defect manifests, or that it was always of a poorer quality than expected?
Geir
Geir,
There are definitely some things my definition leaves out… I think we all want to feel pride in our work, and if our organization has values that support that, the applicataion will have a high minimum level of quality. For example, taking the analogy of the doctor washing his hands…, in my ideal organization, the programmers would always do TDD, the customer/product owner would always have acceptance tests, the testers would be involved all the way through the lifecycle, and would have the experience and skills to test not only functionally, but also security, load, etc. Alas, not every programmer does TDD,and not every project has a hands on customer, and not all the testers have the background (and don't collaborate with the rest of the team) so things fall through the cracks.
I think Geoff's comments about perception is right on. And, yes, sometimes that means managing customer's expectations.
Hi Janet,
nicely written, and in the first moment I thought: this is a good one.
But then I remember the definition of a former boss: "Quality is, what's the customer is willing to pay". I was always fighting. I hate this definition, because it means that if the customer won't pay anything, we should deliver software which has no quality. That's of course rubish. Your definition is quite close to my bosses definition with subtle difference: What are customer's expectations? Is quality the only expectation of a customer. Of course not. The first and I think most important expectation of a customer is that the product does the right thing and second he expect that it does the "right thing right".
If I deliver a none ordered but useful feature then this is in accordance with your definition better quality (even it's bad quality), because I exceeds customer's expectations.
So you're are talking about effectiveness (doing the right thing), not of quality.
Regards Marko
Nice explanation. I like your entire way of writing. Keep Posting.
Your Blog is one of the best top 100 software testing blogs listed in this article:
http://www.testingminded.com/2010/04/top-100-software-testing-blogs.html
but for me, it's just one of the best! Keep the great work!
——————
If you plan to go in Cameroon, please visit: Offres d'emploi au Cameroun