Software Quality

January 16, 2011

Code Freeze 2011 was Great! The theme this year was TESTING

Filed under: Practices, Presentations, Testing — David Allen @ 7:42 pm

The latest Code Freeze 2011 was a great event. The University of Minnesota hosted it again, and it was sold out.  We lead several code dojos. The one I assisted with was intended to demonstrate TDD and pair programming techniques. The audience engaged us with a healthy dose of skepticism.  Here were some of their concerns and reactions.   As usual, I am not satisfied with how we answered them on the spot, so I am reprising the Q & A session here with responses that are more thoughtful and informed by the collective responses of the other dojo leaders I heard.


First question


Programmer: I want to write my code first, deliver it, and then write my automated tests last. That way I can deliver the code to the testers sooner, and I can be writing my automated tests while they are doing their functional testing.

 Coach: What if the requirements you coded against are wrong?

 Programmer: That’s not my responsibility. I cannot recheck the requirements. The business analyst and customer came up with them. I just code what I am told.   Do you seriously expect me, whose specialty is technology, to detect ambiguities and confusion in the requirements?

 Coach: No, not using the methods you currently follow.  But there are alternative methods. Test driven development (TDD) is one of those alternatives.  Using TDD, you write a single test, and then you attempt to code it.

 Programmer: Yes, I understand the sequence of events. You already tried to teach it to me. But I don’t see how writing the tests first will clarify missing or ambiguous requirements.

 Coach: That is because you have never tried it in a real-life situation on an extended basis.  If you did this, you would discover that when you have vague or conflict in requirements, it is difficult to write your test code.  You’re feeling of frustration and difficulty when you attempt to write the test is often a symptom of vague requirements. I have seen time and time again, where a programmer does not seem to have any difficulty designing and coding software to meet a vague requirement. Apparently they fill in missing assumptions in hopes that they are correct. But if you are unable to write the test, because you have unanswered questions, then you go straight to your customer or business analyst and you get answers to those questions. Those answers are the missing and ambiguous requirements that you need to write the code correctly the first time.

 Programmer: Oh, so I don’t have to single-handedly detect and correct flawed requirements?

 Coach:  That was a compound question you just asked.  You are not responsible for correcting flawed requirements. That is the job of the customer and the business analyst. However, it is the job of every member of the team to report ambiguities and defects as quickly as possible.  As a programmer who writes automated tests first, you will be impressed to find that you will often be the first person to detect ambiguities in the requirements.  Then you can collaborate with those who ARE responsible for getting the requirements right. Thus, you become a valuable partner in the process of uncovering and correcting defective requirements as early as possible in the development cycle. The sooner you detect these flawed requirements, the sooner they can be fixed.

 Think back on your experience.  Have you ever had the experience of coding something, delivering it, then discovering that it wasn’t what they wanted? Then you had to recode it? The initial delivery was certainly fast. But you are not done after the initial delivery. That is an illusion of speed. You are not done with code until it is been tested and demonstrated in production to be successful. If you have to rework code, then you are wasting effort in coding, deployment, defect tracking, and testing. One of the goals of using TDD is to avoid that wasteful rework by detecting flaws in the requirements as soon as possible.

 Programmer: Well, I’m still skeptical. Your story sounds reasonable, but I have trouble believing that writing my tests first will uncover flaws in requirements and reduce the amount of rework experience.

 Coach: Skepticism is healthy. But in this case, you will not learn whether this technique works for you unless you try it for some reasonable period of time.   You have to keep in perspective that the growth in popularity of TDD is partly due to numerous people who have had the experience that it adds value to the development process. Whether it also works for you in your unique situation is for you to determine. 


Second question


Programmer: Why not batch test writing?

It seems inefficient to write a test then write some code, write another test, then write some more code. Wouldn’t it be more efficient to just write all of the tests up front, and then write all the code at once? After all, when I’m preparing tests, I’m getting into a certain mindset and I can really crank on the tests. Likewise, when I’m coding, I want to just be coding.

Coach: Well it depends.

If you are talking about high-level business tests, which express the needs of the business in the language of the business, then writing several of these test scenarios in advance of any development work, may actually be an appropriate activity for the business analyst and the customer. In some cases, a professional tester may even get involved to facilitate thoroughness and precision in writing these test cases. In this situation, the tests will probably be written in natural language.  In my work experience, that is in fact exactly what we do.  We write down several stories that are part of the current iteration; for each story, we write several acceptance criteria. The acceptance criteria follow a format that resembles a test case. So essentially, we are expressing our requirements in a testable form.

But if you’re talking about programmer level tests, where the goal is to test the lowest level class design of the software, then writing several tests at once has a risk. The risk is that you will write 10 automated tests, and after you write the first test you will realize that your design was flawed and you will change it. But at that point you have nine other tests that were coded under the original design assumption. Now they all have to be modified. That level of flux and instability is common in the low-level code. It is less true at the highest levels of the business rules. So, if you are writing tests that you believe will be relatively stable in the face of the code you are getting ready to create, and then by all means write several tests at once if you wish. But just be aware of this risk. Programmers who practice TDD enjoy a sense of flexibility and freedom that comes from coding one test at a time. Each step of the way, they may realize there are better ways to implement the solution. They quickly and easily make changes to the code because they don’t have a lot of baggage to carry around.

Like all of these practices, all we can say for sure is that the notion of writing one test, implementing it, and repeating that short cycle, has proven to be the optimal way for many people to develop code. Whether it fits your unique circumstance or not can only be determined by you after you try the technique. I don’t want to coerce you or bludgeon you into following this practice blindly.  But I want you to study the technique, try it several times in practice, and use your intelligence to determine whether it is a useful approach for you. If you have practiced the technique and you know how to use it, then you can make an objective decision about whether to use it or not. If you don’t learn the technique, then you will inevitably be biased towards avoiding it because the learning curve will intimidate you.


Third question


Programmer: Why waste time refactoring?

This re-factoring step that you do at the end of each Read, Green, Refactor cycle looks to me like it would slow me down. I could produce four times as much code if I didn’t waste my time re-factoring all the time.  Making the code pretty seems like a waste of time. Optimizing and tuning the code to squeeze the last little bit of performance out of it at every step of the way seems wasteful and exhausting given that we work on modern computers which often don’t have performance problems .

(This question is an amalgamation of several related questions).

Coach: First we have to correct some misconceptions about what re-factoring is.

Re-factoring is changing the structure of the code to improve it without changing its behavior. I can see how this definition could lead to some confusion among people who have not studied the body of literature that explains the nuances. Let me be clear about what refactoring is not. Refactoring is not performance tuning, per se. Refactoring is not making code “pretty” according to some vague and artistic sense of style which is purely subjective. When we refactor code to make it better we are usually changing the code so that it conforms to some well accepted noncontroversial standards of quality.  This presumes that your shop as some standards for coding practice. Here are some examples of things that are widely accepted in many shops: replace Magic number with named constants, factor out commonality to avoid duplication, use preconditions to guard against null parameter values, ensure that methods are cohesive, etc…  These are just examples of well accepted criteria. Either you believe that your code should conform to the standards, or you don’t. If you do believe it, then you should correct deficiencies at the earliest opportunity. In the TDD cycle Red, Green, Refactor, we have written some code to make the test pass. All we are asking is that you take a look at the code and make sure that it conforms to your standards. If you don’t have any standards you can skip this process, but I feel sorry for you in that case.  If you do have standards, then why wouldn’t you fix it right away? Exactly what are you waiting for? You are not seriously going to say something silly like “Oh we will have time later on to go back and clean up the code.”… 

As Todd Gardner said in response to this “You want to make sure your code is readable and maintainable for those programmers who may come after you. ”  Now, did Todd mean “come after you” as in “their assignments comes after your assignment, and they must carry on the support of what you created?”  Or did he mean “If you don’t clean up the mess you have coded, they will come after you.” <grin>

If my arguments don’t convince you, then you may want to read a book on refactoring where the techniques and benefits are more thoroughly explained than I have done in answer to this question.   One of the core motivations for refactoring is to ensure that your code is always as good as it can be. Or at least good enough to allow you to continue to make changes and add features without needless friction. If you never attend to the code base it will gradually deteriorate under the onslaught of changes. 

And finally, I can speak again from experience. Those of us, who practice refactoring diligently, rarely find that we are spinning our wheels or wasting time. Rather, we make small but important changes that keep the code maintainable and readable as we go along.


Fourth question


Programmer: How can you write automated tests and then code without designing first?

Whether one is following a waterfall process, and intermediate iterative process like RUP, or a fine-grained iterative process like Scrum, the sequence of events is always analyze, design, code, test.

Coach: You correctly described the historic sequence of development activities. But TDD changes some of that.

In order for us to have a productive dialogue, it will help us to have a common language. This conversation is about the following aspects of the software development process

1. Software development activities

2. Their sequencing

3. Their scope or size

Our coding dojo was focused on the following software development activities: Analysis, Design, Code, Test.  We were also demonstrating pair programming but I won’t discuss that at this time.

These activities can be performed in different sequences. For example, here are three different sequences for these activities

Analysis -> Design -> Code -> Test

Analysis -> Design -> Test -> Code

Analysis -> Design -> Test -> Code -> Refactor (redesign)

There is one more element of our model which we have to define: scope or size of activity. Let us consider how variations in scope would look for the test activity. If you were to sit down and write one test, that would be a small scope of activity. If you were to sit down and write all of the tests for a story at one time, that would be a larger scale of activity. That is the idea that was advocated by the programmer in question number two.  If you were to sit down and write all of the tests for an entire system, that would be a large scope of activity.  That might be the approach would take if you were following a waterfall process.

Now that we have our model of activity, sequence, and scope, we can talk about how things work in real life. In the context of a real project, these different activity sequences can be assembled different ways. At this point in history (2011), the first activity sequence might be called the classical sequence.  It is the sequence you described as the natural way possible to develop software. 

The third sequence is referred to as TDD (Test Driven Design).  And I think I understand why you and I were having trouble communicating with each other. You stated, correctly, that you can’t write code or tests until you have a design.  But when you watched us in those TDD demonstration sessions, you didn’t see any evidence of design. Or at least you didn’t see any evidence of design as you are accustomed to thinking about it.  In TDD, the activity cycle is Red, Green, Refactor. First we write a test that fails (Red).  However, there was some implicit design work done to write a test. The test probably refers to code under test that has not even been developed yet. You saw us do this numerous times. We would type the name of a class that had not even been created.  But where did the name of that class come from? Where did the name of the methods and properties we referred to come from? Of course we made up the names. We have in our mind a design: some class with some methods and properties. In the context of the test we were faced with, they seemed like the kind of class features we would need to make the test pass. So we actually do our design work during the very first step of writing the test. The thing that probably throws you off is the fact that in the TDD approach, we are also making very small design decisions, one at a time. We do not know the final shape of our class structure at this point.  Sometimes, if we have uncertainty about our approach, we may get out a pencil and paper, and sketch out some UML diagrams.  If we have questions about what layer the class will go in, we may pull out a software architecture diagram that illustrates the different layers we have designed.  People who practice TDD are not opposed to doing class library design. Most of us are not opposed to architectural design.  But within the boundaries of those library and layer architectures, the detailed design of the classes will evolve as we continue to satisfy tests. This notion of incrementally growing the design, based upon what is needed to make a test pass is what makes TDD so unique. And admittedly it is very disconcerting for people who are accustomed to doing all of their design work up front.  

We are not asking you to permanently adopt this practice without trying it. We are asking you to try it and experience the way it changes your thinking, your designing, and your coding.    TDD is yet another example of deferring decisions to the last responsible moment, a Lean thinking concept.  In this case, we are deferring a design decision by designing only as much of the class structure as we need to get the test pass. It really isn’t that difficult to do as you saw from our demonstration. We had two people who had zero experience with TDD, and they successfully performed the process in 20 min.    Learning TDD is a lot like attending a yoga class if your only idea of exercise is lifting weights and running.  It is completely different from what you have done before. And it will use your muscles in ways they have not been used before.


When the theme of this conference (testing) was chosen, I don’t believe we thought we could convince absolutely everybody of the value of these techniques in the brief amount of time we had.  What we hoped to do by our brief coding dojos was to demonstrate the TDD is really not that difficult to do. It doesn’t really take all that much time. And you can learn it easily just by watching somebody else do it in a couple of hours. Okay, perhaps you may need a little bit more coaching at some point. You can always go buy a book or look on the Internet for guidance on how to practice TDD. But our goal was to whet your appetite and illustrate that it isn’t something that only a rocket scientist can do.  TDD is something that any programmer can practice. But you will not really understand the benefits and the drawbacks to TDD from a lecture or a book. You’ll have to try the practice for yourself in the context of your real work. Then you can use your problem-solving skills to decide whether it solves some of your problems.


  1. David, Excellent post. Those who have done TDD for a while often have trouble explaining the approach and benefits to those who haven’t yet experienced it. Your post is very helpful to bridging that gap. — Brian Westrich (one of the dojo co-presenters).

    Comment by Brian Westrich — January 23, 2011 @ 10:32 am

  2. Thanks, Brian. The experience reminded me that even if one does something often, teaching others how to do it is yet another effort entirely.

    Comment by David Allen — January 23, 2011 @ 6:43 pm

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at

%d bloggers like this: