Where did the Design phase go? There is a misconception that XP does not promote design. In fact, it is quite the opposite. And I will be talking about it at the Agile Brazil 2009 conference in Rio de Janeiro, Brazil on June 27th.
I have been presenting on XP and Design, where I share my experience on how XP promotes continuous design improvement through Simple Design, Continuous Integration, Test Driven Development (TDD) and Refactoring.
This session was very successful when presented in other conferences (please see history of this session below), bringing the mixed audience (junior and senior developers, as well as coach, testers and managers) into what felt like a good conversation about XP।Similarly to XP’s Evolutionary design, this session has evolved into its current form. The feedback from every time the session has been presented was reincorporated as improvements. I don’t try to sell XP as the silver bullet for software development. Instead I talk about my experience in traditional and agile development approaches, and what I have seen working (or not working so well) in the projects I have been exposed to.
History of this Session
From the past three years, I have been giving training on Introduction to Agile, and Agile development practices, such as Test Driven Development, Continuous Integration and Refactoring.
In the beginning of 2007, I created the Introduction to Test Driven Development and the Continuous Integration sessions for internal training at ThoughtWorks.
In October 2007, I gave the Refactoring to Patterns – a practical look into the Agile approach on Evolutionary Design session at the IndicThreads 2007 conference, Pune, India.
In March 2008, I refactored the previous given presentation into its new form: Evolutionary Design through TDD and Refactoring practices. This session was presented with Kurman Karabukaev at the Google TestaPalooza 2008 conference. This session was awarded the conference’s Certificate of Excellence. The audience really liked the pair-presenting style during the session, and the discussion on different Agile (and non-Agile) experiences by the presenters.
In June 2008, the session evolved into the Agile Evolutionary Design for the Agile China 2008 conference. Once again the session was well received by the audience of 300+ people.
In October 2008, I invited Sudhindra Rao to improve the session and co-present in an upcoming conference. Sudhindra and I had been pair-presenting on ThoughtWorks internal Agile training. The session re-emerged as its current version: XP and Design - Where did the Design phase go? This session was given at the IndicThreads 2008 conference, Pune, India. This session was very successful, bringing the mixed audience (junior and senior developers, as well as coach, testers and managers) into what felt like a good conversation about XP and how we saw it working previously.
Speaking at home country
I am very excited to be speaking at my home country and present on one of my favorite topics: XP Design. Also it is an honor to be speaking next to amazing speakers and agilists such as Jason Yip, Adam Monago, Manoel Pimentel, Rodrigo de Toledo and Alisson Vale.
Lately I have been working in a large Agile program. With large Agile programs, comes common large team challenges, and the need to adapt the Agile development and management practices to better suit the large team challenges.
Currently the program has 180 people distributed over 12 teams. Each team is formed by 15 people (project manager, business analyst, quality assurance and developers; for short: PM, BAs, QAs and Devs). All teams are following agile practices such as 2 week Iterations, Daily Scrums and End of Iteration retrospectives.
In this blog entry I will describe how the daily scrum (or daily stand-up) meeting went through 3 different stages as the program grow from one team to 12 teams (of 15 members), adding up to 180 people on the program.
Stage 1: The Daily Scrum meeting
When the program was small (up to 2 teams of 15 people), each team had a daily scrum. The teams were getting good value out of the daily update. The three questions answered by each team member (what did you work on yesterday, what are you working on today, do you have any blocker?), and some informal catch up by the PMs, BAs, QAs and Devs of the two teams was enough to keep all program members in sync.
Stage 2: Scrum of Scrum with PMs and non-PMs
As the program was growing (consider now 5 teams of 15 people) the teams were still having the daily scrum meeting. However the individual teams started losing context on the status of the program, and important inter-teams updates and communications were not flowing as easily as when the overall program size was smaller.
We started having Scrum of Scrums after the teams’ daily scrum. The Scrum of Scrums (SoS for short) had the same format as the daily scrum: the participants would stand in a circle and would quickly give status update by answering a few questions. The questions were slightly different than the daily scrum questions. The SoS participants would answer the following 4 questions: What did your team do yesterday, What will your team do today, is anything blocking your team, is your team about to put anything in other team’s way?
The participants of the SoS were PMs and non-PMs: team member, such as BA, QA or Dev. The PMs presence was mandatory and they would drive the meeting (answer the 4 questions.) Each team had two representatives: the PM, and the non-PM team member. The PM would daily select the non-PM team member to attend the SoS (this was done in the team daily scrum). At times a team member would volunteer to come, as he/she had to give some important update to the overall program.
The non-PM SoS participants brought very important aspects to the SoS. First they would complement their PM with the extra detail (not too much detail, but enough for spawning the interest for the other program team members.) Second, they were effective radiators of information. They would bring information back from the SoS to their teams (at times the PM was pulled into other meetings prior to returning to their teams.) And last, the non-PM SoS participants made sure that the SoS did not become a PM update meeting.
Stage 3: Scrum of Scrum for PMs
The program was growing in size. Consider the phase where the program reached 12 teams of 15 people. The SoS with PMs and non-PMs became less effective. First, it was taking too long for 24 people to talk. Second, the SoS was becoming a mixed forum of discussion. The PMs wanted to go into further project management details, while the Devs wanted details on the development activities.
We changed the SoS format. At this stage, we broke it into several SoSs: The PM SoS, the Dev SoS and the QA SoS. In fact the PM SoS called was named SoS: Scrum of Scrums. The Devs and QAs meetings had other names such as Dev hurdle and QA catch up. The PMs kept the SoS in a daily basis; the other roles varied the frequency of their catch up meetings.
The SoS became a PM update, but it was an essential (and quick) PM update. Actually, the SoS still kept the stand-up in circle format, which prevented the PMs form giving long updates (specially after standing up in each team stand up, and then in the SoS). Also the PMs were still focusing on answering the original 4 questions of the SoS (described in stage 2).
The art of facilitating the SoS made an enormous difference. It was simple: whenever a PM would go into a detailed update (for example, I found a bug pattern in my team that …), or two or more PMs would repeat the same information (e.g. blocker: my team could not commit because the Continuous Integration server was down) the item would be added to the Parking Lot.
The Parking Lot items would be discussed at the end of the SoS. PMs that were not required could leave the meeting. If other people would be required, they would either be dragged into the meeting room, or a meeting would be scheduled later.
In summary, the described stages of the daily scrum worked for the program in different stages of the program. My main learning from this experience: try what works for your team, but do not hesitate on adapting.
“Algebra is a part of mathematics (math) that helps show the general links between numbers and math operations (adding, subtracting, multiplying or dividing) used on the numbers. Algebra does this by using letters (a,b,c,...) or other symbols to represent numbers, either because the numbers are unknown or because the numbers change during the course of the problem, in which case the letters are called variables.”
I decided to write down the formulae commonly used for tracking and planning agile projects. After listing down the formulae, I will validate them by solving a few sample problems.
The Formulas
Vp = Ro / Ip The planned average velocity (Vp) is the number of story points planned for the release (Ro) by the number of planned iterations (Ip).
V = ∑Si / i The average velocity of a team (V) is the number of delivered story points per iteration (∑Si) by the number of iterations (i).
Ve = V + ∆V Velocity expected is the average velocity (V ) plus the expected velocity variation (∆V). Velocity expected is commonly used for planning activities, such as iteration and release planning.
R = Ro + ∆R The current total story points planned for the release (R) is the original story points planned for the release (Ro) plus the delta of story points for the release (∆R).
Rm = R - ∑Si The number of story points remaining for the release (Rm ) is the current total Story points for the release (R ) minus the sum of completed story points for the iterations (∑Si).
Ir = Rm / Ve The number of iterations remaining (Ir) is the number of story points remaining for the release (Rm ) by the expected velocity (Ve)
Problem 1 An agile team plans to deliver 400 story points in 10 iterations. What is the planned average velocity for the team?
Solution: From the problem statement: Ro = 400 sp, and Ip = 10 it
Applying the formula Vp = Ro / Ip Vp = 400 sp / 10 it = 40 st / it
Answer: The planned average velocity for the team is 40 story points per iteration
Problem 2 A team has average velocity of 10 sp/it. Extra resources are added to the team, thus creating an expected rise in velocity of 4 sp/it. What is the expected velocity for the team?
Solution: From the problem statement: V = 10 sp/it, and ∆V = 4 sp/it
Applying the formula Ve = V + ∆V: Ve = 10 sp/it + 4 sp/it = 14 sp/it
Answer: The expected team velocity is 14 sp/it
Problem 3 In Iteration 1, the team delivered 10 story points, in iteration 2 the team delivered 14 story points, in Iteration 3 the team delivered 12 story points. What is the average velocity for the team?
Solution From the problem statement: ∑Si = (10 sp+ 14 sp+ 12 sp), and i = 3 it
Applying the formula V = ∑Si / i : V = (10 sp+ 14 sp+ 12 sp) / 3 it = 36 sp / 3 it = 12 sp/it
Answer: The team average velocity is 12 sp/it
Problem 4 Consider the team iteration history of Problem 3. The original release plan is to deliver 400 story points. Assume that the team is about to start Iteration 4, and that the team is able to keep the same average velocity from the first 3 iteration for the remaining iterations of the release. The project manager decides to add an extra pair of developer to the team; and the velocity is expected to increase by 2 sp/it. How many iterations (in total) will it take for the team to deliver the 400 story points planned for the release?
Solution From the problem statement: Ro = 400, ∆R = 0 (no scope as added or removed for the release), and V = 12 sp/it (verage velocity calculated in the previous problem), ∆V = 2 sp/it (a dev pair is added to the team and the velocity is expected to increase by 2 sp/it)
The Question: How many iterations (in total) will it take for the team to deliver the 400 story points planned for the release? Total iterations = elapsed iterations + the number of iterations remaining (Ir)
So let’;s calculate Ir… Ir = Rm / Ve
And Ve = V + ∆V
Applying the Ve = V + ∆V: V = 12 sp/it and ∆V = 2 sp/it Ve = 12 sp/it + 2 sp/it Ve = 14 sp / it
Now applyuing Rm and Ve to the Iterations remaining formula: Ir = Rm / Ve Ir = 364 sp / 14 sp / it Ir = 26 it
Total iterations = elapsed iterations + the number of iterations remaining (Ir) Total iterations = 3 it + 26 it Total iterations = 29 it
Answer: It will take 29 iterations (in total) for the team to deliver the number of story points planned for the release.
Problem 5 Same as Problem 4, but 140 sp have been de-scoped from the release. How many more iterations will it take for the team to deliver the number of story points planned for the release?
Solution From the problem statement: ∑Si = 36 sp, Ro = 400 sp and ∆R = -140 sp
The number of story points remaining for the release (Rm ) is 224 sp
Now applying Rm and Ve to the Iterations remaining formula: Ir = Rm / Ve Ir = 224 sp / 14 sp / it Ir = 16 it
Answer: Based in the presented assumptions, the team will need 16 iterations (in addition to the 3 elapsed iterations) to deliver the number of story points remaining for the release.
Problem 6 Same as Problem 4. How many story points should be de-scoped from the release to enable the software to be released after a total of 12 iterations?
Solution: From the problem statement: ∑Si = 36 sp, Ro = 400 sp, Ir = 9 it (12 it – 3 it) and Ve = 14 sp/it
Applying the formulae Rm = R - ∑Si and R = Ro + ∆R Rm = Ro + ∆R - ∑Si
Combining Ir = Rm / Ve, we come to: Ir = (Ro + ∆R - ∑Si)/ Ve Ro + ∆R - ∑Si = Ir. Ve ∆R = Ir. Ve – Ro + ∑Si
Continuing with the high school physics formulae, I turned my attention from the average velocity formula to the final velocity and acceleration formula.
I experimented the acceleration formula on the Agile project I was working on at the time: the sample project. I applied the acceleration formula on the sample project. The application of the acceleration formula on the sample project—and agile context--helped my team during the IPM (Iteration Planning Meeting). The details follow in this blog entry.
Team acceleration
From the physics high school formula explanation:
“Final Velocity and acceleration: the final velocity V of an object which starts with velocity u and then accelerates at constant acceleration a for a period of time (t) is V = u + a.t"
And while relating physics high school formulae to the Agile development practices, I came up with the following parallel:
Final Velocity and acceleration (for Agile): the final velocity V of an team which starts with velocity u and then accelerates at constant acceleration a for a sequence of iterations (i) is V = u + a.i
Team acceleration is the change rate in the team velocity. Team acceleration is especially useful for the calculation of expected velocities for teams in early stages, helping them analyze and predict velocity variations.
In Agile terms, the final velocity and acceleration formula reads as:
V = u + a.i, where:
V is the velocity for the final iteration (final with respect to the context you are using the formula for)
u is the velocity for the initial iteration (initial with respect to the context you are using the formula for)
a is the team acceleration (rate of change in the velocity, from the initial to the final velocities)
i is the number of iterations (from the initial to the final iteration)
The sample project
The table below shows the velocity data for the sample project.
Iteration
Velocity
1
5 story points
2
10 story points
3
How many points to sign up for?
The team was completing Iteration 2. The velocity from Iteration 1 to Iteration 2 was significantly different. Iteration 1 velocity was 5 story points and Iteration 2 velocity was 10 story points. The team had to decide the number of story points to sign up for the next Iteration, Iteration 3.
Exercise 1:
What is the team acceleration from iteration 1 to iteration 2 in the sample project?
Solution In this case, considering the final iteration is Iteration 2 and the initial iteration is Iteration 1:
the velocity in Iteration 1 is 5 sp/it; u = 5 sp/it
the velocity in Iteration 2 is 10 sp/it; V = 10 sp/it
the number of iterations from iteration 1 and iteration 2 is 1; i = 1 it
Applying the formula (V = u + a.i):
10sp/it = 5 sp/it + a.1it
a = (10sp/it - 5 sp/it) / 1it
a = 5 sp/it2
Answer: the team acceleration from iteration 1 to iteration 2 is 5sp/it2.
Exercise 2:
In the sample project, if the team keeps constant acceleration for the first 3 iterations, what is the expected velocity for Iteration 3?
Solution In this case, considering the final iteration is Iteration 3 and the initial iteration is Iteration 1:
the velocity in Iteration 1 is 5 sp/it (sample project table); u = 5 sp/it
the number of iterations from iteration 1 to iteration 3 is 2; i = 2 it
the team acceleration (as calculated in Exercise 1) is 5sp/it2; a = 5sp/it2
Applying the formula (V = u + a.i):
V = 5 sp/it + 5 sp/it2.2 it
V = 15 sp/it
Answer: If the team acceleration remains constant at 5 sp/it2, the team velocity for Iteration 3 will be 15 sp/it.
Planning the next Iteration
The team in the sample project was about to go for the Iteration 3 IPM. We had to decide how many story points to sign up for. For making this decision I calculated the average velocity, yesterday’s weather and the velocity based in a constant team acceleration.
Average Velocity
Average velocity is one way to go about how much you think the team will accomplish in the next iteration. Average velocity is very useful for agile team planning activities such as iteration and release planning.
From the sample project:
Average velocity = (5 sp/it + 10sp/it)/2 = 7.5 sp/it; therefore the team should plan for 7.5 points for Iteration 3.
Yesterday’s weather
I have worked with teams that used yesterday’s weather. The sample project iteration 2 velocity was 10sp/it; therefore the team should plan for 10 story points for Iteration 3.
Constant team acceleration
Considering the team acceleration to remain constant in iterations 1, 2 and 3 (calculated in Exercise 1, a = 5 sp/it2), the team would have a velocity of 15 story points in Iteration 3 (calculated in Exercise 2, V = 15 sp/it); therefore the team should sign up for 15 story points for Iteration 3.
From theory to practive
But, how many story points did the team (from the sample project) sign up for during the Iteration 3 IPM?
12 story points.
Yes, 12 story points! And 12 is not any of the numbers calculated before (average velocity, yesterday’s weather or constant team acceleration). Basically Agile is about adapting; Agile is not an exact science (like physics). I was having fun with the formulae, and I even came up with an interesting formula and concept that I might use again: team acceleration.
I found the team acceleration to be as useful as any other projection number (such as average velocity). It was handy during the sample project IPM for Iteration 3. I find it especially useful for the first few IPMs of a team, for which the team velocity is still accelerating.
Back to the sample project (from theory to practice); how did the team decide to sign up for 12 story points?
During the Iteration 3 IPM, I wrote the following on the whiteboard:
Average velocity: 7.5 sp/it
Yesterday’s weather: 10 sp/it
Constant team acceleration: 15 sp/it
Then I briefly explained the velocity based in the constant acceleration (as the team acceleration was a new concept to the team), and asked: So, how many story points should we plan for our next iteration planning and why?
The team talked about the learning curve in the project, their confidence on the domain and the technology stack, the schedule for the next iteration with regard to leaves and holidays. Then the team decided: “We should sign up for 12 story points”.
Two Agile teams start working on Application A, at the same time developing exactly same functionality. Team 1 delivers with a constant velocity…
While searching on the internet about AgileEVP (Agile Earned Value Management) I came across a formula from my high school physics: v=d/t, the Average Velocity formula. I really enjoyed high school math, physics, and its exams: Two trains leave Station A at the same time traveling in the same direction, Train 1 travels with a constant velocity…
Here is the formula that got me started:
High School Physics Average Velocity
The average velocity v of an object moving through a displacement (d) during a time interval (t) is described by the formula: v=d/t
where,
v = Average Velocity
d = displacement
t = time
Exercise 1:
What is the average velocity for a car with a displacement of 120 km in the interval of 3 hours?
V = d/t, where d is 120 km and t is 3 h
Solution:
V = 120 km / 3 h = 40 km/h
Average velocity: 40 km/h (the car average velocity is 40 kilometers per hour)
So I decided to compare high school physics average velocity with team average velocity as an Agile development concept.
Typical Agile projects use iteration fixed in time, so instead of t, team average velocity uses the number of iterations as the time variable.
Displacement in Agile is measured by means of story points completed.
Velocity, in Agile, team velocity, means the number of story points the team completed in the Iteration.
Agile Team Average Velocity
The average velocity v of a team delivering story points (s) during the iterations interval (i) is described by the formula: v=s/i
where,
v = Average Velocity
s = story points completed i = the number of iterations
Consider sp for Story point unit, it for iteration unit, and sp/it as velocity unit (Story points per iteration)
Exercise 2:
What is the average velocity of a team with a completion of 120 sp in the interval of 3 it?
V = s/i, where s is 120 sp and i is 3 it
Solution:
V= 120 sp / 3 it = 40 sp/it
Average velocity: 40 sp/it (the team average velocity is 40 story points per iteration)
Agile projects extensively rely on the concept of average velocity. Average velocity is very useful for agile team planning activities such as iteration and release planning. For example, my team is about to have an IPM (Iteration Planning Meeting) and we had to decide the number of story points to sign up for the next Iteration. The Average velocity from the last 3 iterations is a good initial value for the team to go about signing up for work for the coming iteration.
Habits are habituated routines of behavior that are repeated regularly, tend to occur subconsciously, without directly thinking consciously about them.
Self-discipline refers to the training that one gives one's self to accomplish a certain task or to adopt a particular pattern of behaviour, even though one would really rather be doing something else. For example, denying oneself of an extravagant pleasure in order to accomplish a more demanding charitable deed. Thus, self-discipline is the assertion of willpower over more base desires, and is usually understood to be a synonym of 'self control'. Self-discipline is to some extent a substitute for motivation, when one uses reason to determine a best course of action that opposes one's desires.
---------------------------------------
Agile is a low ceremony process which, according to the Agile Manifesto, favors individuals and interaction over process and tools. So in agile we try to use the least amount of process to achieve a certain goal; usually the success of the project. The good thing about high process environments is that you don't really have to worry about what to do, you don't even need to grasp the big picture. Just follow the process and the stated rules. Training and ramp-up time are low. Sounds good? In theory it does, however, in reality many cases fall between rules and are therefore not covered by them. Ever talked to a overwhelmed person in a call center because your situation was not in the handbook? Often, I feel that they could help but won't as they are not authorized to make the call. Agile approaches this problem by moving authority from higher up -- the rule writers -- to the individuals actually doing the work. Command and control becomes leadership and cooperation. This autonomy comes with a price; (Self-)Discipline. People in charge need to be able to put trust into those empowered individuals. With discipline comes a certain behavior. That behavior and its outcome builds trust over time. Like in Spiderman when uncle Ben told Peter Parker: 'With power comes responsibility'.
So far so good. This will work great for as long as you stay in the comfort zone. What happens if unexpected events cause chaos and distress? A classical social behavior is that you yearn for something proven and trustworthy. Usually, it is the habit that worked (at least you think) in the past. In no time we forget about our responsibility towards discipline, we are driven consciously or more likely subconsciously by our habits and run with them once more. Worse, this often causes the situation to become even more dire and the vicious cycle is started. How can this be avoided? In a typical software project, there are three ways that change can be enacted.
A pool of talented employees go out on their own and secretly implement the new process (they know management doesn't like them to step up).
Management orders that a new process is being used. (command and control)
Both of the former combined -- employees and management want to improve together (leadership and collaboration)
The first two options have a very slim change to succeed, albeit the first one is somewhat better off. Usually good people stick to their conviction and let management talk. However, there is only so much pressure they can take. (You can change the company or you can change the company -- get my drift)
As for the management ordered approach, I see two possible outcomes: A) Management is curious about agile and sees a potential silver bullet in it, so it decides to 'do' agile and then will flip to next silver bullet once the first clouds appear. B) Management is convinced about agile and therefore would not only mandate, but facilitate the successful roll-out with trainings, on site consultants and other resources. This brings us the last option of the three.
In the third case management and the employees have the same strong believes and are not willing to give up easily. Open communication and cooperation between those two parties allows for frequent adjustments. In case of shifts in fundamental assumptions, it even allows for changes to the underlying core process . Those adjustments, another agile principle, reacting to change over following a plan, make it possible to keep up with a discipline. Also, disciplines are not cast in stone, they are not dogmas. Feel free to adjust a discipline. However, those changes should have a strong cause and be revisited after an agreed period of time to assess their effectiveness. Often a change is only temporary until another problem has been mitigated.
Over time, with continuous repetition, a discipline will be morphed into a habit. Don't expect this to happen quickly, it most likely will take month if not years. Once you have been lucky enough to experience this for real, you will be transformed for life. Suddenly you have a clear vision and understanding you never imagined possible. There is no going back.
It is all about drinking the KoolAid once! That's what turns a discipline into a habit.
While being a TDD--Test Driven Development--coach, I am always asked about testing private methods. I finally decided to write about my experience on testing private methods.
When really doing TDD (i.e. writing the test code before writing the functional code, and then refactoring the code) private methods are guaranteed to be test covered. When driving the functional code design and implementation from the unit test (aka Test Driven Development), no method is created private. Instead, the private methods are extracted—extract method refactoring step--from a public or package level method. The Example 1 below presents a scenario on applying TDD and how the private method is created.
New code development following TDD
Example 1: new code development following TDD
Consider the development example following a TDD sequence for creating methodA and methodB.
On the first sequence, the testMethodA is created for validating the functionality expected of methodA. The methodA is successfully created: All tests including the testMethodA are passing. Then you look for improvements in the code; refactoring.
On the second sequence, the testMethodB is created for validating the functionality expected of methodB. The methodB is successfully created. All tests including the testMethodB are passing. Then you look for improvements in the code; refactoring.
While looking for improvements in the TDD Refactoring step, you recognize that methodA and methodB have a common fragment of code (code duplication). You extract the common code fragment into a private method whose name explains the purpose of the method--methodC. Then methodA and methodB invoke methodC.
In this example testMethodA covers methodA, which invokes private methodC; therefore the private methodC is test covered.
Please keep in mind that this is a simplified example. Your methods and tests should not read like testMethodA / methodA, and testMethodB / methodB. Jeff Patton describes how test cases can describe design in his Test-Driven Development Isn’t Testing paper.
When doing TDD, the private methods emerge from your code. And because you test-drive your development, no functionality is added without a test. So, for code fully developed by following TDD you won’t have to test the private method separately: Private methods are already being tested by previously written tests.
Improving legacy code following Test-Driven Refactoring
Now let’s look into a more realistic example. The majority of the code I have been working on is not new code; therefore I am not doing pure TDD; instead, I am doing Test Driven Refactoring,
Test-Driven Refactoring
Test-Driven Refactoring is an evolutionary approach to improve legacy code which instructs you to have test-proven refactoring intent. Basically, you start by writing a passing test around the code to be improved, and then you refactor the code; improving its internals, yet still passing the test suite.
While doing Test Driven Refactoring, I am trying to perform small refactoring steps and, at times, I find myself attempting to test a private method. This typically happens when I am working on an existing code base with very low test coverage. And the public methods are too complex to write tests for.
Example 2: test driven refactoring for existing low test coverage codebase.
Consider that you want to improve the following code:
public void overstuffedMethodX(){
…
// very complex code
…
// invoke private method methodY()
someString = methodY();
…
}
private String methodY (){
…
}
In the scenario presented in Example 2, the public method does not have corresponding unit tests. And I don’t feel comfortable refactoring code which does not have tests around it. Therefore I will follow Test-Driven Refactoring for improving the code. Below I will explain two different approaches for doing Test-Driven Refactoring for improving the code in Example 2.
Top down Test-Driven Refactoring
First you create tests for the complex public method:
public void testOverstuffedMethodX(){…}
At this point there is test coverage around the overstuffedMethodX() functionality, so you are able to refactor the overstuffedMethodX() code, including refactoring for the private method methodY().
In the top down Test-Driven Refactoring approach, first the unit test for the public method is created, and then its internals (including the private methods) are refactored.
Let’s now look into another approach.
Bottom up Test-Driven Refactoring.
No test for the complex public method is created.
Instead you look for smaller pieces of improvement.
You change the access level for the private method to make it accessible from a unit test.
private String methodY (){}
becomes
String methodY (){} // package level access in Java
Then you write test for methodY()
public void testMethodY (){…}
Then you refactor methodY(), and verify that the improvement works as the testMethodY() test still passes.
In the bottom up Test-Driven Refactoring approach, you first improve the test coverage and the code for the private methods and the internals of the complex overloaded method. By doing so, the code becomes less complex; after that you can start moving up the chain, increasing and broaden the test coverage until you are able to take care of the public method.
When applying the bottom up Test-Driven Refactoring approach for Example 2, the private methodY() is made package level in order to be accessible by its corresponding unit test(consider the package access level for the Java language). Similarly to testMethodY(), other tests are added in a bottom up approach. After increasing the test coverage and improving the code internals, it becomes easier to create testOverstuffedMethodX(), and finally, refactor the overstuffed complex method.
Top down versus Bottom up Test-Driven Refactoring
Even I consider the top down approach to be more purist as it does not change the private methods access level, its implementation is not always straightforward. The public method might be almost “untestable” (e.g., static, dependencies, long, complex) in its current state. For such cases, a bottom-up approach might be more suitable. As I see it today, code refactoring activity is a combination of bottom-up and top-down approaches that enables you to have small proven steps towards a cleaner solution, which still keep the same interface (the external behavior which is verified by the test suite).
Bottom-line, while doing Test Driven Refactoring, I have provisionally added tests for the private methods. In Java, I deliberately remove the private namespace from the method declaration, changing the method access level to package access level. This way, the corresponding test class--located under the same package--can invoke the method and test it.
But I won’t stop the refactoring until I remove the test for the private methods. I have experienced two refactoring sequences which finish without explicit tests for private method.
In the first refactoring sequence, some later refactoring step moves the private method to a different class. Basically, you realize that the method’s functionality is beyond the original class’s purpose; you identify, create the new class, and move the method – which now is public access level.
In the second refactoring sequence, the method access level goes back to being private and you delete the test which was directly invoking it. Because of the increasing test coverage--as a result of the Test Driven Refactoring--new test scenarios are added for the public method which invokes the private method. Once you (perhaps assisted by test coverage and analysis tools) realize the extra test coverage on your private method, you perform my preferred refactoring step--unnecessary code deletion. The test for the private method can be safely deleted as its functionality is being tested by another test method.
Should I add test for private methods?
So, here is my answer to the test private method question: After successfully doing TDD or Test-Driven Refactoring, your code will not have specific tests for the private methods.
If you are developing new code, you are following TDD. You test-drive the development and no functionality is added without a test. Private methods are only created as the result of a refactoring step. And the path of code going through the private method is already being tested by previously written tests. So, after successfully doing TDD, your code will not have specific tests for the private methods.
If you are improving a legacy code, you should be following Test-Driven Refactoring. In this case, you may provisionally add tests for private methods. Gradually, with the increasing test coverage, the tests for the public methods will cover all the paths, including the paths going through the private methods. At this point, you don’t require the tests for private methods anymore. So, after successfully following Test-Driven Refactoring, your code will not have specific tests for the private methods.
