Friday, August 7, 2009

Mock Objects In Action at the Agile 2009 conference

How can mock objects help you design your system better? Want to know how mocking saved hours of work? We focus on establishing best practices based on examples with mock objects. We cover design of classes, using mock objects to understand and test interaction between objects of the system. By the end of the session it should become clear how mocking, when applied correctly helps with system design, improves testability by reducing cost of change. An explicit part of this session is dedicated to the Mocking top offenders. We talk with examples about bad usage of Mocks, and its consequences.



Sudhindra Rao and I will be presenting the Mock Objects in Action session at the Agile 2009 conference.

History of this session


I started conducting a session on mocking and testing when we was coaching a few developers. That session then turned into a training session conducted at ThoughtWorks internal training and was quite appreciated for its practicality. After a few additions to that sessions and adding mockito to the mix Sudhindra and I presented it at a couple of conferences. It was received with a lot of enthusiasm.

Recommended readings


Below is the list of recommended readings that I give to the session attendees. Please let me know what you would add to this list.

http://martinfowler.com/articles/mocksArentStubs.html
http://xunitpatterns.com/Mock%20Object.html
http://developer-in-test.blogspot.com/
http://misko.hevery.com/code-reviewers-guide/
http://codebetter.com/blogs/jeremy.miller/archive/2006/01/10/136407.aspx
http://www.jmock.org/oopsla2004.pdf


The demo code for this session will be available at: http://code.google.com/p/mocksamples/

Thursday, August 6, 2009

Agile PM and Architect dialog

At times I get asked if I rather be an Agile PM, or a developer.

Today I had a quick conversation with the “Architect” which made me very proud of being an Agile PM.

So, the Architect (A) walked to my team table and started the short conversation…
A: “Who is the tech lead in this team?”
me: “all developers”
A: “But who makes the important design decisions?”
me: “everyone”

Friday, July 17, 2009

CI, structural changes and merges

Be mindful of the amount of structural changes while in the period that will require more code merge effort.
This was the message I sent to the programmers on a large agile team I have been working with.

Scaling agile has a few caveats to it. This blog post is an awareness message for programmer, which I believe to be complementary to the Continuous Integration practice for large teams.

Below is the message sent to the large Agile team.

We are a large team (+120 programmers: devs and QAs)

Programmers are coding everyday; despite of release candidates and regression periods. How is it possible?

We branch (prior to a new release), then we merge back (after it has been released); all the time ( release branch strategy )…

Consider the following simplified example of a Release formed by 2 dev iterations and 1 regression iteration.

Sample Project Release and Iteration plan

Dev activity –-It1—It2—It3—It4—It5—It6—It7—It8—It9--ItN--
Regression –------------Reg 1----Reg2-----Reg3—-------------
Release sch –------------------Rel1-----Rel2—---Rel3----------


So let’s look at different moments in time:

It2:
Dev activity –-It1—It2
Regression –----------
Release sch –---------

All programmers are working in the main trunk. Any regression activity is happening in the main trunk. There is no release yet. The release candidate is the main trunk itself.


It3:
Dev activity –-It1—It2—It3
Regression –-----------Reg 1
Release sch –-----------------

The release is formed by two dev iteration and one regression iteration.
In this case, the work done for Rel1 is related to It1 , It2, and Reg1; but should not be impacted by It3 development. Therefore the need to branch out the codebase: main trunk, and R1 branch.

Humm… if there is a branch, there will be a merge later: structural changes will have to be merged back!

It4:
Dev activity –-It1—It2—It3—It4
Regression –------------Reg 1----
Release sch –-----------------Rel1

R1 has been released. The code is merged back to the main trunk. The R1 branch now should have very little activity (production support only), therefore it should not create any difficult merge.
Programmers are mainly working in the main trunk; no merge required (remember, we do Continuous Integration!). Go crazy, do the structural changes that you feel are amazing (btw, if it is not, CI will catch it).

It5:
Dev activity –-It1—It2—It3—It4—It5
Regression –------------Reg 1----Reg2
Release sch –-----------------Rel1-----

The development period for R2 is over (It 3 and It 4). The regression work should not be impacted by the development work from It5. A branch for R2 has been created. If there is a branch, the code will have to be merged later on… Please be mindful of structural changes as you will have to merge them.

Have you noticed the pattern?

In the sample presented above:
It 2, it4, It6 development is happening in the main trunk which won’t require large amount of merges later on.

It 3, it5, It7 development is happening in the main trunk, but there is lots of development happening for a release, and those will have to be merged back in the next Iteration.

So if you have a big restructure to be done; should you do it in It7 or It8? (Remember, there is no free lunch, you will have to merge it back!)

Monday, June 8, 2009

Most if-statements are evil

I’ve really come to loath if-statements. It developed over time. At the very beginning they are great as they give you the power to put some logic into you application. Usually a small program in one of your first programming classes. Well, over time I realized the hard way that they clutter your code and make it harder to read and hence understand and maintain. In OO languages you have far more powerful ways to put logic into your application without using if’s. However, this requires a good skill in programming and lots of practice. I’ve come to realize that you can identify a good programmer by her if-avoidance skill. Well, after some time in acceptance but dislike I now try to avoid them at any price and so should you. By the way, I consider switch-statements to be nested if’s.

Frederick Brooks coined once the terms ‘Essential Complexity’ and ‘Accidental Complexity’. The first describes the real complexity which comes from the domain and the problem to solve. The second one is manmade i.e. it is the complexity we get into by our own making like which OS, which language, which framework, … we choose. Using this point of view, there are essential if-statements and accidental if-statements. The accidental ones, what a fitting name, are the ones which need to be eliminated.

Essential if’s
There are things you need to check. Can I save the file, is there enough free disk space.

if (diskHasEngoughFreeSpace()) {
document.save();
}

Those decision are driven from external forces and are out of our control and so we need to check them every time we save.

Accidental if’s
Those buggers creep up everywhere in you code.

if (date == null) {
date = new Date(); // this is called defensive programming, one of the worst things you can do
}

or

if (date != null) {
uiWidget.setText(date.toString());
} else {
uiWidget.setText(“”);
}

Before you get creative in finding a workaround you should analyze whether date actually can be null. If yes, how? Is it a field of the class, a parameter of the method, a return value from a called method. All these questions and their answers provide you with a multitude of options.

/**
* @param date of the document to be printed; must not be null
*/
public void printHeader(Date date) {
// Usually I put those checks into a utility class and a checkParam method.
// i.e. Check.checkNotNull(date, “date”);
if (date == null) throw new IllegalArgumentException(“invalid parameter: date cannot be null. Good bye!”);

// once we are here we know that all parameter are valid
date.doSomething();
…
}

It is a good practice to have an object to be in well pre-defined state after creation.

class DocumentPrinter {
private Date date; // this is the same as private Date date = null;
…
}

It is better to do it this way

class DocumentPrinter {
private Date date = Date();
// if this is not sufficient then have  the date passed in as parameter with the constructor and make the default constructor private
…
}

or even better use the Null-Object pattern

class Document Printer {
private Date date = new NullDate();
…
}

class NullDate extends Date {
public void toString() {
return “not set”;
}

From now on you can use date anywhere in your class without the need to check for null.
Now lets look at a return value from a called method. First I would study the documentation of the method and see whether the method actually can return null objects. In important cases I would even go so far as to quickly write a bunch of tests. With those tests you have clear documentation if the method can return null and under which conditions.

public void doSometing() {
…
Date date = getDateOfTransaction(transaction);
…
}

Knowing, that date might be null we need to see how the variable is being used subsequently. If it is being used for some simple display purposes we can wrap the method and return a NullDate in case the return value is null. If there is some more serious work with the object down the road then ‘Houston we have a problem’! Creating a fake date will not work and invite some big risks. Bypassing the date accesses even more so:

…
doThis();
doThat();

if (date != null) {
rebate = calculateRebate(date);
bookTransaction(rebate);
}
…

Sure, we could move the bookTransaction-method out of the block and guarantee that it will be always booked. But, assuming that we are dealing with a preferred customer who always gets a 10% discount. In this case we will get bunch of calls to our customer service folks which would have to deal with an upset customer. We don’t want that. Depending on whether you have access to the source code of used method you can go in fix the problem if not then you have a problem which you should encapsulate. The encapsulated problem offers a clear separation between working and broken smelly workaround code. Also it adds a lot the readability of the source code and you would have only one place to go and do the fix.

But now let’s look at another piece of code. Assuming we have so transmit data via a selectable protocol. The transmission must be encrypted under specific circumstances. So the code could look like this

public void send(String text, String transmission, String encryption) {
String sendText = “”;

if (encryption.equals(“RSA128”)) {
encryption = new RSA128();
sendText = encryption.encrypt(text);
} else if (encryption.equals(“RSA1024”)) {
encryption = new RSA128();
sendText = encryption.encrypt(text);
} else { // no encryption
sendText = text;
}

if (transimission.equals(“ftp”)) {
FTPTransmitter transmitter = new FTPTransmitter();
transmitter.transmit(sendText);
} else if (transmission.equals(“ssh”)) {
SSHTransmitter transmitter = new SSHTransmitter();
transmitter.transmit(sendText);
}
}

How about this

public void send(String text, String transmission, String encryption) {
Transmitter transmitter = Factory.createTransmitter(transmission, encryption);

Transmitter.transmit(text);
}


First, we extracted a Transmitter interface or if feasible an abstract class. The transmitter internally uses a strategy pattern to encrypt the text. The encryption is injected during construction time of the Transmitter. Again, there is an abstraction for the encryption. The factory could be a own creation or you could use a framework like Spring.

class Factory {
public static Transmitter createTransmitter(String transmission, String encryption) {
Encryption encryptionObj = createEncryption(encryption);
return createTransmitter(transmission, encryptionObbj);
}

public static Transmitter createTransmitter(String transmission, Encryption encryption) {
Transmitter transmitter = createTransmitter(transmission);

transmitter.setEncryption(encryption);
return transmitter;
}
…
}

Sure, if you were following correctly I would expect a ‘but now we have the if-statements in the conveniently not shown createEnryption and createTransmitter method’. Yes, your are right. However, I consider those if-statements at this location to be essential. Also, if you use Spring the condition checking will be externalized into the bean descriptor file.
Also, since the factory is driven by Strings, these could easily come from the UI. In that regard the UI would drive the behaviour without adding any conditional checking and would be easily extendable to other means of transmission and encryption without the need to modify it. And yes, the code will much easier to test. MockObjects anyone ;)

PS If anyone can give a me hint to how show source code in a nice and readable using Blogger. This would be highly appreciated.

Thursday, May 28, 2009

XP and Design at the Agile Brazil 2009 conference

Where did the Design phase go? There is a misconception that XP does not promote design. In fact, it is quite the opposite. And I will be talking about it at the Agile Brazil 2009 conference in Rio de Janeiro, Brazil on June 27th.


I have been presenting on XP and Design, where I share my experience on how XP promotes continuous design improvement through Simple Design, Continuous Integration, Test Driven Development (TDD) and Refactoring.


This session was very successful when presented in other conferences (please see history of this session below), bringing the mixed audience (junior and senior developers, as well as coach, testers and managers) into what felt like a good conversation about XP।Similarly to XP’s Evolutionary design, this session has evolved into its current form. The feedback from every time the session has been presented was reincorporated as improvements. I don’t try to sell XP as the silver bullet for software development. Instead I talk about my experience in traditional and agile development approaches, and what I have seen working (or not working so well) in the projects I have been exposed to.

History of this Session

From the past three years, I have been giving training on Introduction to Agile, and Agile development practices, such as Test Driven Development, Continuous Integration and Refactoring.

In the beginning of 2007, I created the Introduction to Test Driven Development and the Continuous Integration sessions for internal training at ThoughtWorks.

In October 2007, I gave the Refactoring to Patterns – a practical look into the Agile approach on Evolutionary Design session at the IndicThreads 2007 conference, Pune, India.

In March 2008, I refactored the previous given presentation into its new form: Evolutionary Design through TDD and Refactoring practices. This session was presented with Kurman Karabukaev at the Google TestaPalooza 2008 conference. This session was awarded the conference’s Certificate of Excellence. The audience really liked the pair-presenting style during the session, and the discussion on different Agile (and non-Agile) experiences by the presenters.

In June 2008, the session evolved into the Agile Evolutionary Design for the Agile China 2008 conference. Once again the session was well received by the audience of 300+ people.

In October 2008, I invited Sudhindra Rao to improve the session and co-present in an upcoming conference. Sudhindra and I had been pair-presenting on ThoughtWorks internal Agile training. The session re-emerged as its current version: XP and Design - Where did the Design phase go? This session was given at the IndicThreads 2008 conference, Pune, India. This session was very successful, bringing the mixed audience (junior and senior developers, as well as coach, testers and managers) into what felt like a good conversation about XP and how we saw it working previously.

Speaking at home country

I am very excited to be speaking at my home country and present on one of my favorite topics: XP Design. Also it is an honor to be speaking next to amazing speakers and agilists such as Jason Yip, Adam Monago, Manoel Pimentel, Rodrigo de Toledo and Alisson Vale.

Monday, May 4, 2009

Three stages of the Daily Scrum

Lately I have been working in a large Agile program. With large Agile programs, comes common large team challenges, and the need to adapt the Agile development and management practices to better suit the large team challenges.

Currently the program has 180 people distributed over 12 teams. Each team is formed by 15 people (project manager, business analyst, quality assurance and developers; for short: PM, BAs, QAs and Devs). All teams are following agile practices such as 2 week Iterations, Daily Scrums and End of Iteration retrospectives.

In this blog entry I will describe how the daily scrum (or daily stand-up) meeting went through 3 different stages as the program grow from one team to 12 teams (of 15 members), adding up to 180 people on the program.

Stage 1: The Daily Scrum meeting

When the program was small (up to 2 teams of 15 people), each team had a daily scrum. The teams were getting good value out of the daily update. The three questions answered by each team member (what did you work on yesterday, what are you working on today, do you have any blocker?), and some informal catch up by the PMs, BAs, QAs and Devs of the two teams was enough to keep all program members in sync.

Stage 2: Scrum of Scrum with PMs and non-PMs

As the program was growing (consider now 5 teams of 15 people) the teams were still having the daily scrum meeting. However the individual teams started losing context on the status of the program, and important inter-teams updates and communications were not flowing as easily as when the overall program size was smaller.

We started having Scrum of Scrums after the teams’ daily scrum. The Scrum of Scrums (SoS for short) had the same format as the daily scrum: the participants would stand in a circle and would quickly give status update by answering a few questions. The questions were slightly different than the daily scrum questions. The SoS participants would answer the following 4 questions: What did your team do yesterday, What will your team do today, is anything blocking your team, is your team about to put anything in other team’s way?

The participants of the SoS were PMs and non-PMs: team member, such as BA, QA or Dev. The PMs presence was mandatory and they would drive the meeting (answer the 4 questions.) Each team had two representatives: the PM, and the non-PM team member. The PM would daily select the non-PM team member to attend the SoS (this was done in the team daily scrum). At times a team member would volunteer to come, as he/she had to give some important update to the overall program.

The non-PM SoS participants brought very important aspects to the SoS. First they would complement their PM with the extra detail (not too much detail, but enough for spawning the interest for the other program team members.) Second, they were effective radiators of information. They would bring information back from the SoS to their teams (at times the PM was pulled into other meetings prior to returning to their teams.) And last, the non-PM SoS participants made sure that the SoS did not become a PM update meeting.

Stage 3: Scrum of Scrum for PMs

The program was growing in size. Consider the phase where the program reached 12 teams of 15 people. The SoS with PMs and non-PMs became less effective. First, it was taking too long for 24 people to talk. Second, the SoS was becoming a mixed forum of discussion. The PMs wanted to go into further project management details, while the Devs wanted details on the development activities.

We changed the SoS format. At this stage, we broke it into several SoSs: The PM SoS, the Dev SoS and the QA SoS. In fact the PM SoS called was named SoS: Scrum of Scrums. The Devs and QAs meetings had other names such as Dev hurdle and QA catch up. The PMs kept the SoS in a daily basis; the other roles varied the frequency of their catch up meetings.

The SoS became a PM update, but it was an essential (and quick) PM update. Actually, the SoS still kept the stand-up in circle format, which prevented the PMs form giving long updates (specially after standing up in each team stand up, and then in the SoS). Also the PMs were still focusing on answering the original 4 questions of the SoS (described in stage 2).

The art of facilitating the SoS made an enormous difference. It was simple: whenever a PM would go into a detailed update (for example, I found a bug pattern in my team that …), or two or more PMs would repeat the same information (e.g. blocker: my team could not commit because the Continuous Integration server was down) the item would be added to the Parking Lot.

The Parking Lot items would be discussed at the end of the SoS. PMs that were not required could leave the meeting. If other people would be required, they would either be dragged into the meeting room, or a meeting would be scheduled later.

In summary, the described stages of the daily scrum worked for the program in different stages of the program. My main learning from this experience: try what works for your team, but do not hesitate on adapting.

Thursday, April 30, 2009

Agile Basic Linear Algebra

Releasing planning, Iteration planning, velocity and burn up charts are basic linear algebra.

From Wikipedia:
“Algebra is a part of mathematics (math) that helps show the general links between numbers and math operations (adding, subtracting, multiplying or dividing) used on the numbers. Algebra does this by using letters (a,b,c,...) or other symbols to represent numbers, either because the numbers are unknown or because the numbers change during the course of the problem, in which case the letters are called variables.”


I decided to write down the formulae commonly used for tracking and planning agile projects. After listing down the formulae, I will validate them by solving a few sample problems.

The Formulas

Vp = Ro / Ip
The planned average velocity (Vp) is the number of story points planned for the release (Ro) by the number of planned iterations (Ip).

V = ∑Si / i
The average velocity of a team (V) is the number of delivered story points per iteration (∑Si) by the number of iterations (i).

Ve = V + ∆V
Velocity expected is the average velocity (V ) plus the expected velocity variation (∆V). Velocity expected is commonly used for planning activities, such as iteration and release planning.

R = Ro + ∆R
The current total story points planned for the release (R) is the original story points planned for the release (Ro) plus the delta of story points for the release (∆R).

Rm = R - ∑Si
The number of story points remaining for the release (Rm ) is the current total Story points for the release (R ) minus the sum of completed story points for the iterations (∑Si).

Ir = Rm / Ve
The number of iterations remaining (Ir) is the number of story points remaining for the release (Rm ) by the expected velocity (Ve)

Applying the formulae
In case you miss your high school days, I recommend you to try out the questionnaire on Agile linear algebra. Find out how much you score before reading the solution below.

Problem 1
An agile team plans to deliver 400 story points in 10 iterations. What is the planned average velocity for the team?

Solution:
From the problem statement: Ro = 400 sp, and Ip = 10 it

Applying the formula Vp = Ro / Ip
Vp = 400 sp / 10 it = 40 st / it

Answer: The planned average velocity for the team is 40 story points per iteration

Problem 2
A team has average velocity of 10 sp/it. Extra resources are added to the team, thus creating an expected rise in velocity of 4 sp/it. What is the expected velocity for the team?

Solution:
From the problem statement: V = 10 sp/it, and ∆V = 4 sp/it

Applying the formula Ve = V + ∆V:
Ve = 10 sp/it + 4 sp/it = 14 sp/it

Answer: The expected team velocity is 14 sp/it

Problem 3
In Iteration 1, the team delivered 10 story points, in iteration 2 the team delivered 14 story points, in Iteration 3 the team delivered 12 story points. What is the average velocity for the team?

Solution
From the problem statement: ∑Si = (10 sp+ 14 sp+ 12 sp), and i = 3 it

Applying the formula V = ∑Si / i :
V = (10 sp+ 14 sp+ 12 sp) / 3 it = 36 sp / 3 it = 12 sp/it

Answer: The team average velocity is 12 sp/it

Problem 4
Consider the team iteration history of Problem 3. The original release plan is to deliver 400 story points. Assume that the team is about to start Iteration 4, and that the team is able to keep the same average velocity from the first 3 iteration for the remaining iterations of the release. The project manager decides to add an extra pair of developer to the team; and the velocity is expected to increase by 2 sp/it. How many iterations (in total) will it take for the team to deliver the 400 story points planned for the release?

Solution
From the problem statement: Ro = 400, ∆R = 0 (no scope as added or removed for the release), and V = 12 sp/it (verage velocity calculated in the previous problem), ∆V = 2 sp/it (a dev pair is added to the team and the velocity is expected to increase by 2 sp/it)

The Question: How many iterations (in total) will it take for the team to deliver the 400 story points planned for the release?
Total iterations = elapsed iterations + the number of iterations remaining (Ir)

So let’;s calculate Ir…
Ir = Rm / Ve

And
Ve = V + ∆V

Applying the Ve = V + ∆V:
V = 12 sp/it and ∆V = 2 sp/it
Ve = 12 sp/it + 2 sp/it
Ve = 14 sp / it

Now let’s calculate Rm
Rm = R - ∑Si
R = Ro + ∆R

Applying ∑Si = 36 sp, Ro = 400 sp and ∆R = 0 sp
Rm = Ro + ∆R - ∑Si
Rm = 400 sp + 0 sp – 36 sp
Rm = 364 sp

Now applyuing Rm and Ve to the Iterations remaining formula:
Ir = Rm / Ve
Ir = 364 sp / 14 sp / it
Ir = 26 it

Total iterations = elapsed iterations + the number of iterations remaining (Ir)
Total iterations = 3 it + 26 it
Total iterations = 29 it

Answer: It will take 29 iterations (in total) for the team to deliver the number of story points planned for the release.

Problem 5
Same as Problem 4, but 140 sp have been de-scoped from the release. How many more iterations will it take for the team to deliver the number of story points planned for the release?

Solution
From the problem statement: ∑Si = 36 sp, Ro = 400 sp and ∆R = -140 sp

Applying Rm = R - ∑Si and R = Ro + ∆R
Rm = Ro + ∆R - ∑Si
Rm = 400 sp - 140 sp – 36 sp
Rm = 224 sp

The number of story points remaining for the release (Rm ) is 224 sp

Now applying Rm and Ve to the Iterations remaining formula:
Ir = Rm / Ve
Ir = 224 sp / 14 sp / it
Ir = 16 it

Answer: Based in the presented assumptions, the team will need 16 iterations (in addition to the 3 elapsed iterations) to deliver the number of story points remaining for the release.

Problem 6
Same as Problem 4. How many story points should be de-scoped from the release to enable the software to be released after a total of 12 iterations?

Solution:
From the problem statement: ∑Si = 36 sp, Ro = 400 sp, Ir = 9 it (12 it – 3 it) and Ve = 14 sp/it

Applying the formulae Rm = R - ∑Si and R = Ro + ∆R
Rm = Ro + ∆R - ∑Si

Combining Ir = Rm / Ve, we come to:
Ir = (Ro + ∆R - ∑Si)/ Ve
Ro + ∆R - ∑Si = Ir. Ve
∆R = Ir. Ve – Ro + ∑Si

Applying the problem statement values:
∆R = 9it.14sp/it – 400sp + 36sp
∆R = 126it – 400sp + 36sp
∆R = 126it – 400sp + 36sp
∆R = -238sp

Answer: 238 story points should be de-scoped from the release to enable the software to be released after a total of 12 iterations.