Agile is about Deliverables

I just came back from a wedding which took place in a small remote place in northern Bavaria, Germany. What does a wedding have to do with agile software development? Nothing at all, but quite a bit.

The bride and the groom are both living in Paris, France. She is from Germany and he from Algeria. The location of the wedding was chosen for one single reason. A relative of the bride´s family owns a hotel. This guaranted excellent service for a competitive price.

Ok, now I will elaborate about my epiphany I had there. But first here it is:

Agile is about deliverables

whereas

Classical processes are about completed work steps

What do I mean with this? In classical, Taylorism style driven workflows you are completing work steps in a long sequence. If you have 10 steps and are done with 8 you have 80% completed.

In agile it is about usable deliverables, value. It either exist and can be used or you better don't mention it. 0% or 100%.

Now, as you can see from the geographical distribution it was not straight forward for everyone to be there on time. Some people only had to drive, others took the plane or train. Many of them had a combination of all sorts of meaning of transport. Also, as it happened, there were snow storms all over Europe with tons of snow and really chilly weather. Those weather conditions caused quite some traffic havoc.

Nevertheless, every single person was there on time. Not one single one was late. They all were standing in the church when the bride and groom walked down the aisle.

They delivered. They delivered value by being there and paying their respect. I am sure many of them had quite a stressful adventure behind them but all of them only smiled at the couple.

The alternative would have been the following. Imagine the phone ringing and a conversation sounding somewhere like that: '... we are 90% there. In the beginning everything went according to plan but then there was an accident because of the weather ...' or '... we heard about the weather but we assumed everything will be fine, but now the airport is shut down for a couple of hours, we will arrive tomorrow. Guess this is ok?'

Don't track and communicate your progress in regards to your work steps but deliver value. This is the ONLY important thing!

It was a great wedding, mainly because everybody was there!

SW development is empirical

One should learn from previous experiences and previous mistakes. Even better, one should learn from others experiences and mistakes. In SW development, we have been very good at adapting knowledge from other industries. This has been especially the case with the manufacturing industry. From the Taylorism to the more recently influences of Lean thinking, the SW development theories and practices have borrowed a lot from the manufacturing industry.

But we need to be very aware of the differences in our industries. Manufacturing typically deals with repeatable processes, while SW development is empirical.

In manufacturing, a repeatable process converts consistent inputs into consistent outputs. Repeatable means that the conversion of inputs to outputs can be replicated with little variation. In repeatable process, a small variation on inputs typically translates to a small variation to the outputs. But this is not the case with SW development.

SW development is really creation. And creation is not a highly predictable and repeatable thing. There is too much complexity and variability in the SW creation process; SW creation is an empirical activity.

Dictionary.com Empirical definition:
1. derived from or guided by experience or experiment.
2. depending upon experience or observation alone, without using scientific method or theory, esp. as in medicine.
3. provable or verifiable by experience or experiment.

In manufacturing, a great deal of work is about putting pieces together to build a specific thing. Whereas, in SW development, these pieces are often created, re-created, or customized every time. This is not to simplify the kind of work going on a manufacturing assembly line. My point is that the manufacturing discipline and its pieces of work are more mature, stable and well understood than the SW discipline, and its pieces of work.

Further, in SW development a small variation on inputs can result into a big variation to the outputs. For example, small changes in requirement or the architecture can result in huge differences in the software development and its output.

Are you Ready Ready

I’ve been to the Scrum Gathering in Munich this year. In his keynote Jeff Sutherland describes how he gets Scrum teams hyper productive. Essentially, you need to get your stories to Done Done as fast as possible. However, often too many unknowns do exist when a user story is being played. To fix this problem Jeff introduced, as on of his key ingredients, the concept of Ready Ready. Ready Ready removed disruptions and waste caused by issues being clarified with customer or others. He even suggested the use of a Ready Ready checklist to make sure that stories to be implemented do comply with the definition of Ready. Looks like the ‘Definition of Done’ gets a sibling -- ‘Definition of Ready’. The enforcement of DoR helped to increase the flow of stories to the anticipated state of Done Done and thereby increasing the productivity. Applying Ready Ready was core to create hyper productive Scrum teams. It came 2nd after 'Everyone must be trained in the Scrum framework'.

This is essentially what I have been doing or better trying to do in my last projects. However, the Ready Ready idea is very easy to explain and very convincing. Especially since it comes from Jeff Sutherland.

From now on my Product Backlog items will be Ready Ready before they can move into the Sprint Backlog.

The Agile manager is a master programmer

Lean thinking by Toyota says that managers must go and experience at the real place of work (gemba in Japanese) to learn what is going on. In the manufacturing field, gemba means the shop floor. When trying to apply Lean Thinking into my Agile management style, I feel like uploading the codebase from the repository into my computer and looking at the code; or even going further, I should be pairing with the programmers in my team.

The “real place” in software development is the crafting of the software. Therefore the Agile managers should be master programmers who can craft the code, and apply the development practices.

Of course not every master programmer will make a good Agile manager; and, on the other hand, not every traditional manager will make a good Agile manager. Either one will have to master several complementary skills that will empower a good Agile manager.

I have seen great traditional manager became great Agile managers, and great programmers became great Agile managers too. In both cases, the winner (the new Agile manager) did not stand alone. The whole team benefits at the end.

The manager who is new to Agile is to be empowered by the Agile team. And the programmer who is new to Agile is to be empowered by the team, including the Agile manager. Empower the team. Really. Don’t forget to empower the new-to-agile-manager. Please invite your manager to (once in a while) craft the software with you.

A new direction for the Scrum Alliance

September 15 2009, Ken Schwaber and Jim Cundiff resigned from the Board of Directors of the Scrum Alliance.

Below is an excerpt of the message

Message from the Board of Directors

On September 15, Ken Schwaber resigned as President and Chair of the Board of Directors of the Scrum Alliance. Tom Mellor was elected President and Chair of the Board by the Board members. Ken expressed to the Board his intentions to remain active in the Scrum community. The Board expresses its appreciation to Ken for his service and leadership.

Additionally, Jim Cundiff will no longer serve as the Managing Director of the Scrum Alliance effective immediately. Jim will support the transition of his duties to others in the organization and assist as needed until a new managing director is brought aboard. Jim has served the Scrum Alliance very effectively and capably and we thank him for his leadership. A search is underway for a new managing director. The Board anticipates filling the position in the very near future. Questions may be directed to Jodi Gibson at jgibson@scrumalliance.org

The resignation news was released one day after the news on the launch of the CSM Exam.

Launch of CSM Exam

As was referenced in a recent posting to the Certified Scrum Trainers (CST) site, the Scrum Alliance Board of Directors (Board) has de cided to move forward with the launch of the online Certified ScrumMaster (CSM) exam on October 1. The Board understands and appreciates that this decision has caused confusion and concern after the Alliance announc ed earlier in September that the launch of the test would be postponed…

Without much insight regarding the Scrum Alliance, it seems to me that the two events are related. Otherwise, it is absolute bad timing to give them one day apart from each other. Leaving the occurrence of the events aside, I believe we should be watching out for a few unanswered questions: Is the Scrum Alliance taking a new direction? How will this affect the Agile sphere?

Respect the elder bugs

Billy (below) is a known old bug. Old enough that it got a name and is well known to the development team.


Bugs, or defects in your software, are not to be treated as pets! Make sure you close bugs fast. Do not get used to bugs.

Agile software development (as typically is the case in the software industry) does create bugs. In fact I came across new Agilists that were surprised because Agile development practices also create bugs; in fact, I have seen more bugs (I am not talking about defects in production) in Agile development than in more traditional development.

But what I expect from a well-working Agile team is that:

1. The number of bugs in production is low


2. Bugs are closed very fast

Test Early and Often

Agile development practices such as continuous integration, test driven development, and automation facilitate the quick detection and elimination of bugs. Your customer tests (test validating the customer requirements) will detect any addition or change that breaks an existing test. Your developer tests (tests validating the design of your application) will detect anything that breaks the expected design and behavior of the application.

Disclaimer: for smaller teams, this will be as simple as identify and fix the recognized problem. But for very large teams, it might get a little more elaborated, and a bug is created to track the issue (at times, the issue itself is that there was no automated test).

Scaling Agile

Scaling Agile has a few nuances to it. And bug management and resolution is amongst the development practices that require special attention for a large Agile team. From my experience on a large Agile program, I second the importance to classify bugs by priority and severity.

Elisabeth Hendrickson wrote a very good article about it: Arguing Apples and Oranges.


But even with good bug classification and management, I noticed a few old (pet) bugs living in my large Agile program.

Consider the program had an average rate of 100 new bugs per week. Don’t be alarmed, it is a huge program. Imagine 100 developers continuously integrating code for 200 user stories per week. And an average rate of closing 100 bugs per week.

The depicted scenario can be of a very healthy Agile program. For instance, a small period of regression ensures the production readiness for the system. As Regression I mean the period allocated to stabilizing--fixing the bugs-- the next releasable artifact; not adding new features.

But how is this sample program creating old pet bugs?

Low priority, low severity bugs. These are the bugs that can live a long life.

If the reasoning for deciding the order to attack the bugs only considers severity and priority, you might be creating pet bugs. I did recognize a few elder pet bugs (for this reason) in my large agile program.

My recommendation is to adjust your bug triage formula. You should consider severity, priority, and the age of the bug.

I am sorry Billy (my old pet bug). Even though you were low priority and low severity, it is being embarrassing to have your small –almost inoffensive—existence in the system. I am sorry, but you have to go!

ThoughtBlogs in my e-reader

Since I bought my first e-reader (Amazon Kindle 1), I have reduced the number of printed reading material I carry around. Instead I have been carrying my kindle, which is full of reading content.

Today I was reading ThoughtBlogs in a café. I was off-line, away from my computer, and without a mobile phone. I only had my Kindle, and the wireless capability was off (I am in India and the Amazon kindle whispernet only works within the uSA).

For those with e-reader devices, here is how you can upload ThoughtBlogs content to it.

1. Download and install Calibre, a free, open source and cross-platform e-library solution with the Fetch News (RSS) feature.
   


2. Add ThoughtBlogs as a Calibre custom news source.
   

Below are the summarized steps on how to add ThoughtBlogs as a Calibre custom news source:

1. Fetch news / Add custom new source
   


2. Add/Update Recipe (ThoughtBlogs as the Recipe title)
   Feed Title: Planet TW
   Feed URL: http://blogs.thoughtworks.com/rss20.xml
   


3. On Fetch News, ThoughtBlogs should be listed at the Custom news source.
   

You can also check the detailed step-by-step instructions by Calibre.

Enjoy it!

The Retrospective of Retrospectives

Feedback is one of the XP values and key mechanism for all the agile methodologies. For example, Scrum teams religiously have Retrospective at the end of each iteration. This enables the team to reflect and adapt, improving the process for the next iteration. Agile Teams learn from their successes and failures. Retrospectives provide an effective way to incorporate feedback into the project.

But what about large Agile teams? Once again I will use my current large Agile program as a case study:

Lately I have been working on a large Agile program. With the large Agile program, comes common large team challenges, and the need to adapt the Agile development and management practices to better suit the large team challenges.

Currently the program has around 180 people distributed over 12 teams. Typically, each team is formed by 15 people (project manager, business analyst, quality assurance and developers; for short: PM, BAs, QAs and Devs). All teams in the program are following Agile practices, such as 2 week Iterations, Daily Scrums and End of Iteration retrospectives.

The teams were getting good value out of the Retrospective Agile practice. But the improvements were done in a team level. And the overall program was losing focus on continuous improvement. At times, the action point from one team’s retrospective was dependent on other teams. Other times, two team retrospective action points were contradictory.

Another challenge was that the program level management was not able to attend every team retrospective. There were 15 retrospectives happening at the end of each Iteration, The program manager (and other people) would like to attend each team retrospective; but it was just not possible to be at several meeting at the same time.

Retrospective of Retrospectives

Similarly to how the daily scrum is extended to the Scrum of Scrums for large Agile teams, I came up with the idea of having a Retrospective of Retrospectives for large Agile teams.

The Retrospective of Retrospectives (RoR for short) meeting is held after each individual team retrospective. The participants (representatives of all team and the program) meet to discuss what went well and what to improve in the program level. The RoR focus is on improving the overall program output, and not on individual team performance.

RoR format

Data gathering: The top 5 items from each team retrospective is the start point for discussions and possible program action items.

Participants: Similarly to the Scrum of Scrums, all managers participate in this meeting. The attendance of another representative of each team is recommended; and it is encouraged to be in a rotation basis.

Context: The context for the RoR is slightly different than the context for individual team retrospectives. While the context for a end or iteration retrospective might be: what can we do to improve next iteration for our team, The RoR context is: What can we do as one team (the whole program team) to improve the overall output for the program.

Mechanics: Each team retrospective top items are read. Repetitions and alike items are grouped together. Then the items go for voting. (for example 3 votes per participant).

RoR top benefits

Below are the top benefits I noticed from the RoR.

Quick access to all teams – all managers and stake holders have visibility on the top items for each individual team. And all is available in only one meeting.

Focus on improvement – for both the individual teams and the program level. One common problem on large agile projects is that one loses focus on the overall program by constantly focusing on local improvements. It is like being around the trees and lose sight of the forest.

Common goal – All managers (and the selected team members) are periodically talking and acting on the overall goal of the program. The RoR participants carry the common goal message (and action points) back to their teams.

Everyone is heard– Any local team problem will be heard by all the teams. Even if something is specific for a team, the message (team retrospective item) will be heard by all teams.

Big Agile Wave in Brazil

2009 is the year that the Brazilian IT market has been taken by Agile. I have heard about successful projects developed in Brazil using XP since 2002. But IMO 2009 is the year that Agile became mainstream in Brazil.

The following are a few of the Agile conferences held in Brazil this year: Brazil Scrum Gathering (in May at São Paulo), Agile Brazil 2009 (in June at Rio de Janeiro), Maré de Agilidade (in August at Fortaleza) and --coming soon-- Ágiles 2009 (6th to 9th of October at Florianópolis) and Encontro Ágil 2009 (10th and 11th of October at São Paulo).

A few days ago I was at the Agile 2009 conference, Amongst several good memories, connections and learning, I was thrilled with the number of Brazilians speaking at the conference. And while chatting with renowned Agilists, Brazil is a common topic. They are excited about Brazil. The reason being is that they have been, are going soon, or are very interested in going and speaking at Brazil.






Don’t be surprised to find yourself soon in a major Agile conference in Brazil!

Mock Objects In Action at the Agile 2009 conference

How can mock objects help you design your system better? Want to know how mocking saved hours of work? We focus on establishing best practices based on examples with mock objects. We cover design of classes, using mock objects to understand and test interaction between objects of the system. By the end of the session it should become clear how mocking, when applied correctly helps with system design, improves testability by reducing cost of change. An explicit part of this session is dedicated to the Mocking top offenders. We talk with examples about bad usage of Mocks, and its consequences.

Sudhindra Rao and I will be presenting the Mock Objects in Action session at the Agile 2009 conference.

History of this session

I started conducting a session on mocking and testing when we was coaching a few developers. That session then turned into a training session conducted at ThoughtWorks internal training and was quite appreciated for its practicality. After a few additions to that sessions and adding mockito to the mix Sudhindra and I presented it at a couple of conferences. It was received with a lot of enthusiasm.

Recommended readings

Below is the list of recommended readings that I give to the session attendees. Please let me know what you would add to this list.

http://martinfowler.com/articles/mocksArentStubs.html
http://xunitpatterns.com/Mock%20Object.html
http://developer-in-test.blogspot.com/
http://misko.hevery.com/code-reviewers-guide/
http://codebetter.com/blogs/jeremy.miller/archive/2006/01/10/136407.aspx
http://www.jmock.org/oopsla2004.pdf


The demo code for this session will be available at: http://code.google.com/p/mocksamples/

Agile PM and Architect dialog

At times I get asked if I rather be an Agile PM, or a developer.

Today I had a quick conversation with the “Architect” which made me very proud of being an Agile PM.

So, the Architect (A) walked to my team table and started the short conversation…

A: “Who is the tech lead in this team?”
me: “all developers”
A: “But who makes the important design decisions?”
me: “everyone”

CI, structural changes and merges

Be mindful of the amount of structural changes while in the period that will require more code merge effort.

This was the message I sent to the programmers on a large agile team I have been working with.

Scaling agile has a few caveats to it. This blog post is an awareness message for programmer, which I believe to be complementary to the Continuous Integration practice for large teams.

Below is the message sent to the large Agile team.

We are a large team (+120 programmers: devs and QAs)

Programmers are coding everyday; despite of release candidates and regression periods. How is it possible?

We branch (prior to a new release), then we merge back (after it has been released); all the time ( release branch strategy )…

Consider the following simplified example of a Release formed by 2 dev iterations and 1 regression iteration.

Sample Project Release and Iteration plan

Dev activity –-It1—It2—It3—It4—It5—It6—It7—It8—It9--ItN--
Regression –------------Reg 1----Reg2-----Reg3—-------------
Release sch –------------------Rel1-----Rel2—---Rel3----------


So let’s look at different moments in time:

It2:
Dev activity –-It1—It2
Regression –----------
Release sch –---------

All programmers are working in the main trunk. Any regression activity is happening in the main trunk. There is no release yet. The release candidate is the main trunk itself.


It3:
Dev activity –-It1—It2—It3
Regression –-----------Reg 1
Release sch –-----------------

The release is formed by two dev iteration and one regression iteration.
In this case, the work done for Rel1 is related to It1 , It2, and Reg1; but should not be impacted by It3 development. Therefore the need to branch out the codebase: main trunk, and R1 branch.

Humm… if there is a branch, there will be a merge later: structural changes will have to be merged back!

It4:
Dev activity –-It1—It2—It3—It4
Regression –------------Reg 1----
Release sch –-----------------Rel1

R1 has been released. The code is merged back to the main trunk. The R1 branch now should have very little activity (production support only), therefore it should not create any difficult merge.
Programmers are mainly working in the main trunk; no merge required (remember, we do Continuous Integration!). Go crazy, do the structural changes that you feel are amazing (btw, if it is not, CI will catch it).

It5:
Dev activity –-It1—It2—It3—It4—It5
Regression –------------Reg 1----Reg2
Release sch –-----------------Rel1-----

The development period for R2 is over (It 3 and It 4). The regression work should not be impacted by the development work from It5. A branch for R2 has been created. If there is a branch, the code will have to be merged later on… Please be mindful of structural changes as you will have to merge them.

Have you noticed the pattern?

In the sample presented above:
It 2, it4, It6 development is happening in the main trunk which won’t require large amount of merges later on.

It 3, it5, It7 development is happening in the main trunk, but there is lots of development happening for a release, and those will have to be merged back in the next Iteration.

So if you have a big restructure to be done; should you do it in It7 or It8? (Remember, there is no free lunch, you will have to merge it back!)

Most if-statements are evil

I’ve really come to loath if-statements. It developed over time. At the very beginning they are great as they give you the power to put some logic into you application. Usually a small program in one of your first programming classes. Well, over time I realized the hard way that they clutter your code and make it harder to read and hence understand and maintain. In OO languages you have far more powerful ways to put logic into your application without using if’s. However, this requires a good skill in programming and lots of practice. I’ve come to realize that you can identify a good programmer by her if-avoidance skill. Well, after some time in acceptance but dislike I now try to avoid them at any price and so should you. By the way, I consider switch-statements to be nested if’s.

Frederick Brooks coined once the terms ‘Essential Complexity’ and ‘Accidental Complexity’. The first describes the real complexity which comes from the domain and the problem to solve. The second one is manmade i.e. it is the complexity we get into by our own making like which OS, which language, which framework, … we choose. Using this point of view, there are essential if-statements and accidental if-statements. The accidental ones, what a fitting name, are the ones which need to be eliminated.

Essential if’s
There are things you need to check. Can I save the file, is there enough free disk space.

if (diskHasEngoughFreeSpace()) {
document.save();
}

Those decision are driven from external forces and are out of our control and so we need to check them every time we save.

Accidental if’s
Those buggers creep up everywhere in you code.

if (date == null) {
date = new Date(); // this is called defensive programming, one of the worst things you can do
}

or

if (date != null) {
uiWidget.setText(date.toString());
} else {
uiWidget.setText(“”);
}

Before you get creative in finding a workaround you should analyze whether date actually can be null. If yes, how? Is it a field of the class, a parameter of the method, a return value from a called method. All these questions and their answers provide you with a multitude of options.

/**
* @param date of the document to be printed; must not be null
*/
public void printHeader(Date date) {
// Usually I put those checks into a utility class and a checkParam method.
// i.e. Check.checkNotNull(date, “date”);
if (date == null) throw new IllegalArgumentException(“invalid parameter: date cannot be null. Good bye!”);

// once we are here we know that all parameter are valid
date.doSomething();
…
}

It is a good practice to have an object to be in well pre-defined state after creation.

class DocumentPrinter {
private Date date; // this is the same as private Date date = null;
…
}

It is better to do it this way

class DocumentPrinter {
private Date date = Date();
// if this is not sufficient then have  the date passed in as parameter with the constructor and make the default constructor private
…
}

or even better use the Null-Object pattern

class Document Printer {
private Date date = new NullDate();
…
}

class NullDate extends Date {
public void toString() {
return “not set”;
}

From now on you can use date anywhere in your class without the need to check for null.
Now lets look at a return value from a called method. First I would study the documentation of the method and see whether the method actually can return null objects. In important cases I would even go so far as to quickly write a bunch of tests. With those tests you have clear documentation if the method can return null and under which conditions.

public void doSometing() {
…
Date date = getDateOfTransaction(transaction);
…
}

Knowing, that date might be null we need to see how the variable is being used subsequently. If it is being used for some simple display purposes we can wrap the method and return a NullDate in case the return value is null. If there is some more serious work with the object down the road then ‘Houston we have a problem’! Creating a fake date will not work and invite some big risks. Bypassing the date accesses even more so:

…
doThis();
doThat();

if (date != null) {
rebate = calculateRebate(date);
bookTransaction(rebate);
}
…

Sure, we could move the bookTransaction-method out of the block and guarantee that it will be always booked. But, assuming that we are dealing with a preferred customer who always gets a 10% discount. In this case we will get bunch of calls to our customer service folks which would have to deal with an upset customer. We don’t want that. Depending on whether you have access to the source code of used method you can go in fix the problem if not then you have a problem which you should encapsulate. The encapsulated problem offers a clear separation between working and broken smelly workaround code. Also it adds a lot the readability of the source code and you would have only one place to go and do the fix.

But now let’s look at another piece of code. Assuming we have so transmit data via a selectable protocol. The transmission must be encrypted under specific circumstances. So the code could look like this

public void send(String text, String transmission, String encryption) {
String sendText = “”;

if (encryption.equals(“RSA128”)) {
encryption = new RSA128();
sendText = encryption.encrypt(text);
} else if (encryption.equals(“RSA1024”)) {
encryption = new RSA128();
sendText = encryption.encrypt(text);
} else { // no encryption
sendText = text;
}

if (transimission.equals(“ftp”)) {
FTPTransmitter transmitter = new FTPTransmitter();
transmitter.transmit(sendText);
} else if (transmission.equals(“ssh”)) {
SSHTransmitter transmitter = new SSHTransmitter();
transmitter.transmit(sendText);
}
}

How about this

public void send(String text, String transmission, String encryption) {
Transmitter transmitter = Factory.createTransmitter(transmission, encryption);

Transmitter.transmit(text);
}

First, we extracted a Transmitter interface or if feasible an abstract class. The transmitter internally uses a strategy pattern to encrypt the text. The encryption is injected during construction time of the Transmitter. Again, there is an abstraction for the encryption. The factory could be a own creation or you could use a framework like Spring.

class Factory {
public static Transmitter createTransmitter(String transmission, String encryption) {
Encryption encryptionObj = createEncryption(encryption);
return createTransmitter(transmission, encryptionObbj);
}

public static Transmitter createTransmitter(String transmission, Encryption encryption) {
Transmitter transmitter = createTransmitter(transmission);

transmitter.setEncryption(encryption);
return transmitter;
}
…
}

Sure, if you were following correctly I would expect a ‘but now we have the if-statements in the conveniently not shown createEnryption and createTransmitter method’. Yes, your are right. However, I consider those if-statements at this location to be essential. Also, if you use Spring the condition checking will be externalized into the bean descriptor file.
Also, since the factory is driven by Strings, these could easily come from the UI. In that regard the UI would drive the behaviour without adding any conditional checking and would be easily extendable to other means of transmission and encryption without the need to modify it. And yes, the code will much easier to test. MockObjects anyone ;)

PS If anyone can give a me hint to how show source code in a nice and readable using Blogger. This would be highly appreciated.

XP and Design at the Agile Brazil 2009 conference

Where did the Design phase go? There is a misconception that XP does not promote design. In fact, it is quite the opposite. And I will be talking about it at the Agile Brazil 2009 conference in Rio de Janeiro, Brazil on June 27th.


I have been presenting on XP and Design, where I share my experience on how XP promotes continuous design improvement through Simple Design, Continuous Integration, Test Driven Development (TDD) and Refactoring.


This session was very successful when presented in other conferences (please see history of this session below), bringing the mixed audience (junior and senior developers, as well as coach, testers and managers) into what felt like a good conversation about XP।Similarly to XP’s Evolutionary design, this session has evolved into its current form. The feedback from every time the session has been presented was reincorporated as improvements. I don’t try to sell XP as the silver bullet for software development. Instead I talk about my experience in traditional and agile development approaches, and what I have seen working (or not working so well) in the projects I have been exposed to.

History of this Session

From the past three years, I have been giving training on Introduction to Agile, and Agile development practices, such as Test Driven Development, Continuous Integration and Refactoring.

In the beginning of 2007, I created the Introduction to Test Driven Development and the Continuous Integration sessions for internal training at ThoughtWorks.

In October 2007, I gave the Refactoring to Patterns – a practical look into the Agile approach on Evolutionary Design session at the IndicThreads 2007 conference, Pune, India.

In March 2008, I refactored the previous given presentation into its new form: Evolutionary Design through TDD and Refactoring practices. This session was presented with Kurman Karabukaev at the Google TestaPalooza 2008 conference. This session was awarded the conference’s Certificate of Excellence. The audience really liked the pair-presenting style during the session, and the discussion on different Agile (and non-Agile) experiences by the presenters.

In June 2008, the session evolved into the Agile Evolutionary Design for the Agile China 2008 conference. Once again the session was well received by the audience of 300+ people.

In October 2008, I invited Sudhindra Rao to improve the session and co-present in an upcoming conference. Sudhindra and I had been pair-presenting on ThoughtWorks internal Agile training. The session re-emerged as its current version: XP and Design - Where did the Design phase go? This session was given at the IndicThreads 2008 conference, Pune, India. This session was very successful, bringing the mixed audience (junior and senior developers, as well as coach, testers and managers) into what felt like a good conversation about XP and how we saw it working previously.

Speaking at home country

I am very excited to be speaking at my home country and present on one of my favorite topics: XP Design. Also it is an honor to be speaking next to amazing speakers and agilists such as Jason Yip, Adam Monago, Manoel Pimentel, Rodrigo de Toledo and Alisson Vale.

Three stages of the Daily Scrum

Lately I have been working in a large Agile program. With large Agile programs, comes common large team challenges, and the need to adapt the Agile development and management practices to better suit the large team challenges.

Currently the program has 180 people distributed over 12 teams. Each team is formed by 15 people (project manager, business analyst, quality assurance and developers; for short: PM, BAs, QAs and Devs). All teams are following agile practices such as 2 week Iterations, Daily Scrums and End of Iteration retrospectives.

In this blog entry I will describe how the daily scrum (or daily stand-up) meeting went through 3 different stages as the program grow from one team to 12 teams (of 15 members), adding up to 180 people on the program.

Stage 1: The Daily Scrum meeting

When the program was small (up to 2 teams of 15 people), each team had a daily scrum. The teams were getting good value out of the daily update. The three questions answered by each team member (what did you work on yesterday, what are you working on today, do you have any blocker?), and some informal catch up by the PMs, BAs, QAs and Devs of the two teams was enough to keep all program members in sync.

Stage 2: Scrum of Scrum with PMs and non-PMs

As the program was growing (consider now 5 teams of 15 people) the teams were still having the daily scrum meeting. However the individual teams started losing context on the status of the program, and important inter-teams updates and communications were not flowing as easily as when the overall program size was smaller.

We started having Scrum of Scrums after the teams’ daily scrum. The Scrum of Scrums (SoS for short) had the same format as the daily scrum: the participants would stand in a circle and would quickly give status update by answering a few questions. The questions were slightly different than the daily scrum questions. The SoS participants would answer the following 4 questions: What did your team do yesterday, What will your team do today, is anything blocking your team, is your team about to put anything in other team’s way?

The participants of the SoS were PMs and non-PMs: team member, such as BA, QA or Dev. The PMs presence was mandatory and they would drive the meeting (answer the 4 questions.) Each team had two representatives: the PM, and the non-PM team member. The PM would daily select the non-PM team member to attend the SoS (this was done in the team daily scrum). At times a team member would volunteer to come, as he/she had to give some important update to the overall program.

The non-PM SoS participants brought very important aspects to the SoS. First they would complement their PM with the extra detail (not too much detail, but enough for spawning the interest for the other program team members.) Second, they were effective radiators of information. They would bring information back from the SoS to their teams (at times the PM was pulled into other meetings prior to returning to their teams.) And last, the non-PM SoS participants made sure that the SoS did not become a PM update meeting.

Stage 3: Scrum of Scrum for PMs

The program was growing in size. Consider the phase where the program reached 12 teams of 15 people. The SoS with PMs and non-PMs became less effective. First, it was taking too long for 24 people to talk. Second, the SoS was becoming a mixed forum of discussion. The PMs wanted to go into further project management details, while the Devs wanted details on the development activities.

We changed the SoS format. At this stage, we broke it into several SoSs: The PM SoS, the Dev SoS and the QA SoS. In fact the PM SoS called was named SoS: Scrum of Scrums. The Devs and QAs meetings had other names such as Dev hurdle and QA catch up. The PMs kept the SoS in a daily basis; the other roles varied the frequency of their catch up meetings.

The SoS became a PM update, but it was an essential (and quick) PM update. Actually, the SoS still kept the stand-up in circle format, which prevented the PMs form giving long updates (specially after standing up in each team stand up, and then in the SoS). Also the PMs were still focusing on answering the original 4 questions of the SoS (described in stage 2).

The art of facilitating the SoS made an enormous difference. It was simple: whenever a PM would go into a detailed update (for example, I found a bug pattern in my team that …), or two or more PMs would repeat the same information (e.g. blocker: my team could not commit because the Continuous Integration server was down) the item would be added to the Parking Lot.

The Parking Lot items would be discussed at the end of the SoS. PMs that were not required could leave the meeting. If other people would be required, they would either be dragged into the meeting room, or a meeting would be scheduled later.

In summary, the described stages of the daily scrum worked for the program in different stages of the program. My main learning from this experience: try what works for your team, but do not hesitate on adapting.

Agile Basic Linear Algebra

Releasing planning, Iteration planning, velocity and burn up charts are basic linear algebra.

From Wikipedia:

“Algebra is a part of mathematics (math) that helps show the general links between numbers and math operations (adding, subtracting, multiplying or dividing) used on the numbers. Algebra does this by using letters (a,b,c,...) or other symbols to represent numbers, either because the numbers are unknown or because the numbers change during the course of the problem, in which case the letters are called variables.”

I decided to write down the formulae commonly used for tracking and planning agile projects. After listing down the formulae, I will validate them by solving a few sample problems.

The Formulas

Vp = Ro / Ip
The planned average velocity (Vp) is the number of story points planned for the release (Ro) by the number of planned iterations (Ip).

V = ∑Si / i
The average velocity of a team (V) is the number of delivered story points per iteration (∑Si) by the number of iterations (i).

Ve = V + ∆V
Velocity expected is the average velocity (V ) plus the expected velocity variation (∆V). Velocity expected is commonly used for planning activities, such as iteration and release planning.

R = Ro + ∆R
The current total story points planned for the release (R) is the original story points planned for the release (Ro) plus the delta of story points for the release (∆R).

Rm = R - ∑Si
The number of story points remaining for the release (Rm ) is the current total Story points for the release (R ) minus the sum of completed story points for the iterations (∑Si).

Ir = Rm / Ve
The number of iterations remaining (Ir) is the number of story points remaining for the release (Rm ) by the expected velocity (Ve)

Applying the formulae
In case you miss your high school days, I recommend you to try out the questionnaire on Agile linear algebra. Find out how much you score before reading the solution below.

Problem 1
An agile team plans to deliver 400 story points in 10 iterations. What is the planned average velocity for the team?

Solution:
From the problem statement: Ro = 400 sp, and Ip = 10 it

Applying the formula Vp = Ro / Ip
Vp = 400 sp / 10 it = 40 st / it

Answer: The planned average velocity for the team is 40 story points per iteration

Problem 2
A team has average velocity of 10 sp/it. Extra resources are added to the team, thus creating an expected rise in velocity of 4 sp/it. What is the expected velocity for the team?

Solution:
From the problem statement: V = 10 sp/it, and ∆V = 4 sp/it

Applying the formula Ve = V + ∆V:
Ve = 10 sp/it + 4 sp/it = 14 sp/it

Answer: The expected team velocity is 14 sp/it

Problem 3
In Iteration 1, the team delivered 10 story points, in iteration 2 the team delivered 14 story points, in Iteration 3 the team delivered 12 story points. What is the average velocity for the team?

Solution
From the problem statement: ∑Si = (10 sp+ 14 sp+ 12 sp), and i = 3 it

Applying the formula V = ∑Si / i :
V = (10 sp+ 14 sp+ 12 sp) / 3 it = 36 sp / 3 it = 12 sp/it

Answer: The team average velocity is 12 sp/it

Problem 4
Consider the team iteration history of Problem 3. The original release plan is to deliver 400 story points. Assume that the team is about to start Iteration 4, and that the team is able to keep the same average velocity from the first 3 iteration for the remaining iterations of the release. The project manager decides to add an extra pair of developer to the team; and the velocity is expected to increase by 2 sp/it. How many iterations (in total) will it take for the team to deliver the 400 story points planned for the release?

Solution
From the problem statement: Ro = 400, ∆R = 0 (no scope as added or removed for the release), and V = 12 sp/it (verage velocity calculated in the previous problem), ∆V = 2 sp/it (a dev pair is added to the team and the velocity is expected to increase by 2 sp/it)

The Question: How many iterations (in total) will it take for the team to deliver the 400 story points planned for the release?
Total iterations = elapsed iterations + the number of iterations remaining (Ir)

So let’;s calculate Ir…
Ir = Rm / Ve

And
Ve = V + ∆V

Applying the Ve = V + ∆V:
V = 12 sp/it and ∆V = 2 sp/it
Ve = 12 sp/it + 2 sp/it
Ve = 14 sp / it

Now let’s calculate Rm
Rm = R - ∑Si
R = Ro + ∆R

Applying ∑Si = 36 sp, Ro = 400 sp and ∆R = 0 sp
Rm = Ro + ∆R - ∑Si
Rm = 400 sp + 0 sp – 36 sp
Rm = 364 sp

Now applyuing Rm and Ve to the Iterations remaining formula:
Ir = Rm / Ve
Ir = 364 sp / 14 sp / it
Ir = 26 it

Total iterations = elapsed iterations + the number of iterations remaining (Ir)
Total iterations = 3 it + 26 it
Total iterations = 29 it

Answer: It will take 29 iterations (in total) for the team to deliver the number of story points planned for the release.

Problem 5
Same as Problem 4, but 140 sp have been de-scoped from the release. How many more iterations will it take for the team to deliver the number of story points planned for the release?

Solution
From the problem statement: ∑Si = 36 sp, Ro = 400 sp and ∆R = -140 sp

Applying Rm = R - ∑Si and R = Ro + ∆R
Rm = Ro + ∆R - ∑Si
Rm = 400 sp - 140 sp – 36 sp
Rm = 224 sp

The number of story points remaining for the release (Rm ) is 224 sp

Now applying Rm and Ve to the Iterations remaining formula:
Ir = Rm / Ve
Ir = 224 sp / 14 sp / it
Ir = 16 it

Answer: Based in the presented assumptions, the team will need 16 iterations (in addition to the 3 elapsed iterations) to deliver the number of story points remaining for the release.

Problem 6
Same as Problem 4. How many story points should be de-scoped from the release to enable the software to be released after a total of 12 iterations?

Solution:
From the problem statement: ∑Si = 36 sp, Ro = 400 sp, Ir = 9 it (12 it – 3 it) and Ve = 14 sp/it

Applying the formulae Rm = R - ∑Si and R = Ro + ∆R
Rm = Ro + ∆R - ∑Si

Combining Ir = Rm / Ve, we come to:
Ir = (Ro + ∆R - ∑Si)/ Ve
Ro + ∆R - ∑Si = Ir. Ve
∆R = Ir. Ve – Ro + ∑Si

Applying the problem statement values:
∆R = 9it.14sp/it – 400sp + 36sp
∆R = 126it – 400sp + 36sp
∆R = 126it – 400sp + 36sp
∆R = -238sp

Answer: 238 story points should be de-scoped from the release to enable the software to be released after a total of 12 iterations.