SW development is empirical

One should learn from previous experiences and previous mistakes. Even better, one should learn from others experiences and mistakes. In SW development, we have been very good at adapting knowledge from other industries. This has been especially the case with the manufacturing industry. From the Taylorism to the more recently influences of Lean thinking, the SW development theories and practices have borrowed a lot from the manufacturing industry.

But we need to be very aware of the differences in our industries. Manufacturing typically deals with repeatable processes, while SW development is empirical.

In manufacturing, a repeatable process converts consistent inputs into consistent outputs. Repeatable means that the conversion of inputs to outputs can be replicated with little variation. In repeatable process, a small variation on inputs typically translates to a small variation to the outputs. But this is not the case with SW development.

SW development is really creation. And creation is not a highly predictable and repeatable thing. There is too much complexity and variability in the SW creation process; SW creation is an empirical activity.

Dictionary.com Empirical definition:
1. derived from or guided by experience or experiment.
2. depending upon experience or observation alone, without using scientific method or theory, esp. as in medicine.
3. provable or verifiable by experience or experiment.

In manufacturing, a great deal of work is about putting pieces together to build a specific thing. Whereas, in SW development, these pieces are often created, re-created, or customized every time. This is not to simplify the kind of work going on a manufacturing assembly line. My point is that the manufacturing discipline and its pieces of work are more mature, stable and well understood than the SW discipline, and its pieces of work.

Further, in SW development a small variation on inputs can result into a big variation to the outputs. For example, small changes in requirement or the architecture can result in huge differences in the software development and its output.

Are you Ready Ready

I’ve been to the Scrum Gathering in Munich this year. In his keynote Jeff Sutherland describes how he gets Scrum teams hyper productive. Essentially, you need to get your stories to Done Done as fast as possible. However, often too many unknowns do exist when a user story is being played. To fix this problem Jeff introduced, as on of his key ingredients, the concept of Ready Ready. Ready Ready removed disruptions and waste caused by issues being clarified with customer or others. He even suggested the use of a Ready Ready checklist to make sure that stories to be implemented do comply with the definition of Ready. Looks like the ‘Definition of Done’ gets a sibling -- ‘Definition of Ready’. The enforcement of DoR helped to increase the flow of stories to the anticipated state of Done Done and thereby increasing the productivity. Applying Ready Ready was core to create hyper productive Scrum teams. It came 2nd after 'Everyone must be trained in the Scrum framework'.

This is essentially what I have been doing or better trying to do in my last projects. However, the Ready Ready idea is very easy to explain and very convincing. Especially since it comes from Jeff Sutherland.

From now on my Product Backlog items will be Ready Ready before they can move into the Sprint Backlog.

The Agile manager is a master programmer

Lean thinking by Toyota says that managers must go and experience at the real place of work (gemba in Japanese) to learn what is going on. In the manufacturing field, gemba means the shop floor. When trying to apply Lean Thinking into my Agile management style, I feel like uploading the codebase from the repository into my computer and looking at the code; or even going further, I should be pairing with the programmers in my team.

The “real place” in software development is the crafting of the software. Therefore the Agile managers should be master programmers who can craft the code, and apply the development practices.

Of course not every master programmer will make a good Agile manager; and, on the other hand, not every traditional manager will make a good Agile manager. Either one will have to master several complementary skills that will empower a good Agile manager.

I have seen great traditional manager became great Agile managers, and great programmers became great Agile managers too. In both cases, the winner (the new Agile manager) did not stand alone. The whole team benefits at the end.

The manager who is new to Agile is to be empowered by the Agile team. And the programmer who is new to Agile is to be empowered by the team, including the Agile manager. Empower the team. Really. Don’t forget to empower the new-to-agile-manager. Please invite your manager to (once in a while) craft the software with you.

A new direction for the Scrum Alliance

September 15 2009, Ken Schwaber and Jim Cundiff resigned from the Board of Directors of the Scrum Alliance.

Below is an excerpt of the message

Message from the Board of Directors

On September 15, Ken Schwaber resigned as President and Chair of the Board of Directors of the Scrum Alliance. Tom Mellor was elected President and Chair of the Board by the Board members. Ken expressed to the Board his intentions to remain active in the Scrum community. The Board expresses its appreciation to Ken for his service and leadership.

Additionally, Jim Cundiff will no longer serve as the Managing Director of the Scrum Alliance effective immediately. Jim will support the transition of his duties to others in the organization and assist as needed until a new managing director is brought aboard. Jim has served the Scrum Alliance very effectively and capably and we thank him for his leadership. A search is underway for a new managing director. The Board anticipates filling the position in the very near future. Questions may be directed to Jodi Gibson at jgibson@scrumalliance.org

The resignation news was released one day after the news on the launch of the CSM Exam.

Launch of CSM Exam

As was referenced in a recent posting to the Certified Scrum Trainers (CST) site, the Scrum Alliance Board of Directors (Board) has de cided to move forward with the launch of the online Certified ScrumMaster (CSM) exam on October 1. The Board understands and appreciates that this decision has caused confusion and concern after the Alliance announc ed earlier in September that the launch of the test would be postponed…

Without much insight regarding the Scrum Alliance, it seems to me that the two events are related. Otherwise, it is absolute bad timing to give them one day apart from each other. Leaving the occurrence of the events aside, I believe we should be watching out for a few unanswered questions: Is the Scrum Alliance taking a new direction? How will this affect the Agile sphere?

Respect the elder bugs

Billy (below) is a known old bug. Old enough that it got a name and is well known to the development team.


Bugs, or defects in your software, are not to be treated as pets! Make sure you close bugs fast. Do not get used to bugs.

Agile software development (as typically is the case in the software industry) does create bugs. In fact I came across new Agilists that were surprised because Agile development practices also create bugs; in fact, I have seen more bugs (I am not talking about defects in production) in Agile development than in more traditional development.

But what I expect from a well-working Agile team is that:

1. The number of bugs in production is low


2. Bugs are closed very fast

Test Early and Often

Agile development practices such as continuous integration, test driven development, and automation facilitate the quick detection and elimination of bugs. Your customer tests (test validating the customer requirements) will detect any addition or change that breaks an existing test. Your developer tests (tests validating the design of your application) will detect anything that breaks the expected design and behavior of the application.

Disclaimer: for smaller teams, this will be as simple as identify and fix the recognized problem. But for very large teams, it might get a little more elaborated, and a bug is created to track the issue (at times, the issue itself is that there was no automated test).

Scaling Agile

Scaling Agile has a few nuances to it. And bug management and resolution is amongst the development practices that require special attention for a large Agile team. From my experience on a large Agile program, I second the importance to classify bugs by priority and severity.

Elisabeth Hendrickson wrote a very good article about it: Arguing Apples and Oranges.


But even with good bug classification and management, I noticed a few old (pet) bugs living in my large Agile program.

Consider the program had an average rate of 100 new bugs per week. Don’t be alarmed, it is a huge program. Imagine 100 developers continuously integrating code for 200 user stories per week. And an average rate of closing 100 bugs per week.

The depicted scenario can be of a very healthy Agile program. For instance, a small period of regression ensures the production readiness for the system. As Regression I mean the period allocated to stabilizing--fixing the bugs-- the next releasable artifact; not adding new features.

But how is this sample program creating old pet bugs?

Low priority, low severity bugs. These are the bugs that can live a long life.

If the reasoning for deciding the order to attack the bugs only considers severity and priority, you might be creating pet bugs. I did recognize a few elder pet bugs (for this reason) in my large agile program.

My recommendation is to adjust your bug triage formula. You should consider severity, priority, and the age of the bug.

I am sorry Billy (my old pet bug). Even though you were low priority and low severity, it is being embarrassing to have your small –almost inoffensive—existence in the system. I am sorry, but you have to go!

ThoughtBlogs in my e-reader

Since I bought my first e-reader (Amazon Kindle 1), I have reduced the number of printed reading material I carry around. Instead I have been carrying my kindle, which is full of reading content.

Today I was reading ThoughtBlogs in a café. I was off-line, away from my computer, and without a mobile phone. I only had my Kindle, and the wireless capability was off (I am in India and the Amazon kindle whispernet only works within the uSA).

For those with e-reader devices, here is how you can upload ThoughtBlogs content to it.

1. Download and install Calibre, a free, open source and cross-platform e-library solution with the Fetch News (RSS) feature.
   


2. Add ThoughtBlogs as a Calibre custom news source.
   

Below are the summarized steps on how to add ThoughtBlogs as a Calibre custom news source:

1. Fetch news / Add custom new source
   


2. Add/Update Recipe (ThoughtBlogs as the Recipe title)
   Feed Title: Planet TW
   Feed URL: http://blogs.thoughtworks.com/rss20.xml
   


3. On Fetch News, ThoughtBlogs should be listed at the Custom news source.
   

You can also check the detailed step-by-step instructions by Calibre.

Enjoy it!

The Retrospective of Retrospectives

Feedback is one of the XP values and key mechanism for all the agile methodologies. For example, Scrum teams religiously have Retrospective at the end of each iteration. This enables the team to reflect and adapt, improving the process for the next iteration. Agile Teams learn from their successes and failures. Retrospectives provide an effective way to incorporate feedback into the project.

But what about large Agile teams? Once again I will use my current large Agile program as a case study:

Lately I have been working on a large Agile program. With the large Agile program, comes common large team challenges, and the need to adapt the Agile development and management practices to better suit the large team challenges.

Currently the program has around 180 people distributed over 12 teams. Typically, each team is formed by 15 people (project manager, business analyst, quality assurance and developers; for short: PM, BAs, QAs and Devs). All teams in the program are following Agile practices, such as 2 week Iterations, Daily Scrums and End of Iteration retrospectives.

The teams were getting good value out of the Retrospective Agile practice. But the improvements were done in a team level. And the overall program was losing focus on continuous improvement. At times, the action point from one team’s retrospective was dependent on other teams. Other times, two team retrospective action points were contradictory.

Another challenge was that the program level management was not able to attend every team retrospective. There were 15 retrospectives happening at the end of each Iteration, The program manager (and other people) would like to attend each team retrospective; but it was just not possible to be at several meeting at the same time.

Retrospective of Retrospectives

Similarly to how the daily scrum is extended to the Scrum of Scrums for large Agile teams, I came up with the idea of having a Retrospective of Retrospectives for large Agile teams.

The Retrospective of Retrospectives (RoR for short) meeting is held after each individual team retrospective. The participants (representatives of all team and the program) meet to discuss what went well and what to improve in the program level. The RoR focus is on improving the overall program output, and not on individual team performance.

RoR format

Data gathering: The top 5 items from each team retrospective is the start point for discussions and possible program action items.

Participants: Similarly to the Scrum of Scrums, all managers participate in this meeting. The attendance of another representative of each team is recommended; and it is encouraged to be in a rotation basis.

Context: The context for the RoR is slightly different than the context for individual team retrospectives. While the context for a end or iteration retrospective might be: what can we do to improve next iteration for our team, The RoR context is: What can we do as one team (the whole program team) to improve the overall output for the program.

Mechanics: Each team retrospective top items are read. Repetitions and alike items are grouped together. Then the items go for voting. (for example 3 votes per participant).

RoR top benefits

Below are the top benefits I noticed from the RoR.

Quick access to all teams – all managers and stake holders have visibility on the top items for each individual team. And all is available in only one meeting.

Focus on improvement – for both the individual teams and the program level. One common problem on large agile projects is that one loses focus on the overall program by constantly focusing on local improvements. It is like being around the trees and lose sight of the forest.

Common goal – All managers (and the selected team members) are periodically talking and acting on the overall goal of the program. The RoR participants carry the common goal message (and action points) back to their teams.

Everyone is heard– Any local team problem will be heard by all the teams. Even if something is specific for a team, the message (team retrospective item) will be heard by all teams.