juin 2010Monthly :

There has been a debate in the Scrum and Kanban communities about having iterations  (sprints) or not. I am worried that this blog post will generate flame wars and rants. Thus, there will certainly be some energy that will be lost. My hope is that this post will generate real debates and discussions so we can learn from each other’s opinion.

I have been developing software in Scrum for a long time and coaching many teams and organizations adopting Scrum. Therefore, I have been exposed to a lot of situations and feel I have integrated the fundamentals and the theoretical foundations of Scrum.

My general feeling, which you will see expressed throughout this blog post is that the Agile community is falling into the trap of looking for optimizations everywhere and is losing sight of some fundamentals about complexity, creativity, teamwork and commitments.

When I first heard about Kanban, I was intrigued and read about it and even applied it in some situations I felt it could be helpful. There are a couple of nice things that Kanban brings to the table but I also think that it breaks some fundamental things that make Scrum work.

Within sprints, Scrum suggests a simple workflow with sprint backlog items going from ‘To Do’ to ‘In Progress’ to ‘Done’. I have certainly seen some Scrum teams have way too much work ‘In Progress’ and using Kanban techniques to limit the amount of work in progress can certainly help. I also do not think it is necessarily a bad idea that a mature team establishes a more defined workflow and uses Kanban techniques to control its flow of work but going too far (I have seen a Kanban board with 10 columns corresponding to stories’ statuses) in that direction will reduce the possibilities of emergence that creates true performance in self-managing, multidisciplinary teams.

Getting to the actual debate of having sprints or not. Some Scrum proponents say that not having sprints may be problematic because the team needs to hold regular retrospectives to accelerate learning. While I do agree that holding regular retrospectives is absolutely essential, I think that a Kanban team could do regular retrospectives while not completely applying sprints.

I think Ken Schwaber has a much stronger point. In his Waterfall, Lean/Kanban, and Scrum blog post, he presents sprints from the point of view of the complexity theory.

A container is a closed space where things can get done, regardless of the overall complexity of the problem. In the case of Scrum, a container is a Sprint, an iteration. We put people with all the skills needed to solve the problem in the container. We put the highest value problems to be solved into the container. Then we protect the container from any outside disturbances while the people attempt to bring the problem to a solution. We control the container by time-boxing the length of time that we allow the problem to be worked on. We let the people select problems of a size that can be brought to fruition during the time-box. At the end of the time-box, we open the container and inspect the results. We then reset the container (adaptation) for the next time-box. By frequently re-planning and shifting our work, we are able to optimize value.

Vincent also brings an interesting viewpoint in his recent post Scrum is not about project management

.

While explaining the notion of container, Ken mentions above: “We put the highest value problems to be solved into the container.” I would like to push this a little further and relate it to planning and commitment. I have always insisted in my Scrum classes that a successful sprint planning is not about delivering a sprint backlog, it is first and foremost about having a team committed toward a goal that is compelling for them as a whole including the ScrumMaster and Product Owner. I believe, this is one of the fundamentals to create creative hyper-performing, self-managing teams that can sustain.

I have felt during the last few years that as a community we are putting too much focus on the concept of velocity and, therefore, many teams are un-passionately identifying their commitment based on their velocity and do not get to the true nature of what it means to be committed toward a compelling goal.

Before you throw tomatoes at me, I am not saying that measuring velocity is useless. I am saying that while it is useful for a team to measure and be aware of its velocity, I think we let it drive too much the commitment decisions of the team. Some tools are in my opinion putting too much emphasis on using velocity to drive the sprint planning process.

This belief of the importance of being committed toward a compelling goal was reinforced recently while reading the following book: The three laws of performance. Here are the three laws presented in the book:

1. How people perform correlates to how situations occur to them.
2. How a situation occurs arises in language.
3. Future-based language transforms how situation occurs to people.

I will not try to summarize the book here. I thought it is useful to mention this reference because I think it links the importance of creating an environment in planning sessions that enables the team to choose a goal that is compelling for them to some fundamentals of human beings.

In summary, I suggest to use sprints as defined in Scrum because when done in the true spirit of Scrum, they enable a team to look at the highest value problems, imagine a compelling future, and use all of the thinking, collaboration, and creativity possible to put together solutions and plans. Then, you leave the people alone within the container of the sprint to apply their professional skills, without interruption so they can concentrate and focus on their work. This is the core of them being creative people doing creative work rather than resources being managed to optimize productivity.

Warning: The following post might provoke a mild aneurism to some scrum purists.

Don’t you love a nice, clean and deep Product Backlog, filled only with strong user stories tightly linked to clear business objectives and estimated by business value and story points?  As an Agile coach working with large organization with even larger challenges, I don’t usually have that luxury. And If I do, I don’t believe it!  The amount of stuff that needs to be done to ship out quality software is astronomical and never ceases to amaze me. So of course the “Definition of DONE” is equally immense.

The definition of DONE

Creating and shipping software in large organizations involves tasks and deliverables that go beyond straightforward testing, coding and releasing.  To get all this work out into the open, I ask teams to focus on a couple of “juicy” user stories and ask them to identify ALL that needs to be completed to get this product in front of the user.  We might get something like this:

And the list goes on and on…

Teams are then challenged to get everything done within a user story.   Of course this is impossible.  The best the team (and the supporting organization) can do for now is to complete some DONE items at the sprint, milestone, release or project level (see fig.1)

Figure 1

The further away we are from the user story level, the more nervous I get. At the beginning of an Agile transition, I can live with that as long as we identify these “non-story DONEs” as debt and we manage it appropriately.  I don’t judge or question any of the DONE items (ok, maybe I do, but not out loud) But I do want to the team and the organization to clearly see the sheer volume of overhead and offer them some kind of tool to start cutting out the fat.

This tool will be…The Product Backlog.

The team will manage this debt by adding non-functional work items in the Product Backlog.  In other words, all DONE items not included within a user story will become a work item in the Backlog. If the team’s initial estimate is 4 sprints for a release, then those sprint-level DONE items will appear four times. This continues with milestone, release and project DONE items. This makes for one polluted Backlog – And that’s ok…For now!  As you can imagine, this can double the scope of the project.

A product backlog can go from this:

To this:

What does it mean?

Based on the team’s velocity, it’ll probably take 4 to 5 sprints complete the project and not 2 – That is, if nothing changes. It means that it won’t take 15 points to produce $26,000 in business value but at least 33. Nasty!

It can also show an organization that went through a chaotic period and decided to structure all that is I.T. into a defined process; a normal reaction. Over design and over documentation is heavy and expensive, but it’s still better than the anarchy of the early years.

Finally, it means we need find the correct balance for this project and organization.

What do we do?

If the organization is polluted, so shall be the Product Backlog – Deal with it! No really, you need to deal with it.

The Product Backlog can’t remain in that state. It’s needs to be filled, almost exclusively, with items that deliver business value.  That said, don’t hide the real work that currently needs to be done or simply take that work into consideration by inflating story points.  It needs to be out there, for all to see.  We want to provoke a sense of urgency and change.

All the $0 business value items are now action items, things that a Scrum team and the surrounding organization need to deal with now.  If we can’t eliminate an item all together (Ex.: Code document? – I mean come on!), then we need to find out how we can reduce the effort needed to get it done.  We need to reduce waste and those dollars signs might motivate those upper management folks to get things moving.

This approach is provocative but in many large organizations, shock and awe is often required to encourage change.  If a Product Owner and ScrumMaster can clearly show that it costs $1 to create $0.25 of value, I promise you that something will happen.

So between a polluted Product Backlog and a clean one that doesn’t show us the full picture, I choose the former…For now!

Once again the team has committed to a new release Monday morning.

This is the plan for the next trip !

In the first sprint, we will make sure that the Turtle can sprint with the new Team Foundation Server Scrum v1.0 template announced at TechEd on Monday. You can download the new template here. We are really excited that Microsoft has decided to jump in the scrum world!

In the second sprint, we will implement a natural way to filter areas and iterations using a tag concept. You will have the option to put a push pin on some areas and iterations to apply a filter based on those selected and work with a subset of the work items. This will simplify backlog visualization and make sure the team’s focus is on delivering awesome software sprint after sprint.

This will help you doing enterprise scrum and complex project management with the Turtle.

If you want to manage your projects like one big project as suggested by Martin Hinshelwood on his blog, this feature will let you do that with Urban Turtle. I think you will really like this new feature.

Send us your comments on the forums!

Dominic !

It’s almost T-minus zero for the 2010 Agile Coach Camp! This open space forum for Agile coaches will be held on June 11^th and 12^th in Waterloo Canada and will be a great opportunity to see how our profession is evolving and practiced in the field.

Being in an open space format (which I love) we’ll all have the opportunity to expose our own views and experiences and of course learn from fellow practitioners. Looking at all the position papers, I don’t think we’ll run out of topics!

That said, are there any Agile coaching subjects YOU feel passionate about and consider important to bring up during this event? If so, feel free to propose it and it’ll be my pleasure to add it to the queue!

I’m looking forward to sharing my experience in my next post!

Release plan

Durant le sprint 0, une seule epic avait été estimé. Alors une première version d’un release plan a été fait avec une hypothèse farfelue. Elle est farfelue, mais avec l’information que nous avons à ce point du projet, c’est la meilleure que j’ai trouvée.  Mon hypothèse est que toutes les epic ont la même complexité (en considérant que même s’il y en a des plus petites, il y en aurait des plus grosses, donc le tout s’équilibrerait). Dans notre cas, la somme des valeurs des stories qui composent l’epic est 31 points. En faisant des calculs savants de conversion de points en heures selon le découpage en tâches des stories, ça mène à une date de fin potentielle à la mi-juin 2011. Nous allons voir avec le temps si mon hypothèse tient moindrement la route…

Une livraison “pilote” dans un certain nombre de magasins (1 à 3, le nombre reste à déterminer) est prévu pour la fin de l’année 2010. Alors une deuxième itération sur le release plan est nécessaire pour vérifier ce qui peut être réalisé pour le pilote. Une nouvelle estimation des epic est faite en les comparant à la première (celle-ci a été développée durant la première itération). Pour faire la ré estimation, la première epic a été considérée comme un 5, ensuite les autres ont été estimée selon l’échelle de presque Fibonacci habituelle. En refaisant encore des calculs savants, le release plan a été révisé pour mener la fin potentielle du projet vers la mi-mai 2011*, et anticiper que le pilote est réalisable pour le début décembre. Il faudra sûrement jouer sur l’étendue de l’epic, c’est à dire sur les éléments qui composent les epic “essentielles” pourront être livrées pour le pilote.

* l’hypothèse farfelue du départ ne semblait pas être si farfelue puisque la date de fin n’a bougée que d’un mois, ce qui ne me semble pas significatif comme changement.

Pour terminer, nous avons mis sur une échelle du temps la date où certains dépendances externes (interactions avec des systèmes externes) devraient être abordées/mis en place, ceci afin de ne pas risquer les dates anticipées.

Backlog maintenance

Le PO a rédigé quelques stories et bougé des priorités dans le product backlog. L’équipe doit maintenant estimer ces nouvelles stories. Nous avons 4 stories d’estimées (3 qui ont été développées durant la première itération, et 1 non amorcée) qui serviront de base de référence pour les nouvelles estimations.

Je laisse l’équipe estimer 4 stories avant de leur faire part d’une inquiétude. La première itération contenait 6 points. Les nouvelles stories estimées sont de 5, 13 ou 20 points. En principe, ces stories ne pourront donc pas être réalisées durant une itération puisqu’elle dépasse la capacité de l’équipe. On arrête pour une quinzaine de minutes la maintenance pour discuter de la taille des stories et voir si elles peuvent être découpées en plus petites. Le PO dit que les stories ne peuvent pas être découpées plus parce que (par exemple) “une commande ne peut pas être exécutée complètement si on n’a pas les SKU et les items”. D’accord! On discute pour voir si on peut quand même le découper en 2, question de diminuer le risque. Pas certain. Pour ne pas détourner l’objectif du backlog maintenance trop longtemps, la discussion sur le découpage des stories est reportée. On continue à estimer des stories restantes. Même si on n’a pas complètement conclu le sujet, la discussion a portée fruit; on se question si on peut découper les stories suivantes lorsqu’elles semblent un 5 ou plus. Aussi, une nouvelle story est rédigée pour traiter un cas plus spécifique, réduisant ainsi la taille d’une story du PO.

Il y aura de travail à faire avec le PO pour qu’il écrive plus de stories qui sont réalisables dans une itération. Il pense qu’il y a assez de travail pour occuper l’équipe pendant plusieurs itérations, alors ce n’est pas nécessaire d’ajouter des stories. La logique n’est pas fausse, mais la faille est que le travail semble trop gros pour être réalisé dans 1 itération. Si les stories restent de cette taille, elles ne seront que partiellement développée durant une itération. L’équipe s’est d’ailleurs questionnée sur la durée des itérations (présentement 2 semaines). Nous allons probablement (si l’équipe veut aborder le sujet!) en rediscuter durant la rétro d’itération.

Last week Pyxis hosted an intensive three days immersion into Greg Young’s Applied DDD course. This course is amazingly packed with tips and tricks on how to come up with an architecture that delivers business value customers have come to expect from their investments in software systems. Powered by Greg’s energetic tone, this course challenged my assumptions of how software should be designed and I would recommend it to anyone interested in architecture innovations surrounding Domain Driven Design. Following is an overview of the topics we’ve tackled during the training.

DDD recap

The first day was spent refreshing some of the fundamental but often misunderstood concepts of Domain Driven Design. We were reminded that Strategic Design is essential while embarking on a Domain Driven Design project as we must make sure that the value we get out from it is well worth the effort. Notions such as Bounded Contexts and Core Domain allow highly knowledgeable teams to focus on value added development work while leaving less crucial aspects of the software to others.

Another topic that stroke a chord with me was Greg’s presentation of Aggregates and the role the play in well crafted software systems. Aggregates exist to provide transaction boundaries around the behaviors of domain objects. Aggregate roots provide the sole entry point to this grouping of domain behaviors and encapsulate validation logic as well as invariant consistency inside the aggregate.

CQRS

We spent the second day learning about Command Query Responsibility Segregation and Event Sourcing. CQRS is different then Command Query Separation (Bertrand Meyer) which advocates that methods that change state should be separated from functions that return value. CQRS takes the same principle and applies it to the hole architecture. In this style of design some objects (write model) represents the commands the domain respond to and other objects (read model) handles the queries and their results. Having separate models for these concerns brings a lot of benefits that I will bring up in later posts.

The write model

It’s common OO knowledge that objects should be designed around behaviors in a domain. Having a dedicated write model consisting of aggregates that define transaction boundaries around the behaviors in the domain focuses work on the most valuable part of the system. This is the place to maximize the return on the work of your most skilled developers aided by domain experts.

The read model

The query side doesn’t have a domain as it’s mostly straightforward boilerplate code. The preferred implementation of a read model is having request DTOs going through a thin query layer that projects directly off of a data source. This is way simpler that using the domain model to do the same. This way pre-fetched paths and optimizing ORMs queries are avoided.

Event Sourcing

Event sourcing can be seen as a way to “standardize” thus reduce the cost of implementing separated read and write models. The basic idea is that all domain can be modeled as receiving commands that and producing associated events. These events can be stored as a log of what happened in the system and also be used to update the read model.

While exposing concurrency issues and ways to handle them Greg showed us a merging algorithm that’s quite simple and clever. Probably, the only piece of code Greg would ever write with a Goto statement as you can see in the picture.

Fine tuning the architecture

The third day was spent analyzing the properties of this architecture and how it could be fine tuned to produce low latency, high availability and high reliability systems. Most notably CQRS architecture can used to get around CAP theorem issues which states that you can’t guarantee all three Consistency, Availability and Partitionability in one system. Look here for another treatment on how to get around CAP.

All these architecture talks made me realize that using DDD and CQRS is a lot of work. And it’s no wonder that they should be applied to core domains. Now most companies I’ve worked with deal with multiple domains such as Sales, Security, Insurance, etc… And it would have been difficult to single out the one that constitutes their core domain. But every company must have a core domain otherwise it would not have a competitive advantage. Turns out a lot of companies derive their competitive advantage from their business process that integrate all of the domains they deal with. This is the place their core domain reside and that’s where Sagas help.

Pyxis source of high quality technical learning

All in all, these three days were packed with useful technical knowledge and eye opening discussions with an expert in his craft. The participants enjoyed the learning experience in Pyxis’ agile environment where they got a close up look at our facilities supporting agile teams. Pyxis is proud to have sponsored this event and is dedicated to bringing high quality technical training to the public.

Other technical trainings :

• Professional Scrum Developer (.NET)
• Agile Modeling
• Domain-Driven Design (DDD)
• Test-Driven Development (TDD)
• Usability Practices
• Executable Specifications with GreenPepper

Like most of you, I’ve had my fair (or unfair) share of meetings.  Some were great and a lot of them were just ok. But a few weeks ago a colleague of mine said something during a ROTI evaluation that forced me to look back at all those previous “great” meetings and re-evaluate their TRUE value.

What he said was: “Am I my really obtaining value from this activity or am I just having a good time?”

My jaw dropped!

How can such a simple statement stir up so many issues in my mind? It not only forced me to re-evaluate the specific activity he was referring to, but everything else I’d done and was about to do!

Leaving aside the fact that fun has value in itself, why not challenge an initial assessment with a Fun vs. Value Evaluation?  A meeting might have been fun, but did we reach our objectives?  As an Agile coach (or ScrumMaster), if I suspect a misinterpretation of the true value of a meeting or an activity, I might try to validate it by whipping out a Fun/Value graph:

No fun and No value

If the meeting had low value, there’s a good chance that fun is also on the low side.  Those situations are usually easy to detect and numerous corrective actions can be put in place.

No Fun and High Value

Now here’s an interesting situation! We managed to reach our objectives but getting there was about as much fun as a root canal.  This situation might hurt in the long run as valuable contributors conveniently forget to attend.

A Scrum example:

The recurrent Product Backlog estimation meeting (Backlog maintenance) is essential and the team is able to reach its goal with a two-hour meeting…but for most it feels like a five-hour meeting.  The team sits in a large conference room, the projector is humming and item by item, one or two members shoot out some “man days”.

Why not estimate using Poker or Wall planning? Maybe every two weeks, we hold this meeting in a local café or bar. Everyone contributes, estimates increase in value and…IT’LL BE FUN! We might even save time, making it still more fun!

Lots of Fun and High Value

This is good situation to be in!  Some might say, if it ain’t broke, don’t fix it.  I’m more of a “Good is the enemy of great” kind of guy.  But trying to make it even better can be tricky.  If such attempts are made, make sure the team buys in to it or better yet, get the team to come up with new and innovative ways to reach their objectives more efficiently.

Lots of Fun but No Value

Now here’s the situation that motivated me to write this blog.  Is it possible to be blinded by fun?  Of course! The meeting might have been called to shed some light on a problematic issue, but ended being little more than a laugh-fest. We walk out of there feeling pretty good but the real issue is nowhere near being resolved.

Where I work, we have bi-monthly meetings to explore various situations when coaching individuals.  We do this through role playing; two individuals acting as coach and coached.  As you can imagine, some scenarios are quite hilarious.  Fun and laughter is guaranteed!  It was so much fun that I had a (false) sense that we were accomplishing something of substance – that is, until my colleague casually questioned himself and stated, “Am I my really obtaining value, or am I just having a good time?”

Once I filtered out the laughing and that great feeling associated with being with my colleagues, nothing much was left.  There was some value but not nearly as much as I initially thought. It looked something like this: With that new information in hand, we restructured the activity in order to increase its value, all while maintaining The “Fun”.

Value or Just Fun?

It pays to be aware of what zone we are in, especially when in the “Misinterpretation Zone”.  High levels of fun are beneficial to any activity, but reaching clear objectives remains the ultimate goal.

I’m looking forward to applying this in the real world and seeing what happens!  I’ll let you know how it goes.