mars 2010Monthly :

Linux Kernel development process in a nutshell

I came across Linux Kernel development, a paper from the Linux Foundation that highlights a few important facts about the kernel development process. This nicely complements a few of my previous posts :

A few interesting quotes from the paper

“Regular 2-3 month releases deliver stable updates to Linux users, each with significant new features, added device support, and improved performance.”

“The rate of change in the kernel is high and increasing, with over 10,000 patches going into each recent kernel release. These releases each contain the work of over 1000 developers representingaround 200 corporations.”

“Since 2005, over 5000 individual developers from nearly 500 different companies have contributed to the kernel”

“We have seen a roughly 10% increase in the number of developers contributing to each kernel release cycle.”

“The kernel code base has grown by over 2.7 million lines”

“Patches do not normally pass directly into the mainline kernel; instead, they pass through one ofone-hundred or so subsystem trees. Each subsystem tree is dedicated to a specific part of the kernel (examples might be SCSI drivers, x86 architecture code, or networking) and is under the control of a specific maintainer.”

Once again, there might be a few things we can learn from Open Source.

Pyxis Technologies joins The Inner Circle partnership program

By 2012, agile development methods will be utilized in 80% of all software development projects (Gartner “Predict 2010: Agile and Cloud Impact Application Development Directions”). Pyxis Technologies develops tools that help Agile teams deliver successful projects.

Pyxis Technologies announced that it has been selected to join Microsoft Corp.’s Visual Studio Team System Inner Circle program. The Inner Circle program engages specialized application life-cycle management (ALM) partners worldwide who enable customers to achieve ROI on their Visual Studio Team System tools investmen

Pyxis Technologies is very excited about becoming a member of this elite program“, says François Beauregard, CEO of Pyxis Technologies. “With the Microsoft Visual Studio 2010 launch and this new strategic relationship, we will be able to provide even more value to customers using Visual Studio Team System and moving to Agile development methods. Since 2005, we have trained and certified more than 1200 Scrum masters and participated in numerous Agile development projects. At Pyxis, Agility guides our practices every day. This is why we are the reference in Agile software development.“

Urban Turtle is an Agile project management solution fully integrated into Visual Studio Team Foundation Server. This innovative solution simplifies project planning, helps developers update their work in real-time, improves project progress visualization and encourages communication within the team.

GreenPepper has mature support for .NET and Visual Studio IDE integration which allows teams to efficiently use executable specifications and acceptance test driven development practices in their development process. The Inner Circle partnership will help us to rapidly reach one of our strategic goals in 2010, which is to have tight integration with Team Foundation Server and the Testing features of the Visual Studio ALM platform.

Microsoft is delighted to add Pyxis Technologies to the Visual Studio Inner Circle program,” said Cyrill Glockner, director of Business Development, Platform and Tools at Microsoft Corp. “Pyxis Technologies offers a unique set of products and services to help customers consistently improve their software development productivity through the use of Agile development methods”

Pyxis Technologies will release the 2010 version of Urban Turtle simultaneously with the Microsoft Visual Studio 2010 launch on April 12th 2010.

All product and company names herein may be trademarks of their respective owners.

Automated Unit Tests as documentation

Introduction

In our quest to produce quality software and our relentless use of automated unit tests in our development activities, we’ve come to realize they possess other virtues as well. One of those being they stand as a documentation of the system being developed. I’ve taught this idea in several agile and TDD courses. In this blog I want to give you an example of how that may be. I’m purposely not discussing executable specifications and Behaviour Driven Development as it would make this blog too long. My goal is to show how using some convention and Visual Studio 2010 can go a long way in cranking up your unit tests another notch.

Documentation of API usage

One clever use of unit tests is for learning a new API or even a new language. Writing several assertions to validate our understanding and assumptions about an API usage is very rewarding as the feedback is rapid and progress towards a goal is constantly measured. These learning tests may be kept around to document the assumptions made that constitute the basis of our solution. Here is an example of a learning test for the Character.IsWhiteSpace() method of the BCL.


[TestMethod]
[Description(@"White spaces are the following characters: ' ', 't', 'n', 'r'")]
public void May_be_a_white_space()
{
Assert.IsTrue(char.IsWhiteSpace(' '));
Assert.IsTrue(char.IsWhiteSpace('n'));
Assert.IsTrue(char.IsWhiteSpace('t'));
Assert.IsTrue(char.IsWhiteSpace('r'));

Assert.IsFalse(char.IsWhiteSpace(‘A’));
Assert.IsFalse(char.IsWhiteSpace(‘+’));
}

A test can also document how to use the classes we create in our solutions. The following is an example of such a test. In our chess software, this unit test documents the way of obtaining a chess piece which is to call the NewPiece() factory method and passing it a color (either Color.White or Color.Black) and a Kind (such as Kind.Pawn) as arguments.


[TestMethod]
[Description("A chess piece consists of a kind and a color that are specified when the piece is created")]
public void Should_retain_its_color()
{
Piece pawn = Piece.NewPiece(Color.Black, Kind.Pawn);
Assert.IsTrue(pawn.IsBlack);
Assert.IsFalse(pawn.IsWhite);

pawn = Piece.NewPiece(Color.White, Kind.Pawn);
Assert.IsTrue(pawn.IsWhite);
Assert.IsFalse(pawn.IsBlack);
}

Documentation of behaviours

Unit tests document behaviours of our code. Test classes should be given names that provide context for all the test methods they contain.


[TestClass]
public class When_displaying_a_board
{
[TestMethod]
public void A_black_queen_should_show_as_Q()
{

}

}

Test methods should also be named in such a way that the behaviour they validate is evident.


[TestMethod]
[Description("The board is displayed on 8 lines with blacks pieces on line 8 and 7 and white pieces on lines 1 and 2")]
public void It_should_show_all_pieces_in_their_initial_position()
{
Assert.AreEqual(
"RNBQKBNR".Line() +
"PPPPPPPP".Line() +
"........".Line() +
"........".Line() +
"........".Line() +
"........".Line() +
"pppppppp".Line() +
"rnbqkbnr".Line(),
board.ToString()
);
}

Notice that, instead of writing comments, I’m using the Description attribute to tie the behaviour being tested with some requirement or specification of the system. Personally, I’m not doing anything fancy with these descriptions, but you can imagine using a pattern that would provide more traceability in your environment as in the following.


[TestMethod]
[Description("US 234: Black pawns are shown as P, white pawns are shown as p")]
public void A_white_pawn_should_show_as_p()
{
Piece whitePawn = Piece.NewPiece(
Color.White,
Kind.Pawn);
Assert.AreEqual("p", whitePawn.ToString());
}

Documentation of design specifications

Now comes the fun part! Because of the thought we’ve put in selecting meaningful names when the tests are executed in Visual Studio we get a result screen that looks like this.

Explicit names

Notice how combining our test class and method names reads like a phrase that documents the design of our software. If we group the results by Description, we get a list of all the requirements of our software.

descriptions as specifications

If we expand one of the requirements, we see all the tests that validate it so we can trace back to the code that implements it.

traceability

Conclusion

I hope I’ve convinced you that tests can serve as documentation using simple tricks, conventions and tooling you may already have at your disposal. Because this documentation is embedded in code it has a lot more chance of being maintained as the code evolves. These techniques require agreement amongst team members but they foster good practices of choosing explicit names and tying development work with planning activities for traceability purposes.

OSGi-enterprise related open source projects

First, here is an introduction to what OSGi-enterprise means.

Second, here are a set of OSGI-enterprise implementations :

  • Apache Aries : sponsored by IBM and SAP
  • Eclipse gemini : sponsored by Oracle and SpringSource
  • Eclipse virgo : donated by SpringSource (formerly known as spring-DM) – reference implementation of OSGI enterprise)

On a totally unrelated note, please take a look at the “Relationship to existing projects” on Eclipse virgo’s page. This will give you an idea of how decoupled the Open Source world is : each project has its own Project Leader, its commiters, contributors, and even though the projects depend on each other, there is only limited communication between them. Each project can be developed according to its own schedule, and this is the way open source can scale. The limit is the sky !

Once again, there are maybe some things we can learn from Open Source.

Pyxis and Scrum.org

Pyxis Technologies is currently working with scrum.org (company founded by Ken Schwaber) to open new and unprecedented classes to software developers in Canada and in France.

The first course – that will be available very soon – aims at guiding teams on how to turn product requirements into potentially shippable increments of software using the Scrum framework, Visual Studio 2010 and modern software engineering practice.You can watch this video to know more about the course.

Dates will be opened very soon in Montreal, Paris and Toronto. I ‘ll keep you posted or contact us if you have any question!

François Beauregard co-animera un retour d’expérience sur le “dojo de programmation” le 24 mars à Agile Montréal

Retour d’expérience sur l’approche du “dojo de programmation” par des développeurs à Hydro-Québec

L’Agilité nous a appris l’amélioration continue.

Le “dojo de programmation” applique ce concept à la formation et au développement des compétences. Plutôt qu’une formation ponctuelle et magistrale, on inscrit notre formation dans un processus continu où l’on pratique les techniques lors de courtes séances interactives et intensives qu’on répète régulièrement.

La présentation rend compte de l’utilisation de cette approche de formation par des développeurs Java à Hydro-Québec. On présente le concept des dojos et leur organisation. On discute des avantages et de la valeur de cette activité ainsi que des attentes que nous avons par rapport celle-ci. Finalement, on discute des leçons apprises et de la place que nous comptons donner aux dojos dans le futur. Vous verrez comment vous pouvez introduire simplement et à peu de frais, cette activité de formation novatrice dans votre milieu de travail.

Quand: Le 24 mars à 17h30

Lieu : La salle des Boiseries de l’UQAM, local J-2805 au 2e étage du pavillon Judith-Jasmin (J) situé au 405, rue Sainte-Catherine Est (approximativement sous le clocher de la rue Saint-Denis). On y accède depuis la Grande Place. Pavillons de l’Uquam.

Inscrivez-vous à cette rencontreet n’hésitez pas à transmettre cette invitation à vos collègues.

Open Source is Not a Democracy

IT world runs a very interesting article explaining why Open source is not a democracy, and never should. Some interesting quotes :

“No. This is not a democracy. Good feedback, good data, are welcome. But

we are not voting on design decisions.

“Shuttleworth is in the right here. Ubuntu and a vast majority of free and open source software projects, including the Linux kernel, have never been democracies. They are meritocracies, and any member of a community that thinks otherwise is kidding themselves.”

“too many hands on a project with no consensus of direction leads to a pretty crappy project

“It’s critical to recognize that open source does not bring complete democracy to software development. It never did, and it never should. Ultimately, someone in the developer chain will have to make the tough calls.”

So, if you run your software development project as a Democracy, there might be some things you can learn from Open Source.

Agile lessons learned #8: Muscle Memory

Casey Viator@Colorado Experiment

Casey viator@Colorado Experiment - photo credit to oldtime strongman

“If you find a man in the desert who has been digging a hole with his bare hands for months and you show him a shovel, he will probably try to kill you with your own tool. If you dare come back later, he will still be digging the hole with his bare hands, even though he still has your shovel with him.” — Unknown

The muscle memory theory

In 1973, the Colorado experiment was conducted in order to test the effectiveness of brief and extremely intense exercise. The pictured subject, Casey Viator was  a de-trained bodybuilder coming off a year long layoff. The experiment lasted exactly 4 weeks. By the end of the 28th day, he had gained 63 pounds of muscle and lost over 17 pounds of fat. To the author’s knowledge, such results had never been produced prior to this experiment and have not been replicated since. The results have often been attributed to “muscle memory”.

The theory behind muscle memory is that the body always tries to maintain a certain balance. In order to grow, it has to be overloaded in order to give the body the signal it needs to adapt. Over a period of time, once the body grows accustomed to a certain level of muscle size/strength it then becomes the the new balance the body tries to maintain.

When someone is under stimulated, the body will regress somewhat rapidly until it falls back to it’s “balance level”. Once there,  it will try to hold on to the balance it was accustomed to. Eventually, since it is under stimulated, it can actually regress back to previous levels of size/strength since higher levels are no longer required. But with proper stimulation, it can grow back to the previous maximum levels in less time than it first took to reach those levels because of the so-called muscle memory.

When it comes to human behaviour, the same phenomenon is visible. Just like with with muscular balance, the body seems to try and maintain it’s comfort zone as some sort of defence mechanism. Once taken out of the comfort zone, it often triggers the reflex of going back to the old ways of doing things, no matter if it the previous behavior has a positive effect on the person or not.

An Agile transition as an example

When putting forth a transition in an enterprise such behavioral patterns must be understood and dealt with. Let’s take an Agile transition that takes employees who are used to a very hierarchical environment and are moving into self-organized teams as our example.

Applied to teams

For starters, let’s take a look at something as simple as task assignment. This shouldn’t be rocket surgery right ? Well, it depends on you standpoint.  Many people have become so accustomed to having tasks assigned to themselves they have a hard time dealing with the relatively simple responsibility of self task assignment. Actually, many people will look around for somebody to assign something to them or pretend they are committing to something while in fact they let everybody else decide and take what’s left.

Just dealing with the Product Owner on the appropriate scope of a sprint is something a lot of team members might find hard at first. Telling no to somebody who is asking you to do more stuff is not the usual behaviour most of us are accustomed to. Not convinced. Here are some examples. When we were young, we were expected to do what our parents asked us to do. In school, we were supposed to act the way the teacher asked and to do all our homework. At work, we are supposed to accomplish whatever work we are asked to do. See a behavioural pattern there ?

On the technical side, it is quite the challenge to ask developers with 20 years of experience who have never written a single test in their lives to do all their programming test-first. By TDD we don’t just mean writing test. What we mean is that while developing software, not a single line of code is written without a test written to test it first. None.

Funny thing is, once most people get the hang of TDD, they will have a hard time going back to your old ways. Unfortunately, you will often find programmers skipping tests altogether, even though they are at least aware of the benefits. In extreme dissonant cases, some will be sabotaging their own work on purpose for no apparent logical reason other than to maintain their previous ways of working.

Applied to managers

It also works the other way around. Having power over people is something that is seemingly very hard to let go of. In Agile teams, management is often seem as an umbrella protecting the team or as a coaches. Either way, they are helping the team attain it’s goal while supporting it when it needs to and protecting it from distractions. Other responsibilities might be helping team members get the proper training and also by keeping them on track regarding delivery objectives.

It requires a lot more soft skills to be a successful Agile manager than to simply be someone who’s main attribute is to dispatches work. It also involves the humbling task of letting go some of the grasp managers often have over every doing of their employees. It is only too common to hear about managers embracing Scrum, only to find them minutes later behaving exactly as they were before. This is done in spite of them being entirely aware that their previous behavior is unproductive and is what conducted the requirement for change in the first place.

Just like with weight training, it takes a long period of stimulus for the body to recognize the current state as the state the body needs to maintain. It is the same with change, the longer you keep at it, the more and more it will feel natural and it will just become the new habit. Remember, some things you think of as an habit now might have once looked to you as insane too.

-Nicholas Lemay