Agile is lots of Communication

This is the first post of a series of posts where I will try to define what I think are the most important properties of Agile. As I have been reading Alistair's book, the first post is about "Communication".

What is communication and why is so important in Agile?

To understand the importance of communication in Agile, let start by its definition. The Merriam-Webster definition says communication is "to transmit information, thought, or feeling so that it is satisfactorily received or understood". Another definition taken from Alistair Cockburn says "communication is transferring one idea from one person to another". The latter is a stronger definition as it implies the idea needs to be understand by the receiver (as opposed to just received).

Why is so important to communicate in Agile?

Agile reckons that the process of construction of a software solution is a creative game where the individuals involved interact continuously to build a solution. Cockburn calls the process "a cooperative game of invention and collaboration". During this game, all actors involved contribute their particular knowledge and therefore they need to communicate among them as effectively as possible.

Even more, most Agile methodologies deliver the solution incrementally and encourage highly concurrent development in each iteration. This means all actors are usually working in the same piece of functionality at the same time. Abundant and effective communication among all actors is vital.

Let's try to visualize this with an example. At the beginning of the iteration, an increment of functionality is proposed to be built. This functionality is expressed as a set of user stories written usually by the product owner. In the planning meeting, the product owner tries to communicate what s/he thinks are the key ideas of this new functionality. A conversation between the team members and the product owner is usually kicked off where the team members usually provide quick important feedback about the feasibility and also quite often new ideas that could improve what the product owner had in mind. During the iteration, teams need to communicate extensively with the product owner to clarify missed points and to show progress. This constant communication is key to build the right product (i.e. to build what the customer is expecting). Among the teams, communication needs to be fluent and constant as well. Developers and testers (and all other actors involved) need to design, develop and test a piece of software in a short period of time. A great flow of information circulates when designing the solution, the tests and after development has finished when the team converges and agree on the final product. Agile foster the creation of the test scenarios and test cases as a group and also the creation of code in pairs. It can be seen that the flows of communication go in all directions.

Individuals and interactions over processes and tools

What are these guys trying to say? The statement says a lot about the philosophy underlying Agile and it takes a great deal of time to understand it and interpret it. My current understanding says Agile values more a team of crafted software engineers collaborating towards the creation of a solution than the same team following a prescribed process. As it is explained in the Agile Manifesto webpage, it is a "We value more" statement. It doesn't mean processes should not be used or tools are useless. It means that the focus of the methodologies, processes and frameworks is on the people and their interactions rather than in the creation of intermediate work products, filling templates or following instruction lists.

How should this interaction occur? Agile delivers software incrementally in short iterations using highly concurrent development practices. All members need to communicate to produce something in a short period of time and therefore the communication needs to happen as efficiently and as effectively as possible. And which is the most effective way of communicating? The Agile Manifesto lists as one of its 12 principles: "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation".

Which are the most effective ways of communicating?

It is very difficult to communicate. If you think that even between people of the same culture and language talking face to face in a room there could be misunderstandings, you can imagine how difficult it could get (don't you have misunderstandings with your wife after living with her for many years?!!). People have their own context and ideas and so the information they receive is processed according to it. Besides, our natural language is limited and imperfect (and it reduces to the person that is trying to communicate).

So which are the most efficient and effective ways to communicate. How can we minimize misunderstandings? How can we communicate faster? Cockburn studied this and came with the following graph.

In the X axis, there's the temperature of the communication channel. Reading a paper document is the coolest of the channels and therefore, the least effective. Talking face to face in front of a whiteboard, while writing diagrams is the hottest and most effective way to communicate. I don't think many people would disagree with this. There is nothing like talking to someone while you see his face, his expressions and his body language. These "other" channels act in both directions and allow to reformulate or write a diagram if the receptors make "weird" expressions or even speed up if the receptors are getting bored. There is nothing like seeing the face of the people you are communicating with.

So how should this graph be used? All projects and all teams are different and run under different circumstances. If the team is small and it works in the same building, it is clear that they should all work in an open room, where they can talk to each other whenever they want (this is the Agile sweet spot). However, today's reality brings much more challenging projects, where the teams are distributed, in different countries, with different languages, cultures and worst of all, huge timezone differences. Generating the necessary communication channels in these situations could get really hard, but we need to keep in mind that communication is key to success in Agile. Each team will have to use the best of their imagination to figure out ways of heat the communication as much as they possibly can.

It is important not to loose pragmatism when thinking the communication channels. For example, Cockburn mentions that when the development team doesn't have full access to the business people, it is effective to leave some kind of "sticky information" that developers can check whenever they want. I found this example very valuable, because there are always fanatics that read the theory and stop thinking.

How does Agile "manage the incompleteness of communication"?

Agile tries to manage the incompleteness of communication by trying to improve the communication influx and by verifying, as soon as possible that what was communicated is what was really meant to be communicated.

Agile improves the communication:

- By encouraging open spaces where both business and technical users sit together. This fosters conversation among the technical members and with the business people, conversations can be overheard, everybody knows who is on its desk and who isn't and usually what are they working on. In short, it creates a medium where everybody is sharing the same information because the communication channels are shared by everybody. Cockburn calls this "osmotic communication".

- By having a daily standup. This short meeting that occurs daily pushes everybody to communicate.

- By doing pair programming, as XP suggests. This practice can be performed by 2 developers, 1 developer and 1 tester or 1 business people and either a developer or a tester. Although the main objective of this practice is to produce better code and minimize the bugs, it creates a great opportunity to exchange patterns, techniques (when done between technical members) and business information (when a business user is involved).

Agile provides several verification points because it delivers the software in iterations. This means that in a short period of time (from 1 week to 1 month) the business people can see working software and can verify if what they see is actually what they had in mind. Agile also encourages business users to write acceptance tests. These executable specifications are run as soon as the functionality is completed and also serve to verify that the develop functionality is what they are expecting.

Conclusion

Agile development relies on creative individuals which interact continuously to build the proposed solution. Communication could get hard, but is always necessary. Each project could bring its own challenge. Agile relies on these creative individuals to find ways to communicate as much and as good as they possibly can.

2 Intense Days at Agile Open California

I spent the last 2 days at #AOCA, in Fort Mason (San Francisco, 15 blocks from home!). This is my second experience in open spaces and I love it. I think it is the most Agile format as it allows and promotes collaboration and communication in this endless game: your career.

I met wonderful people, like Pat from The Gap who told me how she was thinking of expanding the agile horizons in The Gap. Wait... Expand? I told her the impression that I got when I was there is that everything was already Agile (It is really impressing see a whole department doing Agile..). I also met Tom Looy from Tacit Knowledge and Joshua Kerievsky. They both had been in Florianopolis last week, where they told me had a great time (Joshua even told me that he would've gone to Buenos Aires, had he knew about it).

First day:

The first session I attend was about "Growing Agile" or how to implement Agile in bigger groups, far from the Agile sweet spot. We have a pretty big group in Axway, but some people talked about groups of 100 or 200 or even 500. Pretty amazing, ah? I introduced my question, which is how you maintain the technical coherence in such big groups but didn't get any good answer... Will keep asking!

The second session I attended was Kanban 101. I still really don't understand the difference with Scrum. I thought it was about not having iterations but they told me you could still have iteration and do kanban. They told me that it was about minimizing "work in progress", putting a maximun numbers of items in progress in each phase.

The third session I attended was about team trust. We talked about team trust (collaboration) within the team. We identified things that work well: Having a shared environment, build community, collective ownership, aligned goals, a good compensation system, fairness and freedom to agree and disagree. Among the factors that can hinder collaboration we identified: "your failure equals my success environments", contradictory values, lack of motivation, a non supportive environment and lack of genuine commitment. We ran out of time, but the facilitator wanted to talk about trust within the organization. Much of what Pat is trying to achieve. Very interesting.

The last session I attended on Thursday was about a training game for agile leaders. This game continued as the first session on friday. What I found interesting about this game is that the whole introduction of Agile is through a game that simulates an iteration, it calculates business value, it is timeboxed, etc. What I am trying to get at is that the introduction contains a lot of the philosophy behind Agile.

Second day:

The second session I attended was about Agile Visualization Tools. We went over the different graphs and diagrams that could be used. I knew most of them, but there was something that was mentioned that really caught my attention. A while ago, I sent an email to the scrumdevelopment list (http://bit.ly/vzB4u) asking if there was another way of categorizing user stories. Well, it seems there is and is called "User Story Map". I will keep investigating this for sure because it seems something that could be really useful for us.

The third session I attended was about project chartering. Joshua and a coworker explained which are the most important components that should be agreed upon on a contract and the importance of agreeing on those things (http://bit.ly/3nYtI1)

The last session I attended was about distributed agile. Challenges and tools were listed. It seems most of us are facing the same problems :-)

All in all, I enjoyed the conference a lot.

A First Approximation to Understanding Software Quality

My first thoughts about software quality

First time I thought about software quality was in my first agile project, a few years ago in Globant. I remember a message on the blackboard that said: "Quality is not negotiable". Back there, I didn't know much about agile, but I was impressed with all the good engineering practices that we were following in a very disciplined way. Now, having read a few more books, I could say that we were doing XP. My interpretation of the message at that moment was: We are not going to abandon any good engineer practice due to time constraints, because the consequences of doing so would be worse. In other words, even if it takes more time to do automated tests for all features, refactor every time we felt we need to, do pair programming, code reviews, etc we are going to do it because at the end, it pays off.

Different definitions of software quality

So what is software quality? Years have passed, books have been read and the definition is still vague for me (and looking at the forums, it is still a topic of heated debate).

Juran's definition is "Quality is fitness for use". But for use by who? Juran specified that the customers should be identified and their needs determined.

Crosby's definition is "Quality is conformance to requirements". The problem with this definition is "who creates the requirements?" (i.e. those requirements attend the needs of all users?)

The Poppendieck's make a separation of quality in production and quality in development. According to them, Juran's definition corresponds to quality in development and Crossby's to quality in production. Other differences are "variable results are good" in quality in development, but bad in production quality and "Iteration generates value" in quality in development, while it generates waste in production.

Quality in software development results in both "perceived and conceptual integrity". Perceived integrity means the products attains "a balance of function, usability, reliability and economy that delights the user". Conceptual Integrity means that "the system's central concepts work together as a smooth, cohesive whole".

Weinberg's definition is "Quality is value to some person". This definition acknowledges that quality may mean different things for different persons (i.e. quality is relative).

For example:

- Quality is Zero Defects for people that are disturbed by those defects

- Quality is having a lot of features for people that use them

- Quality is elegant coding for developers that appreciate that

- Quality is high performance to users whose work taxes the capacity of their machines

- Quality is low development to managers with low budgets

- Quality is rapid development to users who are waiting for the software or to marketers that want to sell it

- Quality is user friendliness to users who spend 8 hours a day in front of the software

Weinberg's reckons that the definition of quality is political and emotional, because it "involves a series of decisions about whose decisions count, and how much they count relative to one another"

Quality that can be measured versus Subjective Quality

A lot of the quality definitions are very subjective. Value to a person.. Fitness for use.. How can we measure that? (I guess the best measurement is how well the product sells). On the other hands, there are things that can be measured. For example, CMMI approach, in its levels 4 and 5 try to measure quality using statistical analysis.... But does that assures a satisfied customer? In Agile, certain measurements can be taken, for example how many tests there are, the code coverage. Do those number assure that the tests are well written?

Open Questions

- Can we divide Internal and External Quality? The Poppendieck's talk about perceived (external) and conceptual (internal) integrity. Marick, in his testing quadrants also specifies that some quadrants aim at internal quality while some others to external. Does it make any sense to make this separation? I know you cannot have external quality without internal quality. However, it could be entirely possible to develop an excellent product that doesn't provide any value to the customer (so external quality implies internal quality, but not viceversa).

- Weinberg's definition brings other dimensions to my current ideas. For example, that rapid development or time to market can be considered as quality attributes.This is not intuitive to me. I understand that having a product in a couple of weeks has a lot of value for some persons, but if that product is bad programmed, without tests (etc.!) I wouldn't consider it a quality product.

Conclusion

The definition is still vague to me. However, to think a little bit about quality made me realize that most of the Agile values and principles aim at improving it. For example, short iterations and user involvement improve the value to the user because something nearer to what is expected is built. Engineering practices like automated testing, refactoring and continous integration improve the conceptual integrity and thus the perceived one. In other words, Agile tries to maximize the quality - value - at the least possible cost.

Agile without automated testing

Sometime ago, I was talking to my friend Matt about what would be the 'Cost of Change Curve' of Agile projects without an efficient automated testing strategy. Basically, we were talking about Boehm's Cost of Change Curve.




And Beck's famous flat cost of curve.



The latter, assumes all XP practices are performed efficiently. So the question, how would the curve grow if automated testing was not performed or was not performed efficiently. I don't want to go into much detail about what kind of automated testing (unit, functional). Let's assume the selected strategy doesn't work.

Today, reading a Cockburn's book, I saw the graphic that I was imagining, but with another topic: Agile without refactoring.

This is exactly what I thought. At the beginning it is cheaper not to think about an automated testing strategy. As the project evolves. the cost of changes without having the automated test harness grows exponentially. The analogy with refactoring is perfect. At the beginning, it is possible to do small changes without cleaning the code. A few weeks, after that, the code becomes so messy that it becomes costly or impossible to do the changes.

Self-organized sardines

Self-organized sardines swimming together at the Monterey Aquarium

Updated: A coworker (with whom I went the first time to Monterey) went recently to Monterey to check the self-organized & self-disciplined sardines, but there was an ungrateful surprise. The sardines where not organizing very well: some of them were going one way while other were going some other way and even some anarchic sardines not following any circle!! The result, the sardines where going much more slower!!!!

Disorganized sardines... Lack of leadership in that group?












Thank you Eli for the pic and the comments!