The secret ingredient.... feedback

A few days ago, I went to a OO course, given by Hernan, a colleague at 10 pines. The course started with the basics: a definition of software. Hernan gave this definition: software is a computable model of a problem domain from reality. Our job as software developers is to interpret that problem domain and construct a computable (executable) model that suits the needs of the users.

Now the question is how you get to that model? I started thinking that sequential approaches (waterfall) take an approach of trying to understand very well the problem domain before starting the construction of the model. The gap is bridged by domain experts that specify thorough-fully the problem domain so that technical people start from there.  The Agile approach, on the contrary, tries to understand a bit, models that bit, seeks feedback and then starts again. The model is built iteratively and incrementally, in short cycles. Agile builds the model through feedback.

I started thinking about all agile components I've been thinking lately and realize they all seek feedback.. a lot of feedback. Think about it:

Technical components: TDD, automation, continuous integration, etc.... TDD seeks continual feedback. Does the code do what is specified in the tests? The execution provides the feedback immediately. Automation.. tests are executed all the time so that the team receives feedback. Continuous integration provides feedback of whether the local changes integrate well with the whole (in a neutral environment).

Process components: an incremental and iterative process seeks business feedback. Incorporating this feedback makes the software much more valuable.

Human components: Agile seek to augment the flow of communication. The best communication is the one that includes feedback, because everyone gets to know what is understood (Cockburn says the best one is talking face to face, where you even see the feedback in the other person expressions)

Feedback, feedback, feedback.. Agile relies on feedback to bridge the gap between the business domain and the model being built. Technical and business users collaborating to create a valuable model. Knowledge is built through feedback. Software is built through feedback.

Agile is Ruthless Automated Testing

Test automation is one of these components whose importance is difficult to understand at the beginning. It's not obvious and by consequence it is often underestimated. Probably, it should have been stated in the Agile Manifesto in a more obvious way (now you need to know what 'working software' means for the signers). What's worse, it is difficult to do and it requires a great deal of work!!  For what??!! Let's see.

Why Automated Tests are important in an Agile Context

Automated tests are important both in projects that use traditional and agile methodologies because they allow to regress all the functionality without effort. However, in agile projects they are more important. Here are the reasons:

1. It supports a sustainable process.

Agile states the importance of a sustainable process, that is a process that allows to deliver features continually in a long period of time (i.e. deliver features each iteration, while keeping the existing ones working). In other words, a process that maintains the "cost of change" stable (not growing exponentially) is needed.

Getting to this point is really hard. In a short period of time (an iteration), new features need to be constructed and old ones need to be tested. As more and more features exist, there is more and more regression work, which becomes very time consuming, tedious and error prone. Automating this work becomes the key point to keep the process sustainable.

Agile suggests there should be an extensive suite of automated tests that cover most of the functionality. This "test harness" should be running continually to maximize the feedback and it should be up to date at all times. This means that for every new functionality added to the project, there should be tests that are added to the test suite.

There is another important point on sustainability. Part of being able to have a sustainable process is minimizing the introduction of bugs during the iterations. Computer systems usually have a large entropy. That means that changing one part of the system may impact other areas which are very difficult to predict just by analyzing static code. Good automated tests provide this information very quickly allowing developers to make the changes and see the results immediately. This continual feedback prevents lots of bugs from appearing, amplifies the knowledge of the developer and also gives the courage to introduce changes in the design or make complicated refactorings.

2. The quality of the code is increased

This is like a turn around of things. The main objective of tests is to regress functionality, but it has been proven that just the fact of having to write them improves the quality of the code under test. Interfaces are better understood and designed and the code is cleaner. It also limits the code to do exactly what is supposed to do, as programming is done to make the tests pass.

TDD takes writing tests to the extreme. Tests are written before the code. The code is then written to make the tests pass. So, the tests drive the development. This technique has many advantages: We can be certain that the software is working at all times (which improves the morale) and there is a very high code coverage which gives the possibility of making bigger changes in the code.

3. It allows the team to be agile

This reason is very tied with the first, but I think it's worth mentioning it in a separate point. The design in agile emerges iteration after iteration. Every iteration, the design of the application could be changed to accommodate the new requirements. This changes should be done in the cleanest way (It is not acceptable under the agile principles to accommodate changes with patches). The cleanest design for the new requirements should be thought out and all changes and refactorings should be performed (otherwise, it wouldn't be sustainable)

There is also the possibility that features that have been completed need to change (possibly in reaction to market changes or because the implementation of a new user story so requires). Agile is prepared for very dynamic environments and therefore it should be prepared for these changes. It's nothing but the test harness what allows to make "big" changes or turns of direction.


4. It improves speed

With all the work involved in creating and maintaining tests, it can be easily derived that a project with automated tests would take longer to finish. Well, this is not often the case. In many occasions the speed is improved. The main reason for this is fewer bugs are introduced along the way. The flow is stable and, at all times, the code is known to be working. Even more, if there are bugs, these can be fixed with a fewer probability of introducing a new one (minimizing the effect that Brooks called: one step forward, two backwards)

Dean Wampler's post explains this in more detail:

Basically, what the graph represents is a comparison between a traditional project without automated tests (orange line) and a project that uses TDD. When the project is reaching the end (up until this moment the speed seems to be greater) QA starts working and discovering errors leading to a big increase of work. Not having integration tests leaves the possibility of discovering integration problems as well. After developers start fixing bugs, new ones can be introduced more easily because there is not test suite that provides feedback on the changes made. Finally, the project ends with all the stress derived from having "discovered" all the problems at the very end. On the contrary, a project that uses TDD has more work at the beginning, but then it maintains a more stable flow of work. At all times, the code base is known to be working and therefore the expectation is to have a smooth ending.

5. It lowers Maintenance Costs

Most of the systems have a much greater cost in their maintenance phase than in the construction phase. A lot of systems that take only a few months to be built are years in production. The cost of maintaining projects without automated tests is considerably higher than maintaining one with a good test automated suite.  Fixing a bug many months after it has been introduced is extremely difficult. Even more, the changes necessary to fix the bugs may have unforeseen consequences. In traditional projects, Brooks reckons that fixing a bug during the maintenance phase has a 20% to 50% probability of introducing a new bug. He mentions that the reason for this high probability is again the high entropy of computer systems and also the possibility that these bugs are solved by people who was not involved in the construction of the system.

An automated test harness is invaluable in these circumstances.The tests allow the developer who will fix the issue to understand the logic of what needs to be solved. After introducing the fix, it allows him/her to be more confident that no new bugs have been introduced.


6. It is the best documentation

Automated tests represent the best possible documentation. Conversations with the Product Owner and the different stakeholders live into GUI tests and design discussion live into integration and unit tests. The key aspect is they are executable documentation. They are much more valuable than static documents like models and written documentation because they provide continual feedback. Besides, they are written in a language that provides no ambiguity.

7. It supports other practices

It is important to point that test automation provides a lot of advantages per se, but it also supports other practices allowing to reach the synergy Beck talked about in his first XP book. Automated tests can be a great help in requirement specifications (with the latest BDD approaches, the Product Owner is able to write the requirements as executable specifications). It supports coding: Agiles strives to have clean code that works at all times and automated tests are a great help in this quest. It supports collective ownership, as automated tests provide great help to understand certain aspects of the functionality. It supports refactoring. In fact, it would be impossible to refactor continually if there isn't a test harness that prevents the introduction of bugs. In conclusion, if it is performed correctly, it makes all the other practices work better. If it is not done at all, other practices are debilitated. The whole approach relies on having automated tests to drive the project.

Why don't we automate them then?

Which are the ropes that prevent us from doing it, even if we understand all the advantages?.

1. Tight schedules

Whenever we feel the pressure of the schedule, we stop writing tests (or don't fix the tests that are broken). After all, tests don't constitute the core of our system. Of course, this has direct consequences in the near future.

Ward Cunningham coined the "Debt Metaphor" to explain the refactorings they were doing at WyCash product. He explained that the knowledge gained during the project need to be reflected in the code. In other words, the code should look as if it was written with all the knowledge in mind, while at the beginning fewer things were known. To justify the design changes and refactorings, Cunningham drew the analogy to financial debt. The needed changes represent interest, which if it was not paid would inevitably increase and would inevitably have to be paid later at greater costs. This analogy was expanded to all tasks in software development that might be postponed (tests, TODOs, etc.). Postponing them generates an immediate increase in speed at the cost of creating debt. If this debt is not paid soon, greater interests would have to be paid later.

2. Technical Difficulties

Writing tests is really difficult. It takes time to understand how unit or integration tests should be written (and a bad written test is useless or even worse: it creates a false sense of security). Besides this, there is also the need of investing time and effort in creating an infrastructure for testing (if there isn't anything that could be used).

A Project's Strategy towards Automation

Crispin/Gregory talk about defining an strategy towards automation. An automation strategy defines the plan to deal with automation in a particular project. These are just one of the few things that need to be defined:

- Layers that are going to be tested
- Tools that are going to be used
- The process that is going to be followed to commit code, create the tests,  what will be done if the build is broken, etc.

It is important to have everything in mind when thinking about the strategy

Technical factors, such as:
- Available frameworks and tools
- Familiarity of the developers with the frameworks and tools chosen
- Initial necessary investment (in developing the tools or training)

and factors more related with the business, such as:
- Importance of the project for the enterprise
- How much business involvement will be during the construction
- The life of the product post implementation

Each project is different, so each project will have its own strategy (i.e. there isn't a best strategy). Green field projects will have a different strategy than projects with legacy code. The technologies chosen for the project are a decisive factor as well (not the same to do an outlook plugin, an iphone application, a CRM, a mesh, etc).  All the team should adhere to this strategy, otherwise it is doom to fail.

Conclusion

Test automation is hard, time consuming and doesn't add value directly. However, not doing cripples many of the Agile benefits. An Agile project with no test automation is not sustainable, its quality decreases and has many chances of taking longer. In a project without test automation, is hard to see where you are standing (Is everything working?). And it definitely requires more repetitive, boring and error prone work.

Every project is different. The needed product is different, under different circumstances and with different groups. Everything needs to be taken into account to define a strategy towards automation. The strategy should be one that everyone believes and commits to.

References

1. Lisa Crispin, Janet Gregory - Agile Testing: A Practical Guide for Testers and Agile Teams.
2. http://www.scrumalliance.org/articles/106-definition-of-done-a-reference
3. http://www.agile-software-development.com/2007/07/definition-of-done-10-point-checklist.html
4. http://blog.objectmentor.com/articles/2007/09/30/why-you-have-time-for-tdd-but-may-not-know-it-yet
5. http://gamesfromwithin.com/?p=50
6. Kent Beck - Test Driven Development - By Example
7. http://www.youtube.com/watch?v=pqeJFYwnkjE
8. http://en.wikipedia.org/wiki/Technical_debt

Agile is Clean Code

But ... Isn't this a requirement for all approaches? Why do I have the impression that in Agile, the technical aspect is more important? When I was at University, I remember vividly having heard (and unfortunately later repeated) that the programmer had one of the less important tasks in the chain of producing software. The most important, more senior and smarter people are the architects/designers that analyse and design the application, giving the programmers some diagrams that they 'just' have to translate into code. That seemed like an easy task!! 
   

What means Clean Code?

Ron Jeffries says that we should write "clean code that works". Uncle Bob then made an entire book of Clean Code. What is clean code? Following uncle Bob's explanation, at the high level, it is code that is well structured and which follows SOLID design principles. At a lower level, it's code that has meaningful names, good and useful comments, elegant functions and formatted code.  It's code that expresses the intent of the developer in a clear way. It's code that contains no duplication. And is code that is easy to understand.


And how do you know it works? You don't know because you code it and you are a great programmer :-). You know it because there are automated tests that certify it. Agile development relies on test automation to know that features work and to know thay they do all the time. Beck says "Software features that can't be demonstrated by automated tests simply don't exist". Test automation is one of the pillars of clean code because it allows to continually refactor, it allows to redesign and ultimately it allows to deliver a continuous flow of value.

Why is so important in Agile?

These are a few reasons to justify the 'pain' of writing better code:

1. Cost of Change is lowered: Agile welcomes change, even late in the development process. However, to be able to change, it should be cheap to change. Code that is well designed, easy to read and thoroughfully tested is easy to change and can be changed without fear of introducing new bugs. On the contrary, code that is hard to understand, is not well modularized and has no automated tests is difficult to change and the risks of introducing bugs while modifying it are many.
2. Iterative & Incremental Development means more time coding: In sequential development, there are big phases of analysis and design before the coding phase starts. When the code phase starts, most of the design decisions should have been taken, so development proceeds fast. In Agile, development starts earlier so this phase takes more time. As the code base needs to be manipulated for a longer time, it pays to have it clean. Reading Clean Code, one of the a-ha moments that I had was in the chapter where Martin explains that in the life of a system, the majority of the time is spent reading code, not writing it. I started thinking about my experience, and that was entirely true. I have worked in many systems that already have a lot of features in it. In those systems, each time that I needed to add a new feature, I spent most of my time finding where to put the feature and how to accomodate it to the existing features than actually coding it.
3. Iterative & Incremental Development means the code is changed frequently: Design is performed incrementally in Agile. Every iteration, new features are included in the code base. The design of the system should be one that cares about all features included until that moment. Therefore, this means that a lot of times the existing design needs to be modified. If the code is not maintained clean, understanding and modifying it may become difficult in a short period of time. 
4. Working software is more important than Comprehensive Documentation: This is what the Agile Manifesto says. But having the code base in good state, with good comments, good names, well modularized and with a comprehensive set of automated tests is the best documentation that a developer may have. Been able to understand a piece of code easily because it's simple and it expresses its purpose with clarity is much better than going through UML design documents. Being able to run and debug a set of unit tests for certain functionality provides the best tool to understand that functionality. After all, what really happens is what is in the code, not what is in the documentation. Code is always up to date while documentation is hard to maintain. 

Why don't we do it then?

Lack of skill? Sometimes... This is a craft. A difficult one and we need to spend a lot of time getting better on it. Commitments.. Yes. Our own commitments and the commitments of our managers. Being under pressure and with the deadlines on sight, make it 'work' takes another meaning (I mean, you just want to throw some code that at least compiles and shows something on the screen). I won't follow. There are always plenty of reasons to do things worse!!

Humm.... but Scrum doesn't include a topic on technical practices

I've seen great debates last year over the lack of technical practices in Scrum. Some people believe it is a shortcoming of Scrum while others say Scrum is a product development framework and therefore it is out of scope to include a technical section. Without entering into this debate, something that I've seen a number of times is software development teams starting 'just' with Scrum, leaving the introduction of some XP practices like TDD for the future (if you take a look the popularity of XP and Scrum, I guess many people observed the same!). This may be justified by an approach where changes are introduced gradually (after all, Scrum per se represents a major change at the organizational and process level).

But what happens if you start with 'just' Scrum?. Basically, I believe Scrum alone is not sustainable. At least, in software development it is not. As iterations go by and the code base grows, there is more and more work to finish each feature. By consequence, teams cannot keep their commitments, the business people get mad and you hit the Scrum Wall (ouch, hit it many times)

This is still a topic I think about often. Scrum, per se, is not sustainable in software development. However, it is the most succesful process in the Agile world. It seems that leaving technical aspects out of it is both something that has help Scrum become more popular  and the reason of many of its failures in the development world. 

Conclusion

Clean Code is one of the pillars of Agile. Being able to deliver clean code that works greatly enhances the chances of deliver value sustainably and it makes our life as programmers better. Clean code provides the best ROI and the best documentation. Of course it is difficult to do it and it certainly takes a lot more time than writing crappy code. However, by not doing it, we lose a lot of the advantages that Agile claims.

Real Men of Agile Genius

Enter the @VersionOne #AgileGenius sweepstakes. Prizes include an Apple iPad. Takes about 1 minute. Visit http://bit.ly/agile-genius

Agile is having a lightweight methodology

First, what is the meaning of being light? A lightweight methodology is a methodology with a low number of control elements (e.g. deliverables, standards, quality measures) and low ceremony.

Cockburn suggest to take into account these factors when designing a methodology:

• Problem size: bigger problems need more people to resolve it, and by consequence more coordination mechanisms. 2 developers working in a garage hardly need any coordination rule. 100 people working in a big building need a lot of coordination rules.
• Criticality: Will the life of some people depend on the software you are making? If it will, tighter controls are needed. 
• Precision and accuracy: Are you dealing with wall street money or is it just a movie rental system?

He also says that there is not a one size fits all methodology. The methodology needs to be designed based on the problem that needs to be resolved. Smaller, less critical projects with 6 members on the team need much less artifacts and coordination mechanisms than bigger, more critical projects. It is not the same to have a collocated team of 6 people, all of the same culture/country than having 100 people, in different continents with different cultures, languages and time zones.

Therefore, why are we saying that Agile is having a lightweight methodology? Probably what we should say is "Agile is having the lightest methodology that the problem in hand needs". Highsmith states that the methodology needs to be "barely sufficient" ... "A little bit less than just enough". The methodology should be light in the context where it is applied.

Why do we want a lighter methodology?

First, because we want to deliver value early and often. Cockburns states that excess methodology weight is costly because more time is spent writing and maintaining work products (like GANTT charts, plans, etc.) that don't directly contribute to adding business value. Teams that use lighter methodologies are more productive because they spend most of the time on productive activities. The second reason, which is as important as the first, is Agile welcomes change. With a heavy methodology, it is difficult to change because there are many artifacts and more ceremonies. If there is a lot of documentation to change and many meetings, changing will be painful for everyone and therefore everyone will want to avoid it.

Making your methodology lighter

There should be constant instrospection on the artifacts and ceremonies that take place inside the team. Face to face communication is more effective than communicating through an intermediate deliverable and it is much lighter. Having the team together increases the flow of communication and therefore decreases the need for written documentation and meetings. Highsmith says "don't confuse documentation with understanding". In the end, what matters is understanding. Feedback provided by business people is of utmost important. Instead of trying to define big diagrams, which usually contain a lot of guesses, do a little bit and show it to someone that understands. This type of communication makes the methodology lighter while maximizing the chances of doing what is expected. Another important point is discipline. Agile teams are highly self disciplined. As a developer, I have felt the pressure of doing things the best way I possibly could because I was pushed by the high standards of the team. This is in contrast of being pushed by a process. Agile teams constantly introspect what's giving them value and what isn't. Why are we making this document? Is this meeting being helpful or is just a waste of time?

Conclusion

Artifacts, meetings, intermediate deliverables, etc. are all tools that we use towards our goal: give the user software that brings some value. Agile tries to maintain these tools to a minimum so as to be able to produce more value and to be able to adapt to changes throughout the project. The goal is giving value, not following a process. How light it could be depends on the circumstances. Bigger projects require heavier methodologies as there is a bigger need of coordination. However, the principle doesn't change: keep it as light as possible, but still sufficient.

Questioning Scrum

I have been working with Scrum for the past 4 years. I love it. It was my introduction to the Agile world and It is so much better than what I did before. Now I am in a moment where I am questioning some things. I attended a couple of conferences about Kanban (so I am far from expert), but it was enough to plant some seeds of doubt about the things I do with Scrum. Below are some examples

Cross-functionality

- Scrum states that cross functional teams are needed, but the way to enforce it usually fails. Having iterations with stories that correspond to each area of specialization 'suggests' specialists for each story. I have struggled with many teams to be more cross-functional, to pair and to spread the knowledge, but Scrum doesn't give the tools to do that per se. On the other hand, Kanban with its 'minimize work in progress' principle enforces cross-functionality in a much better way as stories are not pulled until one of the current ones is finished. This 'forces' developers to pair or just work on areas of the system where they are not so comfortable.

Attack the risks

- Previous point also applies to tacking the risk first. There's usually the case in a scrum iteration that individuals choose the tasks they are more familiar with or they feel more comfortable with (yes, I do it all the time). Scrum 'sort of' allows this. In a self-organized team, team members decide which task they want to work with. How can we be biased to choose the task where the bottleneck is or the task that is representing the highest risk?. If there is no option other than working in the bottleneck/risky task, we don't have any other choice.

Sprints:

- Having iterations of equal size forces the team to break up the functionality in chunks that fit into a Sprint and also forces closure during the iteration to be able to finish them. A cadence is created as iterations go by and there's usually a sense of urgency because there is a lot of work to finish a story end to end. This is really nice but there are things that I don't like about these sprints. Imagine, we, the team 'commit' to a certain amount of work. When we are too careful with the estimations, all work is finished either 2/3 days before, and we (usually prefer to) relax and wait until the next sprint. What happens if we over-commit? We usually works harder to try to finish (which is not that good) and if we don't finish and the story is rolled, that story stretches in the next sprint in unthinkable ways!!. No scenario seems to be perfect. Personally, I prefer more stability. Committed and motivated people working 8 hours a day to complete the work. Does this mean stories should not be estimated? Well, I think in highly exploratory projects, stories should still be estimated even if using Kanban and the team should strive for closure in the estimated time. But the sprint fixed size sometimes is an artificial constraint that creates certain dysfunctions.

Iteration Plannings:

- I have been in many iteration planning where stories are 'guesstimated' because there is not enough information at that point (in 3 week iterations, that is entirely possible). Teams are encouraged not to pad, but when there's too much uncertainty, team members fall into this. Estimating the story when it's pulled allows in many cases to have more information, to avoid assumptions and makes this meeting faster and more productive. Probably, this is a case of 'Decide as late as possible'. For example, if one story depends on the another and many of the uncertainties will be cleared when then first story is completed, why spending so much time to speculate in estimating the second? (we do this very often in Scrum, don't we?)

- Iteration Plannings tend to be to large. And sometimes boring. Scrum suggests a ratio and the meeting should be timeboxed. If there's uncertainty in some of the stories, this could get worse. Who hasn't been in endless debates that later translate into a short 2 hours research that sheds all the necessary light? I guess this point is very related to the previous. A crisis of faith in estimating things I don't understand.   

Scrum taskboards versus kanban boards

- Kanban boards provide more information that Scrum boards. They allow to visualize flow and discover bottlenecks. Many times when working with Scrum I find my self looking at both the taskboard and the burndown to give me a real sense of the progress. With scattered tasks all over the board, usually I don't understand where we are.

- Be able to spot the bottleneck and provide an impersonal way of pushing the team is great. Constraining work in progress in certain phases forces the whole team to focus on the most critical things that will make the iteration successful. And this is not intuitive. Personally, If I looked at a scrum task board, I usually pick the tasks I am more familiar with even if they are not the highest priority or the ones that minimize the risk in the iteration. I don't think I am different than the majority :-)

Daily Standups:

- Something I read recently (and I liked) is Kanban teams do the standup looking at the board instead of looking at each other. I like that idea. I believe it foster collaboration in a less personalized way. Let me explain: sometimes I think Scrum daily standups put too much pressure on the people that are going slower or are stuck. People is looking at each other and sometimes there could be managers (chicken, but managers). I believe this could cause a dysfunction: lying about the status. Or even worse. Try to finish faster, dropping quality. Focusing on the board switches the focus from a person stuck to a task stuck. It's subtle. I like it.

All in all, I think kanban brings values that would make a team better. Minimize work in progress (and thus reduce the size of the increments), defer decisions until the last reponsible moment (estimate when you have all the needed information), focus on the problems and help resolve them (as opposed to help someone resolve it). I never used Kanban in practice. But I am at a point that with what I read/hear, I can spot moments or situations where we are not doing things in the best way with Scrum. I am not saying these are reasons to justify moving from Scrum to Kanban. And I am not saying these are shortcomings of Scrum. These are just some questions that arise from knowing that there are other ways of doing things. And I believe this questions are helping me improve how we do things now.

Agile is Self-Organization and Self-Discipline

Agile teams are self-organized teams that work in an environment that has certain constraints to achieve certain goals. A self-organized team lacks a central authority. Instead, members self-organize and lead themselves to achieve the goals.

To understand how Self-Organization works, we can start reading about "Complex Adaptive Systems". In complex adaptive systems, the solution emerges. The term "emergence" means that "Rather than being planned or controlled the agents in the system interact in apparently random ways. From all these interactions patterns emerge which informs the behavior of the agents within the system and the behavior of the system itself." Complex Adaptive Systems accurately describe a software development ecosystems. Members in this ecosystem usually take every day decisions that impact the final product and they interact with other members to do that.

Self-Organization doesn’t mean anarchy or sloppiness. On the contrary, Agile self-organized teams are very disciplined. This discipline is the one that is borne because of the high standards of the group and because no one wants to disappoint their peers and not because of rigid orders of a manager.

Self-organization needs to happen within the limits set by the organization and guided by leaders. Managers still need to set the direction, the constraints and the guidelines that the team needs to follow. It is also important to have leaders within the team to coach and spread the good practices. 

Why?

Which are the reasons that justify having this structure instead of a more traditional command and control structure. This is a small list of those:

Self Organized teams take better decisions: The fifth lean principle in the Poppendieck's "Lean Software Development" book is Empower the team. Empowered and educated employees guided by a leader are motivated employees that can take better decisions that their managers. Scientific Management, as stated by Taylor, was designed with another type of employee in mind.. very different from the software engineer. In today, knowledge-based organizations, delegating power down (empowering) has proven to be much more effective because the engineers working at the front line have much better information and because they have the knowledge to take those decisions. 

Individuals in a Self-Organized team are motivated: Motivation comes from participation. Being part of the solution… Something to keep in mind when working with knowledge-based workers is they are highly educated people. Therefore, motivation not only comes from the salary or the position, but also by being able to do something that satisfies them.  

Self-Organization improves trust and commitment: Knowledge-based organizations are totally dependent on the commitment and ideas of their employees. Knowledge is a resource locked in the human mind. As the Novel laureate economist  Friedrich Hayek has argued, "Practically every individual... possesses unique information" that can be put to use only "with his active cooperation". 

Self-Organization enables innovation: Trust and commitment makes people go beyond their call of duty by sharing their knowledge and applying their creativity. The synergy created in the interaction from individuals with this attitudes is one of the key enablers to build innovation. Contrast this to other structures were employees are just told what to do. In this case, what the person can contributed is much more bounded. In Self-Organized groups, innovations thrives.

The best solutions emerge from self-organized teams: last but not least, let’s mention one of the principles in the Agile Manifesto. I completely agree. A self-organized solution contains a mix of different backgrounds on it. There is no comparison between a solution thought by just one person and one reached by a group that went through different options and agreed on the best one.

Managers to Leaders

If members are not told what to do, you probably would wonder what happens with manager. There is a shift in the way management is performed. Managers become leaders in Agile. They don't tell its employees what to do. They fix the goals and set up the constraints to get there. And they lead by example. Lissa Adkins states in her article "The Manager Role's in Agile" that "agile managers coach, inspire, and lead teams more than they measure and manage them."

Wait a minute.. Does this mean managers don’t take decisions anymore? I had this doubt for a lot of time, until I read Highsmith “Agile Project Management” book. The answer is “yes, they still need to take some decisions”. Even more. The group expects the leader to take some decisions. I’ve seen it many times. When the leader is respected, there are a lot of people that even prefers the leader to take the decision for them. But to reach this point, there should be respect. Lots of it.

I just read the paper “Fair Process: Managing in the Knowledge Economy” by Chan Kim and Renee Mauborgne. This paper explains very clearly how people respect and even commit to decisions they have not taken if and only if the process is “fair”. A fair process is transparent and known before the decision is taken and it allows for the people affected to at least expressing their points of views. The process is as important as the outcome. 

Conclusion

Self-Organization is one of the tenets of Agile. It improves trust, motivation, commitment and most of all, it improves the quality of the produced work. Managers and organizations used to old command and control structures need to change their beliefs and their culture. Agile leaders need to understand their new role, which is by no means easier than before.