Questioning Scrum

I have been working with Scrum for the past 4 years. I love it. It was my introduction to the Agile world and It is so much better than what I did before. Now I am in a moment where I am questioning some things. I attended a couple of conferences about Kanban (so I am far from expert), but it was enough to plant some seeds of doubt about the things I do with Scrum. Below are some examples

Cross-functionality

- Scrum states that cross functional teams are needed, but the way to enforce it usually fails. Having iterations with stories that correspond to each area of specialization 'suggests' specialists for each story. I have struggled with many teams to be more cross-functional, to pair and to spread the knowledge, but Scrum doesn't give the tools to do that per se. On the other hand, Kanban with its 'minimize work in progress' principle enforces cross-functionality in a much better way as stories are not pulled until one of the current ones is finished. This 'forces' developers to pair or just work on areas of the system where they are not so comfortable.

Attack the risks

- Previous point also applies to tacking the risk first. There's usually the case in a scrum iteration that individuals choose the tasks they are more familiar with or they feel more comfortable with (yes, I do it all the time). Scrum 'sort of' allows this. In a self-organized team, team members decide which task they want to work with. How can we be biased to choose the task where the bottleneck is or the task that is representing the highest risk?. If there is no option other than working in the bottleneck/risky task, we don't have any other choice.

Sprints:

- Having iterations of equal size forces the team to break up the functionality in chunks that fit into a Sprint and also forces closure during the iteration to be able to finish them. A cadence is created as iterations go by and there's usually a sense of urgency because there is a lot of work to finish a story end to end. This is really nice but there are things that I don't like about these sprints. Imagine, we, the team 'commit' to a certain amount of work. When we are too careful with the estimations, all work is finished either 2/3 days before, and we (usually prefer to) relax and wait until the next sprint. What happens if we over-commit? We usually works harder to try to finish (which is not that good) and if we don't finish and the story is rolled, that story stretches in the next sprint in unthinkable ways!!. No scenario seems to be perfect. Personally, I prefer more stability. Committed and motivated people working 8 hours a day to complete the work. Does this mean stories should not be estimated? Well, I think in highly exploratory projects, stories should still be estimated even if using Kanban and the team should strive for closure in the estimated time. But the sprint fixed size sometimes is an artificial constraint that creates certain dysfunctions.

Iteration Plannings:

- I have been in many iteration planning where stories are 'guesstimated' because there is not enough information at that point (in 3 week iterations, that is entirely possible). Teams are encouraged not to pad, but when there's too much uncertainty, team members fall into this. Estimating the story when it's pulled allows in many cases to have more information, to avoid assumptions and makes this meeting faster and more productive. Probably, this is a case of 'Decide as late as possible'. For example, if one story depends on the another and many of the uncertainties will be cleared when then first story is completed, why spending so much time to speculate in estimating the second? (we do this very often in Scrum, don't we?)

- Iteration Plannings tend to be to large. And sometimes boring. Scrum suggests a ratio and the meeting should be timeboxed. If there's uncertainty in some of the stories, this could get worse. Who hasn't been in endless debates that later translate into a short 2 hours research that sheds all the necessary light? I guess this point is very related to the previous. A crisis of faith in estimating things I don't understand.   

Scrum taskboards versus kanban boards

- Kanban boards provide more information that Scrum boards. They allow to visualize flow and discover bottlenecks. Many times when working with Scrum I find my self looking at both the taskboard and the burndown to give me a real sense of the progress. With scattered tasks all over the board, usually I don't understand where we are.

- Be able to spot the bottleneck and provide an impersonal way of pushing the team is great. Constraining work in progress in certain phases forces the whole team to focus on the most critical things that will make the iteration successful. And this is not intuitive. Personally, If I looked at a scrum task board, I usually pick the tasks I am more familiar with even if they are not the highest priority or the ones that minimize the risk in the iteration. I don't think I am different than the majority :-)

Daily Standups:

- Something I read recently (and I liked) is Kanban teams do the standup looking at the board instead of looking at each other. I like that idea. I believe it foster collaboration in a less personalized way. Let me explain: sometimes I think Scrum daily standups put too much pressure on the people that are going slower or are stuck. People is looking at each other and sometimes there could be managers (chicken, but managers). I believe this could cause a dysfunction: lying about the status. Or even worse. Try to finish faster, dropping quality. Focusing on the board switches the focus from a person stuck to a task stuck. It's subtle. I like it.

All in all, I think kanban brings values that would make a team better. Minimize work in progress (and thus reduce the size of the increments), defer decisions until the last reponsible moment (estimate when you have all the needed information), focus on the problems and help resolve them (as opposed to help someone resolve it). I never used Kanban in practice. But I am at a point that with what I read/hear, I can spot moments or situations where we are not doing things in the best way with Scrum. I am not saying these are reasons to justify moving from Scrum to Kanban. And I am not saying these are shortcomings of Scrum. These are just some questions that arise from knowing that there are other ways of doing things. And I believe this questions are helping me improve how we do things now.

Agile is Self-Organization and Self-Discipline

Agile teams are self-organized teams that work in an environment that has certain constraints to achieve certain goals. A self-organized team lacks a central authority. Instead, members self-organize and lead themselves to achieve the goals.

To understand how Self-Organization works, we can start reading about "Complex Adaptive Systems". In complex adaptive systems, the solution emerges. The term "emergence" means that "Rather than being planned or controlled the agents in the system interact in apparently random ways. From all these interactions patterns emerge which informs the behavior of the agents within the system and the behavior of the system itself." Complex Adaptive Systems accurately describe a software development ecosystems. Members in this ecosystem usually take every day decisions that impact the final product and they interact with other members to do that.

Self-Organization doesn’t mean anarchy or sloppiness. On the contrary, Agile self-organized teams are very disciplined. This discipline is the one that is borne because of the high standards of the group and because no one wants to disappoint their peers and not because of rigid orders of a manager.

Self-organization needs to happen within the limits set by the organization and guided by leaders. Managers still need to set the direction, the constraints and the guidelines that the team needs to follow. It is also important to have leaders within the team to coach and spread the good practices. 

Why?

Which are the reasons that justify having this structure instead of a more traditional command and control structure. This is a small list of those:

Self Organized teams take better decisions: The fifth lean principle in the Poppendieck's "Lean Software Development" book is Empower the team. Empowered and educated employees guided by a leader are motivated employees that can take better decisions that their managers. Scientific Management, as stated by Taylor, was designed with another type of employee in mind.. very different from the software engineer. In today, knowledge-based organizations, delegating power down (empowering) has proven to be much more effective because the engineers working at the front line have much better information and because they have the knowledge to take those decisions. 

Individuals in a Self-Organized team are motivated: Motivation comes from participation. Being part of the solution… Something to keep in mind when working with knowledge-based workers is they are highly educated people. Therefore, motivation not only comes from the salary or the position, but also by being able to do something that satisfies them.  

Self-Organization improves trust and commitment: Knowledge-based organizations are totally dependent on the commitment and ideas of their employees. Knowledge is a resource locked in the human mind. As the Novel laureate economist  Friedrich Hayek has argued, "Practically every individual... possesses unique information" that can be put to use only "with his active cooperation". 

Self-Organization enables innovation: Trust and commitment makes people go beyond their call of duty by sharing their knowledge and applying their creativity. The synergy created in the interaction from individuals with this attitudes is one of the key enablers to build innovation. Contrast this to other structures were employees are just told what to do. In this case, what the person can contributed is much more bounded. In Self-Organized groups, innovations thrives.

The best solutions emerge from self-organized teams: last but not least, let’s mention one of the principles in the Agile Manifesto. I completely agree. A self-organized solution contains a mix of different backgrounds on it. There is no comparison between a solution thought by just one person and one reached by a group that went through different options and agreed on the best one.

Managers to Leaders

If members are not told what to do, you probably would wonder what happens with manager. There is a shift in the way management is performed. Managers become leaders in Agile. They don't tell its employees what to do. They fix the goals and set up the constraints to get there. And they lead by example. Lissa Adkins states in her article "The Manager Role's in Agile" that "agile managers coach, inspire, and lead teams more than they measure and manage them."

Wait a minute.. Does this mean managers don’t take decisions anymore? I had this doubt for a lot of time, until I read Highsmith “Agile Project Management” book. The answer is “yes, they still need to take some decisions”. Even more. The group expects the leader to take some decisions. I’ve seen it many times. When the leader is respected, there are a lot of people that even prefers the leader to take the decision for them. But to reach this point, there should be respect. Lots of it.

I just read the paper “Fair Process: Managing in the Knowledge Economy” by Chan Kim and Renee Mauborgne. This paper explains very clearly how people respect and even commit to decisions they have not taken if and only if the process is “fair”. A fair process is transparent and known before the decision is taken and it allows for the people affected to at least expressing their points of views. The process is as important as the outcome. 

Conclusion

Self-Organization is one of the tenets of Agile. It improves trust, motivation, commitment and most of all, it improves the quality of the produced work. Managers and organizations used to old command and control structures need to change their beliefs and their culture. Agile leaders need to understand their new role, which is by no means easier than before.

Can estimations be used to push the team? How?

In all teams, there are guys that are really performant, there are average and there is people that are slower (be it because of skill reasons, personal reasons or whatever). When tasks are estimated in ideal hours using a technique similar to planning poker (where everyone contributes), the result should be an estimate that falls into the average, right? To be more clear, if the task is taken by a fast member, probably it will be finished before the estimated time. If it's taken by an average, it would finish around the estimated time and if it's taken by a slower member, it will take more (probably much more) that what was estimated.

So these estimations should give us an idea of where we are in the Sprint, right? The question is: can we use them to 'push' the team? If they are used to push the team, how can it be done without hurting the sensibility of anyone and without causing people fear of taking the most difficult tasks?

These are some options:

- Asking the individual/s that hold the task why is taking so long in the standup.. This is brutal and I don't see any possible benefits (probably people will be terrified of taking the most difficult tasks very rapidly)
- Asking the whole team why certain tasks are taking longer. Much better cause there is depersonalization of the problem.
- Try to make the problem more visible. Using information radiators, everyone should be able to see the problem and pitch in to try to solve it.
- Minimize work in progress. This is another, more effective way, to make the problem visible. If no new tasks can be pulled because there is something stuck, the group as a whole will try to un-stuck it. Completely depersonalized and very effective. Wait... I am not using estimations this way :S

Ultimately, Agile relies on a group of motivated and committed individuals. If the speed at which the group is going is not enough to reach the objectives, it is a problem of all the team: managers, product owners, developers, etc. The group as a whole needs to resolve this problem. They need to figure out how to go faster. If the group shares the responsibility this way, they will find the best way to overcome it. However, there is a basic requisite that the group must hold: There should exist trust in the group. It should exist the trust that allows you to tell the group that you are stuck and need help without been judged. There should exist the trust that allows to treat the problems as problems of the team.

Comments???

Agile is collaboration

I categorized the collaboration component as Human because I believe it deals with human characteristics. Funny thing.. What do I know about humans? :-) Seriously.. not too much. I went to a high school with a focus on science. I studied Computer Science and I work all my life with developers!!

When I was writing it, I started doubting about having 2 different posts: one for customer collaboration and one for collaboration within the team. I think the Agile Manifesto talks only about customer collaboration, but I think the Agile community shares the feeling that team collaboration is equally important to success and makes for a better workplace. Do you agree?

What is collaboration?

To collaborate is to work jointly towards a shared objective, to help each other, to have good intentions, to share all the information and ultimately to do what's best for the team as a whole.  

Endless projects have failed in the past because of lack of user involvement. We used to believe that it was possible for the user to order what he needed and collect the product once it was finished. So, the user was involved at the beginning, defining the requirements and in the end, when the product was already built. Of course, in lots of situations, the user may have not liked what he was receiving but making changes at that point was very expensive. Therefore, the user was forced to specify correct and complete requirement definitions at the beginning of the project. Analysts and developers were 'forced' to understand these specifications which, as we all know are very difficult to do.

The Agile approach is very different as it tries to incorporate the customers throughout the creation of the product. The Agile Manifesto says "Business people and developers must work together daily through the project". So the customer is of course involved defining the requirements, but they also collaborate with the developers in their analysis and provide feedback with each increment of software. There's close collaboration between developers and the customer during the whole project. Customers and developers work together towards a share objective.

Agile collaboration does not end with customer collaboration. Collaboration needs to happen inside the team as well. Agile team members help each other, share all the information and strive for team success.  Agile builds on a team, not on isolated individuals. In a collaborative software development team, members contribute their efforts to build something bigger than themselves.

How is collaboration achieved?

The first requisite to enable customer collaboration is have a contract that allows the customers to be involved in the development process. For the business people, it may be difficult to understand that they will need to 'lose' their time working with developers to build the system. Besides this, having a business person involved in the project will probably generate changes as the project progresses. If there's a contract signed for a defined scope, it may be difficult to apply these changes as they may imply changing the conditions of the agreement. So the contract should be one that welcomes these changes.

Once changes are 'financially' feasible, it should be technically feasible. Agile relies on incremental development to be able to incorporate the customer in the creation process. That is build a bit, show it to the customer, get feedback and keep building while incorporating this feedback. This collaboration that happens in the construction of each feature helps build what the customer is expecting. Building software incrementally is not enough. For this to be sustainable, the cost of change of each new increment should not increase exponentially. If it did, those changes suggested by the customer wouldn't be welcomed at all.

How is collaboration achieved inside the same organization? Well, it may be very difficult to achieve as it deals with human nature. In any organization, there may be conflicts of interests as the interests of the organization are not exactly the same as the interests of its individuals. Cockburn says there are many "games" in place. The organization is playing its game where the project it's just another move. The individuals that work in the organization play another game which is their career. Personal interests may undermine the sucess of a project, but there are things an organization can do to improve collaboration:

Definition of Success or Failure

Either the team succeeds or the team fails. With this definition, there is no place for individualistic behaviors. What's best for the team is what's best for everyone. Individualistic behaviors don't help the team succeed. This definition of success encourages everyone to act in a collaborative way.

Productivity Rewards in the Organization

This point is very related to the previous one, but I think deserves to be mentioned separately. How does an organization rewards its employees? If rewards are made on an individual basis, employees will try to demonstrate their work and they will try to excel over everyone else. This type of reward encourages employees in selfish behaviors and diminish collaboration. I think this point is very interesting as I've seen many organizations doing Agile that have incompatible rewards schemes.

One question that I often asked myself is if a medium that favors collaboration could be unfair with high performers. Someone that works hard and also helps his/her team-mates a lot could feel his/her work is disguised under this scheme. Someone that doesn't work as hard could be 'hidden' under team goals. Probably the only way for managers to verify who works hard and who doesn't, who is talented (and ultimately who deserves a raise!) is to live in the day to day of the team. High performers Agile developers are those who help the team succeed and if you live the day to day with the team, they are easily noticeable. 

Attack dysfunctions

Patrick Lencioni describes in his book "The Five Dysfunctions of the Team" the dysfunctions that a team must overcome to be able to work well as a group.

Image taken from: http://www.thepracticeofleadership.net/2007/03/21/book-review-the-five-dysfunctions-of-a-team/

The first dysfunction is Absence of trust. Lencioni says 'this stems from their unwillingness to be vulnerable within the group'. Well.. It's hard for anyone to expose its vulnerabilities. Even more when your seniority is high. However, if team members don't trust each other, they don't ask for help and they hide information. Ultimately, they do what's best for them instead of doing what's best for the group. On the other side, teams where members trust each other are much more healthy. They ask for help, they help each other and they provide all the necessary information to anyone.

Failure to build trust generates Fear of conflict. If members don't trust each other, they don't engage in constructive arguments. Teams that cannot engage in arguments where everyone expresses their honest opinions lose much of the team's power.

Failure to engage in constructive arguments becomes the greatest cause of members not committing to the decisions taken. If members were not able to express their opinions freely and discuss them with their peers, they don't buy into what has been decided.

Members that don't commit are members that are not accountable. How can you ask a member that hasn't committed to a plan to follow it, right?

Finally, non accountability creates an environment where the five dysfunction can thrive: Inatention to results.

This book is pretty amazing. Once you read it, you begin thinking about those attitudes that were hard to understand when you just started working. For example, replying with a copy to the boss. Or you start thinking about those guys you used to admire: I used to say "this guy, I don't know what he does, but he is so politically smart!!". That sucks men... I love other attitudes now. Those that encourage transparency and bonding in the team. For example, saying in the standup 'Yes, I screw with this decision.. I apologize' or 'I really tried to make it work, but coudn't. I need help' and I also like the guys that step and say 'I will help you'. This makes a team.

Conclusion

I like seating, thinking and trying to order my ideas to write a post. I must confess I was not making the difference between customer collaboration and team collaboration. I was putting everything in the same bag. Now I think making the difference makes sense. I think the Agile Manifesto deals with customer collaboration, but I see no explicit mention to collaboration within the team in it.

Customer collaboration is of utter importance to the success of the project. A customer, someone that knows and understand what is needed, throughout the project increases exponentially the chances of success. Collaboration needs to be enabled, both at a contractual level and at a technical level.

Team members that collaborate with each other make the other half. There is nothing like a health environment for people to reach their highest potential.   

There's also a side effect benefit. Collaboration over contracts makes the process lighter and thus centered in value. I believe there are some processes or quality standards that suggest contracts to supply lack of trust. If you don't trust the other part, you want a signature to be assured that the other part is committed to do something. In contrast, if you trust the other part, you just need his acknowledgement of understanding.