Son las estimaciones tan malas?

Una de las charlas que mas me gusto del Bolivian Scrum Day 2013 es la de Thomas Wallet "No estimarás" - Mensaje fuerte, no? Y creo que esta bueno para mover el bote (como dicen en 10 Pines), pero, como siempre, tenemos que tomar el mensaje con cuidado y no echarle la culpa a la herramienta, como escuche alguna vez decir a Henrik Kniberg. Es estimar realmente malo? Existen muchas situaciones donde claramente no lo es. Si yo tengo que tomar un vuelo en Viru Viru y quiero saber cuando es que debería estar en el aeropuerto, realizo una estimación de cuanto voy a tardar y cuando tendría que levantarme. Hablando de software, muchas veces necesito o me es útil tener una 'estimación' del costo de una feature (para saber si me va a alcanzar la plata - poder planear -  o para saber si las voy a construir o voy a construir otra - poder priorizar -). Entonces cuando la estimación es mala o se convierte en un problema?

- Si lo voy a hacer si o si: tiene sentido estimar algo que si o si tengo que completar? Piensen en sus tareas personales? Se ponen a pensar cuanto van a tardar en hacer algo si lo tienen q hacer si o si? Perder tiempo en estimar en estos casos es realmente eso, perderlo.

- El otro gran problema con las estimaciones es cuando se toman como un contrato y no se tiene en cuenta que son estimaciones!! Y que es díficil estimar. Y que mientras menos se entienda el problema, peor se va a estimar!!

Entonces, yo que soy un poco más moderado propongo que entendamos porque estamos estimando y que tengamos en cuenta el retorno de inversión de esta estimación. Que quiero decir con esto? Si de esta estimación no depende absolutamente nada importante de la empresa, no perdamos mucho tiempo!! (y tampoco perdamos el tiempo haciendo tests). Si por el contrario, para la empresa es muy importante, entonces 'invirtamos' más tiempo en la estimación. Preocupemosnos un poco más!!

Otro de los puntos que me llamó la atención de la charla es que el modo talibán de dejar las estimaciones es básicamente intentar tener items de mas o menos el mismo tamaño y abandonar las iteraciones, midiendo el throughput y el ciclo... Esto también me hace ruido. Yo no creo que este cambio en el ciclo de vida del proceso este relacionado con las estimaciones y si con la no utilidad de las iteraciones. Este punto me va a ser mucho más díficil de fundamentar (será porq no lo tengo tan claro), pero voy a intentarlo. Tener items del mismo tamaño siempre es bueno, en Scrum, Kanban, XP o donde quieras. Permita calcular cualquier métrica mas facilmente, permite encajar los items dentro del sprint backlog, etc. Otro punto (no relacionado, pero bueh.. lo tiro!): Samuel Crescencio dijo que en su opinion, las retrospectivas son el componente mas importante de Scrum (quizás lo dijo para exagerar la importancia de las retrospectivas..). Bueno, pues para mi el componente más importante de Scrum son las iteraciones (timeboxedas). Por que? Porque las iteraciones son las q mantienen el foco, crean el pulso del proyecto, permiten calcular la velocidad y son el enabler de la inspeccion y adaptación. Adonde voy con todo eso? (ni yo se). A q si queres dejar de estimar usando Scrum podes hacerlo. Simplemente pone un numero (aleatorio) de ítems en una iteración y dps. ajustalo en las iteraciones subsiguientes (si tenes ítems del mismo tamaño, esto debería ser muy fácil). Pero debe haber otras razones mas importantes para eliminar la iteración (o para no hacer iteraciones o Scrum si nunca hiciste). Mucha variabilidad en la capacidad del equipo.. Un equipo de desarrollo que comparte responsabilidades de producción.. hasta podría enumerar una diferencia grande en el tamaño de los ítems (haciendo q algunos no entren en la iteración), pero partimos de la base q el tamaño de los ítems es similar..Entonces quizás es a mi q se me mezcló un poco la ultima opción o no se que, pero me cuesta ver la opción de ítems del mismo tamaño + Kanban como una opción para dejar de estimar. Para mi, dejar las iteraciones tiene que tener que ver con otra cosa.


En conclusión creo que las estimaciones sigue siendo buenas, cuando se las usan para lo que son y cuando existe confianza entre las gente que realiza las estimaciones y la gente que las va a usar. Desde que leí a Alistair Cockburn, siempre dije que creo q tenemos que ser críticos y entender el porque de cada uno de los componentes/tecnicas que usamos en nuestra metodología para tener la metodología más liviana que resuelva nuestros problemas (componentes que no aportan valor o son mal usados son desperdicio). Las estimaciones pueden ser un componente que agrega valor o no. Evaluemosló.

Daré una charla en el Scrum Bolivia Day 2013!!

Si, estoy muy contento porque a fin de mes tendré la oportunidad de asistir al Scrum Bolivia Day en Santa Cruz de la Sierra, un lugar que hacer mucho tiempo quería ir! Gracias Juan por la invitación y nos vemos pronto!!

Descripción

La revolución ágil introdujo grandes cambios en todas las áreas del desarrollo de software y el area de management no fue una excepcion. Pero cuales fueron estos cambios? Cuales son los desafíos del manager en la era ágil? En esta charla abordaremos estos tópicos con el objetivo de entender mejor las responsabilidades de un manager ágil y brindaremos herramientas que los ayudaran en sus desafíos diarios. Entre los puntos incluidos están: teoría de sistemas adaptativos complejos, como crear equipos motivados y como delegar responsabilidades.

Agile Knowledge Repository

Summary

This area deals with knowledge acquisition and knowledge management in an organization. One of the main points of CMMI is to maintain a process repository (as expressed in the Organization Process Definition Area) that enables the organization to acquire and maintain its knowledge. Agile Organizations should maintain this repository, by creating an Agile Culture (I expressed my doubts about being able to create a documented knowledge repository in an agile software organization, as I think software development is a craft). Therefore, processes and techniques live inside the people's brains of the organization and is acquired and maintained by practicing and teaching those process and techniques. Knowledge is maintained using the model used in the past by guilds.

Goals

- There is a knowledge path that people follow inside the organization, starting from apprentice (someone that is learning, does exercises, pair programs with someone and has a lot of time to educate himself) to Journeyman (which is someone that knows the craft and is able to teach it).
- Knowledge is shared openly and collaboratively inside the organization. Culture is strong. More senior software craftsmen lead and teach the rest of the group on "how we do things inside here". Values float in the air.

Standard Practices

I have taken these practices from 8thlight, a company I visited a few months ago. In 8thlight:

- Employees start as apprentices. Their job at that moment is to learn how to develop software and how the company manages their projects. They do exercises and pair programming with more senior craftsmen.
- Software craftsmen have one or more apprentices in charge. So their responsibility is to teach apprentices the processes and techniques used inside the organization.
- Knowledge is shared and maintained by practicing together. Employees do code retreats or katas together. The philosophy is programmers need to practice their craft and this practice is performed on code that is not business oriented.

Presentation in Agiles 2012

Transforming Cultures on Prezi

Agile Collaboration Area

Summary

The Agile Manifesto says "Business people and developers must work together daily through the project". Everyone, within the organization and beyond it (i.e. clients) must work jointly towards a shared objective, help each other, have good intentions, share all the information and ultimately to do what's best for the everyone involved.

Goals

- Collaboration between the organization and its clients is enabled financially. Contracts should enforce collaboration.
- Collaboration among team members in a project and among different departments of the organization should be enforced. 
- All stakeholders should be involved throughout the duration of a project.

Standard Practices

Clients work with team members to specify the requirements and prioritize them in a backlog that will be used continually during the project. Features are presented to these clients early in the life of the project to be able to receive feedback and change according to it. This change should be enabled in the contract (if the contract penalizes delays, the team won't be willing to accept any changes during the life of the project). Everyone in the team should share the same objectives. Therefore, policies and rewards should be devised with this in mind.

Agile Communication Area

Summary

Communication is key in an Agile organization. Developing software is a communication problem. Communication needs to flow as fast and clear as possible among all members of the organization. An environment that enables and enforces communication should be built.

Goals

- There is an environment that allows interactions among all members in the organization freely.
- All members of the organization can easily  communicate with each other.
- There is complete transparency to everyone about the goals, status, direction of the project/organization.

Standard Practices

- The office is an open space, where all members share the same information. There are visual radiators in many places of the office that allow members of a project and all members in the organization to understand the status and direction of the different projects.
- Team members in a project interact as much as possible to synchronize and make sure there is a shared understanding of what needs to be built. In Scrum, there are different meetings that enable and enforce this communication: 1) Planning meeting at the beginning of each iteration 2) Daily Standup every day 3) Demo and Retrospection at the end of the iteration. Besides this, practices like pair program or code review sessions push the members to interact even more. Understanding the effectiveness of the different types of communication types helps devise a communication strategy. When is possible, face to face communication is the most effective type of communication but there are situations when the team is distributed geographically and therefore other communication methods should be used. Increasing the communication to its highest level should be the direction in these cases.

Self Organization Area

Summary

This area's goal is to create an environment where all members feel empowered to contribute all their knowledge and energy for the organization's benefit. Leaders and more experienced individuals should guide less experienced ones and set constraints and directions.

Goals

- Team members are empowered
- The team self organizes to create the best solution to the specified problems.
- Leaders create and maintain an environment where self-organization can thrive

Standard Practices

- Within a project, a cross functional team self organizes to develop the best solution to the problem.
- Managers are servant leader that teach through example and help other members to grow within the organization and to give the best of them.

Agile Process Development Lifecycle Area

Summary

The process should be an enabler of the objectives the organization needs to fulfill. It should allow the organization to deliver value fast and sustainably. It should be revised and improved continually (Kaizen).

Goals

- The process enables incremental delivery of features. In other words, the process should allow to slice the functionality to be delivered in multiple chunks, that when delivered allow the business to receive the most valuable/important functionality sooner.
- Process should be light and empiric. All artifacts in the process should give value to the product creation process.
- Process should be empiric. The team should continually introspect on the process and change it to function better within the specific context.

Standard Practices

- Common methodologies/frameworks like Scrum and Extreme Programming use iterations to deliver features incrementally and iteratively. Features should be potentially shippable at the end of each iteration. This doesn't mean that they could be shipped, as a feature could be part of a larget minimum viable feature.
- At the end of each iteration, the team can show to all business stakeholders the business functionality that was completed. If requirements are sliced vertically and provide business value, they can be shown.  Stakeholders can provide their feedback, which could be incorporated in the product backlog through the product owner.
- The team should inspect the process (among other things) at the end of each iteration. Usually, the team tries to find what is working with the current process and isn't, devising action items that can improve its performance.

Agile Project Monitoring Area

Summary

All members of the organization are in charge of monitoring the progress of the project, suggesting corrective actions and making sure they are completed. There is complete transparency and a lot of communication among everyone, thus ensuring that everyone understand the current status of the project.

Goals

- The team monitors the actual progress of the project at timeboxed intervals. It should be measured the actual completion of requirements, risks tackled and discovered, changes in business requirements (i.e. changes in scope), etc.
- The team monitors the progress daily, trying to discover blockers (things that prevent the team from doing progress) and synchronization points (e.g. among dependent requirements)
- The team and business stakeholders agree on corrective actions or adaptions to the plan and execute them.

Standard Practices

- After each iteration, there is a demo that allows the team to show the business side the actual progress made on requirements. There is a great deal of knowledge won in this meeting as the business people get a feeling of the actual product being constructed. This knowledge gained should be reflected in the plan, as the horizon should be clearer after each iteration. The team should have also gained a clearer understanding of what it takes to develop the requirements and the amount of work remaining.
- Daily, the team introspects on the progress (compared to the expect one) and blockers trying to complete all work they have committed to and remove all impediments as fast as possible. In Scrum, this meeting is called "daily scrum" and it is usually the ScrumMaster who is in charge of removing any impediment that might prevent the team from doing progress.
- Each action item detected during either the daily scrum or the review should be completed to end. It is a good practice for one of the team members to be hold accountable.
- Visual Radiators should be hanged in various places of the open space the team works on. One of the tenets of Agile is transparency, and therefore everyone should be as aware as possible of the actual progress.

Agile Project Planning Area

Summary

Initial planning sets up the initial direction, which is continually updated based on new discoveries (business and technical). Therefore, planning is a continuous work that endures the whole project.

Goals

- A plan is built when the project starts and updated it continually throughout the project.
- Estimations are performed for all requirements. The time invested in estimation should be directly proportional to the level of approval and understanding of the requirement (A greater lever of understanding and approval enables a greater level of detail on the estimation)
- All involved members should be involved in estimations, aware of the plan and committed to the success of the project.

Standard Practices

- In the 'Release Planning' meeting, a backlog that contains all stories known so far is constructed. As there is a great deal of uncertainty and unknowns, stories are estimated roughly, to have a first approximation to the dimension of the project.
- If the project is too big, it is a common practice to slice the project in releases. First release should contain the most important business functionality, what is needed more urgently or what lets the company enter a field.
- During the execution of the project and as more is learned about the project, estimations and forecasts are updated to reflect the gained knowledge in the plan.
- A common technique for doing the estimation is the Poker Planning, where all members of the project contribute their wisdom (this technique relies on the Wisdom of Crowds)
- Involvement and transparency enable full commitment of the team.

Agile Requirements Management Area

Summary

Requirements are built by business owners and technical members and refined through all the duration of the project.They should be prioritized based on their value and technical risk to maximize the probabilities of success of the project.

Goals

- At all times, a project backlog is maintained with all requirements prioritized by their business values and technical risk. This backlog might have different levels of understanding and approval. The highest priority requirements should have a greater level of detail, understanding and approval.
- The backlog should be easy to maintain and light. The product owner should be able to modify and re-prioritize it at all times.
- All requirements should have a set of acceptance automated tests up to date at all times. If there are changes to the already implemented requirements, the acceptance tests need to be modified to reflect that.

Standard Practices

- A product backlog is created at the beginning of the project and maintained through out it. The main input should come from the product owners, who understand the business needs and the value of the requirements and from the development team, who should advise about the technical risks.
- A common practice is to write the requirements as user stories. A user story is written using the business lingo, it has an actor (someone that will use the system) and it provides some (ideally measurable) business value.
- Acceptance tests should be defined and validated with the product owner and implemented for all stories. These acceptance tests act as an scaffold that assures all requirements are being met at all times and also as a traceability tool.
- An Agile Management Tool is needed in case the project is considerably big and/or geographically distributed. This tool should allow anyone to make modifications on the stories and to reprioritize them.

Agile Maturity Model Categories & Areas

A time ago I did a post about the Agile Architecture which contains what I think are the most important 'components' (architecturally speaking) of Agile:

I believe this is a good start to define the areas that will make up this maturity model.

Process

In the process category, I will group all areas related to the process development lifecycle. I am tempted to call this category the methodology one, but the methodology encompasses much more elements than just the process ones. The methodology is everything you do in your organization to develop software (with this definition, methodology and culture are almost the same thing?). Some of the areas in this category include

- Requirements Management
- Process Development Lifecycle Management
- Project Planning
- Project Monitoring (or execution)

Human

- Self-Organization and Self-Discipline (I moved this to the human category in the original diagram)
- Communication
- Colllaboration
- Develop Competence
- Creating an Agile Culture?

Technical

- Technical Excellence
- Iterative Architecture and Emergent Design
- Verification & Validation
- SCM

First steps towards an Agile Maturity Model

I wonder how a model for maturing in an Agile context should be thought. Let's see some examples of previous attempts:

Agile Process Maturity Model

"The goal of the Agile Process Maturity Model (APMM) is to provide a framework which provides context for the plethora of agile methodologies and practices out there today. The APMM defines three levels, each of which build upon each other, for agile processes and practices."

APMM Level 1: Core Agile Development

Level 1 agile processes address a portion of the system life cycle (SDLC). Examples of Level 1 agile processes include Scrum, Extreme Programming, Agile Modeling and Agile Data.

APMM Level 2: Disciplined Agile Delivery

Level 2 agile processes extend Level 1 to address the full system delivery life cycle (SDLC). As the 
criteria for disciplined agile development suggests they also tend to dial up certain aspects of agile development, such as testing, measurement, and process improvement. Furthermore, they include explicit mechanisms to support effective (ideally lean) governance. Examples of Level 2 agile processes include Hybrid Processes (Scrum@XP), Rational Unified Process, Open Unified Process, Harmony ESW and Dynamic System Development Method.

APMM Level 3: Agility at Scale

In the early days of agile, the applications where agile development was applied were smaller in scope and relatively straightforward. Today, the picture has changed significantly and organizations want to apply agile development to a broader set of projects. Agile hence needs to adapt to deal with the many business, organization, and technical complexities todays software development organizations are facing. This is what Level 3 of the APMM is all about explicitly addressing the complexities which disciplined agile delivery teams face in the real world.

The goal of the APMM is "to define a framework which can be used to put the myriad agile processes into context. 

Agile Maturity Model

The AMM, "tries to quantify degree of responsiveness of a team to an organization's business needs. The philosophy being that increasing the discipline and proficiency of executing some best practices will increase the degree of responsiveness."

The AMM primarily focuses on application development activities; the dimensions of the AMM are

• Testing
• Source code management
• Collective code ownership
• Collaboration
• Responsiveness to business
• Assurance and governance
• Story formation
• Design simplicity
• Build process

For each of the above dimensions, the AMM lists behavioral patterns exhibited by teams. Teams can use the AMM as a reference to identify and compare the behavior that they currently exhibit and plan to alter and tailor them.

Testing

1. Acceptance testing in lieu of all other tests
2. No unit testing framework, ad hoc testing (Manual testing of individual components/ controls)
3. Unit testing, manual functional testing; application has a testable architecture
4. Unit testing integrated into build process, testing automated as much as reasonable given application
5. Developers write unit tests before writing functional code
6. Test are identified and produced as part of a story creation
7. Automated functional testing (e.g.. GUI testing); stories remain in development until all bugs are fixed or deferred

Source Code Management

1. Traditional schemes based on fire-locking and time-sharing
2. SCM supports version of code
3. IDE(s) integrate with SCM; developers include meaningful comments in commits
4. SCM support merging; optimistic check-ins
5. SCM support atomic commits
6. All development collateral is in SCM
7. SCM is transparent to delivery team, behind build process

Collective code ownership

1. Knowledge held by specific team members; people work in isolation
2. No pairing, people work alone, some informal process for keeping people informed
3. Pairing; no code locking
4. Pairing scheme ensures rotation
5. Team signs up for functionality rather than assignment to individuals
6. Within an sprint, functionality delivery is signed up for just in time
7. Old (bugs) and new functionality is queued, development team pops the stack in real time

Collaboration

1. Regular progress updates from management to the delivery team (as opposed to the other way around); irregular team meetings
2. Project collaboration tools (wiki, mailing list, IM) in place and used throughout project team; project status published and visible to all
3. Daily stand-ups and sprint meetings; problem solving is bottom-up as opposed to top-down; team sets sprint objectives and agrees on estimates.
4. Integrated, continuous build process, with build status notification to the team and collective responsibility for state of the build
5. Business is part of the team, stakeholders accept working software at reviews in lieu of other tracking or progress metrics
6. Frequent (near-real time) prioritization of old and new functionality
7. Build automatically deploys to QA environment available to any interested party

Responsiveness to business

1. Frozen specification, unresponsive to business value
2. Team tries to accommodate new requirements by ad hoc change requests ("pile it on")
3. Iterative process with sprints of length short enough to respond to business change
4. Showcases per sprint; business prioritizes functionality per sprint
5. Continuous involvement of the business on the team: business certifies story complete out of dev; involved in (rather than approval of) story definition and acceptance tests
6. Business writes high-level stories for all requests; organizational story queue exists; prioritization decisions made from full story backlog
7. Development process is integral to business initiatives; development teams work as a part of the business unit rather than as a service to the business unit

Assurance and Governance 

1. Status reports document progress; schedule acts as plan
2. Concept of release planning is introduced
3. Sprint planning is introduced
4. Plan becomes communication and tracking tool rather than an exception report
5. Every sprint review examines value delivered and assesses next sprint by need, priorities (and delivery)
6. Organizational adoption: Introduction of Portfolio Planning and global optimization of resources (use of metric and standard measures)
7. Business review of value and return helps teams on regular basis based upon status

Story Formation (Requirements)

1. Development tasks extracted from voluminous, frozen requirements artifacts
2. Production of lightweight artifacts (e.g. panel flows) to drive high-level requirements development
3. Mapping granular requirements (e.g., use cases) in high level user requirements
4. Marshaling use cases into discreet statements of functionality that can be delivered in time boxed development sprints
5. Stories are an expression of end-to-end functionality to be developed, including testable acceptance criteria and a statements of value
6. Spontaneous story development provided by business for in-flight projects; stories not derived from extant sources but are immediate expressions of customer demand/need
7. Global repository of functional requirements for stories developed by business for any business requirements, including a formal measure of business value delivered

Simplicity

1. Big up-front design that attempts to accommodate all potential future needs
2. Application of fundamental design patterns
3. Structural refactoring to decouple application architecture
4. Doing only what needs to be done
5. Aggressive and constant refactoring to improve code quality/simplicity extant code
6. Spiking solution with each release to introduce new ideas and challenge architectural decision
7. Technical design decisions taken with each story

Build

1. Ad-hoc build requests; scripting and component marshalling performed manually
2. Consistent, repeatable build process with durable artifacts executed manually with each release
3. Build is automated, executed on a timed basis
4. Unit tests are integrated with the automated build; team is constantly notified of the status of the build; build triggered by SCM updates
5. Test and metrics (e.g., code quality, complexity, check style, etc.) are integrated as gatekeeper events; build data archived and reported in build portal/dashboard
6. All product components advertise dependencies; master repository established
7. Product integration tests included; build process executes automatic deployment to QA testing environment

This approach ressembles in a way to how CMMI is expressed as a model. It lists dimensions (key process areas) and then different maturity levels for each one. The model's scope is very narrow and well expressed (IMO) and it could be useful for an organization to define how responsive in agile terms is the development department to the business.

The 5 levels of Maturity

Martin Proulx sketch this agile maturity model:

Level 1 – Team Level Maturity

At this level, team members have decided to adopt Scrum and/or software engineering practices without asking for approval from their manager. Some of the well known practices are used but without consistency.

Level 2 – Department Level Maturity

At this level, the practices adopted by the team members have started to be imitated by other teams within the software development department. Some of the managers have noticed the positive results of adopting the Agile approach and are tempted to replicate what they observed.

Level 3 – Business Level Maturity

At this level, the solution teams have integrated the business people in the model. Collaboration (and trust) has increased and a partnership relationship is increasing.

Level 4 – Project Management Level Maturity

At this level, the project management approach is modified to include some of the Scrum practices. Although the department still mostly relies on the traditional PMBOK recommendations, Scrum has been integrated in the project management approach.

Level 5 – Management Level Maturity

At this level, managers have adapted their management style to support an Agile organization. Organizational structures and reporting mechanisms are better adapted for collaboration and improved for increased performance.

This model measures basically how spread are the Agile values throughout the organization, starting from the team level with certain practices until reaching the maximun level, where the whole organization shares the same values and behaviors. 

How to measure Agile maturity then?

It seems there are 2 dimensions to be included when measuring the Agile maturity:

• How mature are the Agile practices and process: this is basically the 2nd approach listed above and applies mainly at the project level (or team level using the 3rd model above). How mature are you to do testing? You do all testing manually (or not do it at all!) or you have a solid testing strategy that includes a great deal of automation. How mature are you programming? How mature are you managing your requirements? In all these dimensions, it can be defined an agile way of doing things. The 2nd approach uses behavioral patterns. Another option would be to use objectives and key practices. 
• How spread are these Agile practices and processes throughout the organization: Basically this point says that a mature Agile organization does business in an Agile way. It is fast to deliver business value, it adapts, there is a good level of communication and collaboration throughout the company, etc. A model that encompasses this dimension is much mode ambitious. But probably it is the only way of making Agile work in an organization. The way the software is constructed needs to fit with the way the organization does business around it. The structure of the organization should enable and encourage the right connections that make communication smart and effective throughout the organization. The organization needs to have also the structure to learn and maintain this Agile culture.

CMMI follows a similar approach as it starts with the practices dimensions (level 2) and follows towards a standarized process in the organization (level 3). This seems the natural projection to follow when seeking maturity. Starting with mastering Agility at the team level and then managing the whole organization with the same values and principles.

For both dimensions, a means of measuring maturity is needed. A first attempt using the Shu-Ha-Ri scale will be used.

A very inmature Agile organization would then be an organization where the practices are in the SHU-SHU level at the practices dimension and organization dimension. A RI-RI organization is one that has mastered the Agile practices at the team level and that manages its whole business with Agility.

Agile Capability Maturity Model

Agile Capability Maturity Model

After all the differences we have talked about, the question arises. Does it make sense to have a Capability Maturity Model for Agile? How would it be? Let’s analyze the semantics of the words again, but in an Agile context.

What does it means to be capable, in an agile context?

In the CMMI model, Capability was defined as “the inherent ability of a process to produced the planned results. As the capability of a process improves, the process becomes predictable and measurable, the quality of the result product augments and the productivity of the teams too”. Thus, an organization gets more capable when it has defined and implemented a set of processes throughout the organization that enables it to be predictable, measurable, and continuously improving.

Does the word have the same meaning in an agile context? Agility is more like a mindset or culture than a set of processes. Thus, in Agile you are capable when you have the mindset and the organization is capable when it has the mindset. The semantics of the word changes considerably and worst of all, it makes it unmeasurable. How would you measure how much of the mindset the organization has? Nonetheless, and perhaps very subjectively, one can recognize a team/organization that is more Agile than another. You do it because it adheres better to the Agile values (communication, collaboration, technical excellence, etc). Using this subjectivity, perhaps we can define the meaning of capable in an Agile context. One is capable when it has grasped the Agile values, when you are at the Ri level (in the Shu-Ha-Ri scale) or, as some people say, when you ARE agile. 

How do you get more mature?

Using CMMI, you get more mature as you implement the different processes that conform the model throughout the organization, you can measure them and draw conclusions out of the metrics that allows the organization to improve. 

If we follow the definition of capability in an Agile context given above, the definition of maturity falls in the subjective side as well. An organization is more mature when it can communicate better, it can collaborate better, is able to produce its products with the required quality sustainably, etc. 

Senge talks about the intelligent organization. A mature Agile organization is an intelligent organization. It’s an organization where its members are smart agents that collaborate among them to solve problems and improve continuously. Today’s knowledge organization need to mature in this direction.

The Schneider Culture Model

This diagram is taken from The Reengineering Alternative: A plan for making your current culture work, the book by William Schneider. In it, there are 4 cultures. Each has its name, picture and some words that characterized the quadrant.

There are also two axis that indicate where the focus or an organization is:

1. Horizontal: People Oriented (Personal) vs. Company Oriented (Impersonal)
2. Vertical: Reality Oriented (Actuality) vs. Possibility Oriented

And if we map the Agile values and principles to this model...

We can see how Agile mainly falls into the collaboration and cultivation cultures. I believe that the introduction of CMMI in an Agile Culture moves the culture to the “Control” quadrant. Many people believe this is the way to scale agile and therefore it could be argued that CMMI could be a good way to scale Agile. I don’t feel very comfortable with this idea, as I believe that a good maturity framework should allow the organization to mature without losing its values, its heart. In other words, I believe the maturity needs to happen in the collaboration/cultivation quadrant mainly and the competence one. 

Conclusion

A capability maturity framework for Agile needs to redefine the meanings of capability and maturity to suit the context as being capable and mature have very different semantics in the CMMI world and the Agile one. An Agile Capability Maturity Model should make the organization mature towards a collaboration/cultivation culture. The Agile definitions of capable and mature makes introduce a lot of subjectivity. Could we still measure the results?

CMMI desde una Perspectiva Agil

Mi presentación en #agiles2011

CMMI = Control & Agile = {Collaboration/Cultivation}

Pete Behrens gave a talk on The Culture of Agility @Agile2011 that left me thinking on the topic of my latest posts, the relationship between CMMI and Agile.

In his presentation, he shows a quadrant which contains 4 different types of cultures:

And he defined certain characteristics for each culture. Look the characteristics for a control culture:

This really looks like CMMI, right?. "Policy and procedures are extremely important", "The system is more important than people" (these phrases really sound as the explanation that CMMI gives as to why processes are so important "What holds everything together? It is the processes used in your organization. Processes allow you to align the way you do business. They allow you to address scalability and provide a way to incorporate knowledge of how to do things better. Processes allow you to leverage your resources and to examine business trends.")

Look now the characteristics of a Collaboration Culture:

And the Cultivation Culture:

These really look like Agile, right?

Look now the characteristics of personal and impersonal cultures:

These one's REALLY looked like Agile on the left side and CMMI on the right side. It is clear that CMMI would fit a control culture much better than a collaboration/cultivation culture and , inversely, Agile fits perfectly on collaboration/cultivation cultures. It's clear how Agile is "Personal", while CMMI is "Impersonal".

So what about compatibility. From the perspective of culture, what is the result of combining them?

I can imagine a couple of scenarios. In a collaboration/cultivation culture, applying CMMI would push the organization culture towards a more Control/Impersonal culture. From the other side, a control organization (which may be CMMI certified, as it's totally compliant) and wants to become more Agile may encounter a CMMI an obstacle.

So in the culture context, I see the 2 forces going in opposite directions.

Could this be synergetic or complementary or the effect would be the total opposite? Could this lead to an undefined culture or to an organization where employees are confused on their culture and values?

Software Craftsmen Movement

When software development started, perhaps I guess about 50 years ago (probably more), I believe programming was the most important task in all software development life-cycle. Other tasks or phases probably didn't exist. They were embedded in what they called programming (right, the old ones?). After a few years, we started learning how to develop software better. This learning derived in methodologies that were created. In these methodologies, new phases such as analysis and designed appeared and in  many of them, those phases became more important than programming itself. This was weird because the actual task of building the software lost importance. In heavyweight waterfall methodologies, analysis and design took most of the work. Programming was just a translation work that could be performed (probably by anyone) when all the important work had already been done.


Changing the subject, I usually asked myself: How do we, programmers, know that we do our work correctly? Programming is one of those jobs where anyone, with google's help, can combine a few frameworks, throw a few lines of code and create a basic software system. But does that make you a good programmer? Another circumstance which perhaps is different than in other areas, is that our job market has been in need of professionals for many years, absorbing most or all the people in the field. So for many of use, getting a job as a programmer as been an easy thing. We just got off from University and they were waiting for us. But does that make us good programmers? I believe I can speak on behalf of lots of people when I say that we go from receiving very theoretical classes of Computer Science at University to try to resolve down to earth problems using a programming language. And this happens, a lot of times, without guidance (because supossedly we learned this at University?). Or isn't it true that lots of us have throw our first few thousands of lines of code to production without anyone checking them? I wonder: who teaches us how we should do our job as programmers, the day to day tasks. I'm not talking about learning a programming language. I'm talking about how to use it to resolve a problem, which is the best process to do it, what are the techniques.  This lack of guidance, of someone that verifies our first steps, of someone that shows us the way, makes it difficult for us to understand how we progress in our career. The leap from finishing University to becoming a good programmer is huge. And how do we fill it? How are we supposed to know what it takes to be a good programmer? Is there any curricula we could follow? Is there a career path somewhere? For sure, I wasn't aware of one.

I believe those are the 2 main reasons that fueled the creation of the software craftsmanship movement. Agile was in part a revolution against the essence of heavyweighted methodologies. A revolution that put the programming task as the most important task in all software development life-cycle again. The software craftsmanship movement took this revolution a step further. The actual task of coding the software is the most important task and it is not by any means a simple task. It is a task that requires understanding the business problem to solve, model it in the best possible way, program it using a programming language and by many programmers who all need to agree and implement the same model. To add more complexity,  many times the business scenarios change and software needs to change, to adapt to those changes. University doesn't prepare us for this job and companies need developers developing as soon as possible.

The Software Crafstmanship Manifesto states the following:

As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:

• Not only working software, but also well-crafted software

• Not only responding to change, but also steadily adding value

• Not only individuals and interactions, but also a community of professionals

• Not only customer collaboration, but also productive partnerships

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.

The first important thing to note in this manifesto is that it treats software programming as a craft. A craft is by the dictionary "an art, trade, or occupation requiring special skill, especially manual skill". A clear example of a craft is a carpenter. And how do you learn the carpenter's craft? First you start besides a carpenter as an apprentice and follow him, for a lot of time. Then you start doing small things on your own. And after a long time you master that craft and are able to perform it on your own. Look how different it looks from the path that many of us have followed. Also note that a craft is something that needs to be practiced. Software craftsmen practice their profession not only daily at work, but also through code katas and getting together to improve their craft in code retreats.

Look at the values now. Not just software, but well crafted software. Although some people may say that well crafted software is not the goal (producing value is), software craftsmen know that the best way of producing value is to produce good software,software that is robust, adaptable and can keep growing and producing value for a long period of time. I believe the third value indicates what should be the way to achieve this. A community of professionals. Through the software craftmanship community, I believe that now it's possible to follow a career path that takes you from apprentice to good developer. Professionals on this community are very open to teach what they know and help others become good professionals.

The software craftsmanship movement provided a new perspective to our profession of software programmers. It stressed the importance of programming and also of becoming a good programmer. And it also indicated the way to achieve this. A programmer should start as an apprentice who learns from someone that knows the craft. With practice, and more practice, this apprentice will learn the craft and become a good programmer.

Code & Beyond: Video: ¿Qué esperas de Agile? con Federico Zuppa

Code & Beyond: Video: ¿Qué esperas de Agile? con Federico Zuppa: "Este video es de una de las reuniones mensuales de Agiles @ Buenos Aires , pero me había quedado en el tintero procesarlo y publicarlo, así..."

The Lean Tribe

The Agile Manifesto triggered the growth of "the so called until that moment lightweight methodologies". Scrum and XP became very popular and the tribes associated to them grew with them. However, not long after the Agile Manifesto was signed a new actor entered the scene with outreagous strength. His friends use to call him "Lean" :-)

So what is Lean? Lean has its roots in the Toyota Production System developed by Taichi Ohno. Its core idea is to maximize customer value while minimizing waste.

Lean was brought into the Agile world by Mary and Tom Poppendieck who in their book "Lean Software Development: An Agile toolkit" translated 7 lean principles used in manufacturing to software development language.

Although Lean and Agile share many of the same values and principles as they both are people centric and empower people and they both try to adapt/improve the process continually, they also have some differences as well. The main one in my opinion is its main objective: Agile was developed mostly by software developers and therefore its main objetive is "to uncover better ways of developing software". Lean has in its essence a broader objective (and a broader target) as it seeks to improve the flow of value and this could only happen at an organizational level.

The Agile world was greatly modified by the entrance of Lean. So much that probably now it is an Agile/Lean world.

So who are the referents of this new Lean tribe. Where do they get together. Do they breed with Agile folks? :D I know I may be missing many of the referents of the manufacture industries but these are the ones I know:

- Mary and Tom Poppendieck
- Eli Goldratt (RIP)
- David Anderson
- Karl Scotland
- Alan Shalloway

These folks get together in conferences like "Lean Software and Systems" (in its website, you'll be able to find all the speakers that attended in 2011)

In the last few years, the Lean tribe grew incredibly fast. Particularly, a lean tool called Kanban has gained a great deal of attention and is becoming the new "Scrum" of the moment (or the new star of the moment).

How is the relationship between Agile and Lean? Well, I believe that although there have been some fights (or discussions should I say) because sometimes tribes defend their interests, the Agile community has a lot to learn from the Lean community. Although in the Agile world, Lean is new, it really is much older than Agile.

Agile Tribes

The people that got together 10 years ago in Salt Lake City had different backgrounds, specialties, positions, etc.There were some folks already working on their own methodologies, such as Beck with XP, Schwaber and Sutherland with Scrum and Cockburn with Crystal. Some of them were more technical, specializing in OO, programming and testing while some others were managers or people more interested in the methodology aspects of software development. This is more or less what I believe were the backgrounds that depict the folks that were there:

Now I wonder how these 'tribes' (or communities or subfields of interests - I wonder what is the best name) develop over the years. How did they grow or mutated? Which are the new tribes that make up the Agile world?. Who are the new referents of these tribes? Will you help me create this map?
How do I define what a tribe is? Let's list some items that define a tribe:

• They share the same values and principles (which should enter the big Agile umbrella)
• They work with a certain methodology or method
• They have some referents
• They get together in one of more congresses (this could be a consequence of the previous)

Am I missing something to define a tribe? Hummm, yeah for sure. Will see it as we go (iteratively). For the moment, I will research for each tribe the following items:

• Reason to be (values/principles they share, objectives)
• History
• Referents
• Present/Future?

Now I just need to select the most important tribes in today's Agile map. I will start with these ones:

• Kanbaners
• Software Craftsmen
• Lean-startupers
• Scrumers
• XPs

I will focus on the first 3 groups, as the last 2 exist since the beggining of Agile.
I'd love to receive suggestions/corrections, etc.