First steps towards an Agile Maturity Model

I wonder how a model for maturing in an Agile context should be thought. Let's see some examples of previous attempts:

Agile Process Maturity Model

"The goal of the Agile Process Maturity Model (APMM) is to provide a framework which provides context for the plethora of agile methodologies and practices out there today. The APMM defines three levels, each of which build upon each other, for agile processes and practices."

APMM Level 1: Core Agile Development

Level 1 agile processes address a portion of the system life cycle (SDLC). Examples of Level 1 agile processes include Scrum, Extreme Programming, Agile Modeling and Agile Data.

APMM Level 2: Disciplined Agile Delivery

Level 2 agile processes extend Level 1 to address the full system delivery life cycle (SDLC). As the 
criteria for disciplined agile development suggests they also tend to dial up certain aspects of agile development, such as testing, measurement, and process improvement. Furthermore, they include explicit mechanisms to support effective (ideally lean) governance. Examples of Level 2 agile processes include Hybrid Processes (Scrum@XP), Rational Unified Process, Open Unified Process, Harmony ESW and Dynamic System Development Method.

APMM Level 3: Agility at Scale

In the early days of agile, the applications where agile development was applied were smaller in scope and relatively straightforward. Today, the picture has changed significantly and organizations want to apply agile development to a broader set of projects. Agile hence needs to adapt to deal with the many business, organization, and technical complexities todays software development organizations are facing. This is what Level 3 of the APMM is all about explicitly addressing the complexities which disciplined agile delivery teams face in the real world.

The goal of the APMM is "to define a framework which can be used to put the myriad agile processes into context. 

Agile Maturity Model

The AMM, "tries to quantify degree of responsiveness of a team to an organization's business needs. The philosophy being that increasing the discipline and proficiency of executing some best practices will increase the degree of responsiveness."

The AMM primarily focuses on application development activities; the dimensions of the AMM are

• Testing
• Source code management
• Collective code ownership
• Collaboration
• Responsiveness to business
• Assurance and governance
• Story formation
• Design simplicity
• Build process

For each of the above dimensions, the AMM lists behavioral patterns exhibited by teams. Teams can use the AMM as a reference to identify and compare the behavior that they currently exhibit and plan to alter and tailor them.

Testing

1. Acceptance testing in lieu of all other tests
2. No unit testing framework, ad hoc testing (Manual testing of individual components/ controls)
3. Unit testing, manual functional testing; application has a testable architecture
4. Unit testing integrated into build process, testing automated as much as reasonable given application
5. Developers write unit tests before writing functional code
6. Test are identified and produced as part of a story creation
7. Automated functional testing (e.g.. GUI testing); stories remain in development until all bugs are fixed or deferred

Source Code Management

1. Traditional schemes based on fire-locking and time-sharing
2. SCM supports version of code
3. IDE(s) integrate with SCM; developers include meaningful comments in commits
4. SCM support merging; optimistic check-ins
5. SCM support atomic commits
6. All development collateral is in SCM
7. SCM is transparent to delivery team, behind build process

Collective code ownership

1. Knowledge held by specific team members; people work in isolation
2. No pairing, people work alone, some informal process for keeping people informed
3. Pairing; no code locking
4. Pairing scheme ensures rotation
5. Team signs up for functionality rather than assignment to individuals
6. Within an sprint, functionality delivery is signed up for just in time
7. Old (bugs) and new functionality is queued, development team pops the stack in real time

Collaboration

1. Regular progress updates from management to the delivery team (as opposed to the other way around); irregular team meetings
2. Project collaboration tools (wiki, mailing list, IM) in place and used throughout project team; project status published and visible to all
3. Daily stand-ups and sprint meetings; problem solving is bottom-up as opposed to top-down; team sets sprint objectives and agrees on estimates.
4. Integrated, continuous build process, with build status notification to the team and collective responsibility for state of the build
5. Business is part of the team, stakeholders accept working software at reviews in lieu of other tracking or progress metrics
6. Frequent (near-real time) prioritization of old and new functionality
7. Build automatically deploys to QA environment available to any interested party

Responsiveness to business

1. Frozen specification, unresponsive to business value
2. Team tries to accommodate new requirements by ad hoc change requests ("pile it on")
3. Iterative process with sprints of length short enough to respond to business change
4. Showcases per sprint; business prioritizes functionality per sprint
5. Continuous involvement of the business on the team: business certifies story complete out of dev; involved in (rather than approval of) story definition and acceptance tests
6. Business writes high-level stories for all requests; organizational story queue exists; prioritization decisions made from full story backlog
7. Development process is integral to business initiatives; development teams work as a part of the business unit rather than as a service to the business unit

Assurance and Governance 

1. Status reports document progress; schedule acts as plan
2. Concept of release planning is introduced
3. Sprint planning is introduced
4. Plan becomes communication and tracking tool rather than an exception report
5. Every sprint review examines value delivered and assesses next sprint by need, priorities (and delivery)
6. Organizational adoption: Introduction of Portfolio Planning and global optimization of resources (use of metric and standard measures)
7. Business review of value and return helps teams on regular basis based upon status

Story Formation (Requirements)

1. Development tasks extracted from voluminous, frozen requirements artifacts
2. Production of lightweight artifacts (e.g. panel flows) to drive high-level requirements development
3. Mapping granular requirements (e.g., use cases) in high level user requirements
4. Marshaling use cases into discreet statements of functionality that can be delivered in time boxed development sprints
5. Stories are an expression of end-to-end functionality to be developed, including testable acceptance criteria and a statements of value
6. Spontaneous story development provided by business for in-flight projects; stories not derived from extant sources but are immediate expressions of customer demand/need
7. Global repository of functional requirements for stories developed by business for any business requirements, including a formal measure of business value delivered

Simplicity

1. Big up-front design that attempts to accommodate all potential future needs
2. Application of fundamental design patterns
3. Structural refactoring to decouple application architecture
4. Doing only what needs to be done
5. Aggressive and constant refactoring to improve code quality/simplicity extant code
6. Spiking solution with each release to introduce new ideas and challenge architectural decision
7. Technical design decisions taken with each story

Build

1. Ad-hoc build requests; scripting and component marshalling performed manually
2. Consistent, repeatable build process with durable artifacts executed manually with each release
3. Build is automated, executed on a timed basis
4. Unit tests are integrated with the automated build; team is constantly notified of the status of the build; build triggered by SCM updates
5. Test and metrics (e.g., code quality, complexity, check style, etc.) are integrated as gatekeeper events; build data archived and reported in build portal/dashboard
6. All product components advertise dependencies; master repository established
7. Product integration tests included; build process executes automatic deployment to QA testing environment

This approach ressembles in a way to how CMMI is expressed as a model. It lists dimensions (key process areas) and then different maturity levels for each one. The model's scope is very narrow and well expressed (IMO) and it could be useful for an organization to define how responsive in agile terms is the development department to the business.

The 5 levels of Maturity

Martin Proulx sketch this agile maturity model:

Level 1 – Team Level Maturity

At this level, team members have decided to adopt Scrum and/or software engineering practices without asking for approval from their manager. Some of the well known practices are used but without consistency.

Level 2 – Department Level Maturity

At this level, the practices adopted by the team members have started to be imitated by other teams within the software development department. Some of the managers have noticed the positive results of adopting the Agile approach and are tempted to replicate what they observed.

Level 3 – Business Level Maturity

At this level, the solution teams have integrated the business people in the model. Collaboration (and trust) has increased and a partnership relationship is increasing.

Level 4 – Project Management Level Maturity

At this level, the project management approach is modified to include some of the Scrum practices. Although the department still mostly relies on the traditional PMBOK recommendations, Scrum has been integrated in the project management approach.

Level 5 – Management Level Maturity

At this level, managers have adapted their management style to support an Agile organization. Organizational structures and reporting mechanisms are better adapted for collaboration and improved for increased performance.

This model measures basically how spread are the Agile values throughout the organization, starting from the team level with certain practices until reaching the maximun level, where the whole organization shares the same values and behaviors. 

How to measure Agile maturity then?

It seems there are 2 dimensions to be included when measuring the Agile maturity:

• How mature are the Agile practices and process: this is basically the 2nd approach listed above and applies mainly at the project level (or team level using the 3rd model above). How mature are you to do testing? You do all testing manually (or not do it at all!) or you have a solid testing strategy that includes a great deal of automation. How mature are you programming? How mature are you managing your requirements? In all these dimensions, it can be defined an agile way of doing things. The 2nd approach uses behavioral patterns. Another option would be to use objectives and key practices. 
• How spread are these Agile practices and processes throughout the organization: Basically this point says that a mature Agile organization does business in an Agile way. It is fast to deliver business value, it adapts, there is a good level of communication and collaboration throughout the company, etc. A model that encompasses this dimension is much mode ambitious. But probably it is the only way of making Agile work in an organization. The way the software is constructed needs to fit with the way the organization does business around it. The structure of the organization should enable and encourage the right connections that make communication smart and effective throughout the organization. The organization needs to have also the structure to learn and maintain this Agile culture.

CMMI follows a similar approach as it starts with the practices dimensions (level 2) and follows towards a standarized process in the organization (level 3). This seems the natural projection to follow when seeking maturity. Starting with mastering Agility at the team level and then managing the whole organization with the same values and principles.

For both dimensions, a means of measuring maturity is needed. A first attempt using the Shu-Ha-Ri scale will be used.

A very inmature Agile organization would then be an organization where the practices are in the SHU-SHU level at the practices dimension and organization dimension. A RI-RI organization is one that has mastered the Agile practices at the team level and that manages its whole business with Agility.

Agile Capability Maturity Model

Agile Capability Maturity Model

After all the differences we have talked about, the question arises. Does it make sense to have a Capability Maturity Model for Agile? How would it be? Let’s analyze the semantics of the words again, but in an Agile context.

What does it means to be capable, in an agile context?

In the CMMI model, Capability was defined as “the inherent ability of a process to produced the planned results. As the capability of a process improves, the process becomes predictable and measurable, the quality of the result product augments and the productivity of the teams too”. Thus, an organization gets more capable when it has defined and implemented a set of processes throughout the organization that enables it to be predictable, measurable, and continuously improving.

Does the word have the same meaning in an agile context? Agility is more like a mindset or culture than a set of processes. Thus, in Agile you are capable when you have the mindset and the organization is capable when it has the mindset. The semantics of the word changes considerably and worst of all, it makes it unmeasurable. How would you measure how much of the mindset the organization has? Nonetheless, and perhaps very subjectively, one can recognize a team/organization that is more Agile than another. You do it because it adheres better to the Agile values (communication, collaboration, technical excellence, etc). Using this subjectivity, perhaps we can define the meaning of capable in an Agile context. One is capable when it has grasped the Agile values, when you are at the Ri level (in the Shu-Ha-Ri scale) or, as some people say, when you ARE agile. 

How do you get more mature?

Using CMMI, you get more mature as you implement the different processes that conform the model throughout the organization, you can measure them and draw conclusions out of the metrics that allows the organization to improve. 

If we follow the definition of capability in an Agile context given above, the definition of maturity falls in the subjective side as well. An organization is more mature when it can communicate better, it can collaborate better, is able to produce its products with the required quality sustainably, etc. 

Senge talks about the intelligent organization. A mature Agile organization is an intelligent organization. It’s an organization where its members are smart agents that collaborate among them to solve problems and improve continuously. Today’s knowledge organization need to mature in this direction.

The Schneider Culture Model

This diagram is taken from The Reengineering Alternative: A plan for making your current culture work, the book by William Schneider. In it, there are 4 cultures. Each has its name, picture and some words that characterized the quadrant.

There are also two axis that indicate where the focus or an organization is:

1. Horizontal: People Oriented (Personal) vs. Company Oriented (Impersonal)
2. Vertical: Reality Oriented (Actuality) vs. Possibility Oriented

And if we map the Agile values and principles to this model...

We can see how Agile mainly falls into the collaboration and cultivation cultures. I believe that the introduction of CMMI in an Agile Culture moves the culture to the “Control” quadrant. Many people believe this is the way to scale agile and therefore it could be argued that CMMI could be a good way to scale Agile. I don’t feel very comfortable with this idea, as I believe that a good maturity framework should allow the organization to mature without losing its values, its heart. In other words, I believe the maturity needs to happen in the collaboration/cultivation quadrant mainly and the competence one. 

Conclusion

A capability maturity framework for Agile needs to redefine the meanings of capability and maturity to suit the context as being capable and mature have very different semantics in the CMMI world and the Agile one. An Agile Capability Maturity Model should make the organization mature towards a collaboration/cultivation culture. The Agile definitions of capable and mature makes introduce a lot of subjectivity. Could we still measure the results?

CMMI desde una Perspectiva Agil

Mi presentación en #agiles2011

CMMI = Control & Agile = {Collaboration/Cultivation}

Pete Behrens gave a talk on The Culture of Agility @Agile2011 that left me thinking on the topic of my latest posts, the relationship between CMMI and Agile.

In his presentation, he shows a quadrant which contains 4 different types of cultures:

And he defined certain characteristics for each culture. Look the characteristics for a control culture:

This really looks like CMMI, right?. "Policy and procedures are extremely important", "The system is more important than people" (these phrases really sound as the explanation that CMMI gives as to why processes are so important "What holds everything together? It is the processes used in your organization. Processes allow you to align the way you do business. They allow you to address scalability and provide a way to incorporate knowledge of how to do things better. Processes allow you to leverage your resources and to examine business trends.")

Look now the characteristics of a Collaboration Culture:

And the Cultivation Culture:

These really look like Agile, right?

Look now the characteristics of personal and impersonal cultures:

These one's REALLY looked like Agile on the left side and CMMI on the right side. It is clear that CMMI would fit a control culture much better than a collaboration/cultivation culture and , inversely, Agile fits perfectly on collaboration/cultivation cultures. It's clear how Agile is "Personal", while CMMI is "Impersonal".

So what about compatibility. From the perspective of culture, what is the result of combining them?

I can imagine a couple of scenarios. In a collaboration/cultivation culture, applying CMMI would push the organization culture towards a more Control/Impersonal culture. From the other side, a control organization (which may be CMMI certified, as it's totally compliant) and wants to become more Agile may encounter a CMMI an obstacle.

So in the culture context, I see the 2 forces going in opposite directions.

Could this be synergetic or complementary or the effect would be the total opposite? Could this lead to an undefined culture or to an organization where employees are confused on their culture and values?

Software Craftsmen Movement

When software development started, perhaps I guess about 50 years ago (probably more), I believe programming was the most important task in all software development life-cycle. Other tasks or phases probably didn't exist. They were embedded in what they called programming (right, the old ones?). After a few years, we started learning how to develop software better. This learning derived in methodologies that were created. In these methodologies, new phases such as analysis and designed appeared and in  many of them, those phases became more important than programming itself. This was weird because the actual task of building the software lost importance. In heavyweight waterfall methodologies, analysis and design took most of the work. Programming was just a translation work that could be performed (probably by anyone) when all the important work had already been done.


Changing the subject, I usually asked myself: How do we, programmers, know that we do our work correctly? Programming is one of those jobs where anyone, with google's help, can combine a few frameworks, throw a few lines of code and create a basic software system. But does that make you a good programmer? Another circumstance which perhaps is different than in other areas, is that our job market has been in need of professionals for many years, absorbing most or all the people in the field. So for many of use, getting a job as a programmer as been an easy thing. We just got off from University and they were waiting for us. But does that make us good programmers? I believe I can speak on behalf of lots of people when I say that we go from receiving very theoretical classes of Computer Science at University to try to resolve down to earth problems using a programming language. And this happens, a lot of times, without guidance (because supossedly we learned this at University?). Or isn't it true that lots of us have throw our first few thousands of lines of code to production without anyone checking them? I wonder: who teaches us how we should do our job as programmers, the day to day tasks. I'm not talking about learning a programming language. I'm talking about how to use it to resolve a problem, which is the best process to do it, what are the techniques.  This lack of guidance, of someone that verifies our first steps, of someone that shows us the way, makes it difficult for us to understand how we progress in our career. The leap from finishing University to becoming a good programmer is huge. And how do we fill it? How are we supposed to know what it takes to be a good programmer? Is there any curricula we could follow? Is there a career path somewhere? For sure, I wasn't aware of one.

I believe those are the 2 main reasons that fueled the creation of the software craftsmanship movement. Agile was in part a revolution against the essence of heavyweighted methodologies. A revolution that put the programming task as the most important task in all software development life-cycle again. The software craftsmanship movement took this revolution a step further. The actual task of coding the software is the most important task and it is not by any means a simple task. It is a task that requires understanding the business problem to solve, model it in the best possible way, program it using a programming language and by many programmers who all need to agree and implement the same model. To add more complexity,  many times the business scenarios change and software needs to change, to adapt to those changes. University doesn't prepare us for this job and companies need developers developing as soon as possible.

The Software Crafstmanship Manifesto states the following:

As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:

• Not only working software, but also well-crafted software

• Not only responding to change, but also steadily adding value

• Not only individuals and interactions, but also a community of professionals

• Not only customer collaboration, but also productive partnerships

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.

The first important thing to note in this manifesto is that it treats software programming as a craft. A craft is by the dictionary "an art, trade, or occupation requiring special skill, especially manual skill". A clear example of a craft is a carpenter. And how do you learn the carpenter's craft? First you start besides a carpenter as an apprentice and follow him, for a lot of time. Then you start doing small things on your own. And after a long time you master that craft and are able to perform it on your own. Look how different it looks from the path that many of us have followed. Also note that a craft is something that needs to be practiced. Software craftsmen practice their profession not only daily at work, but also through code katas and getting together to improve their craft in code retreats.

Look at the values now. Not just software, but well crafted software. Although some people may say that well crafted software is not the goal (producing value is), software craftsmen know that the best way of producing value is to produce good software,software that is robust, adaptable and can keep growing and producing value for a long period of time. I believe the third value indicates what should be the way to achieve this. A community of professionals. Through the software craftmanship community, I believe that now it's possible to follow a career path that takes you from apprentice to good developer. Professionals on this community are very open to teach what they know and help others become good professionals.

The software craftsmanship movement provided a new perspective to our profession of software programmers. It stressed the importance of programming and also of becoming a good programmer. And it also indicated the way to achieve this. A programmer should start as an apprentice who learns from someone that knows the craft. With practice, and more practice, this apprentice will learn the craft and become a good programmer.

Code & Beyond: Video: ¿Qué esperas de Agile? con Federico Zuppa

Code & Beyond: Video: ¿Qué esperas de Agile? con Federico Zuppa: "Este video es de una de las reuniones mensuales de Agiles @ Buenos Aires , pero me había quedado en el tintero procesarlo y publicarlo, así..."

The Lean Tribe

The Agile Manifesto triggered the growth of "the so called until that moment lightweight methodologies". Scrum and XP became very popular and the tribes associated to them grew with them. However, not long after the Agile Manifesto was signed a new actor entered the scene with outreagous strength. His friends use to call him "Lean" :-)

So what is Lean? Lean has its roots in the Toyota Production System developed by Taichi Ohno. Its core idea is to maximize customer value while minimizing waste.

Lean was brought into the Agile world by Mary and Tom Poppendieck who in their book "Lean Software Development: An Agile toolkit" translated 7 lean principles used in manufacturing to software development language.

Although Lean and Agile share many of the same values and principles as they both are people centric and empower people and they both try to adapt/improve the process continually, they also have some differences as well. The main one in my opinion is its main objective: Agile was developed mostly by software developers and therefore its main objetive is "to uncover better ways of developing software". Lean has in its essence a broader objective (and a broader target) as it seeks to improve the flow of value and this could only happen at an organizational level.

The Agile world was greatly modified by the entrance of Lean. So much that probably now it is an Agile/Lean world.

So who are the referents of this new Lean tribe. Where do they get together. Do they breed with Agile folks? :D I know I may be missing many of the referents of the manufacture industries but these are the ones I know:

- Mary and Tom Poppendieck
- Eli Goldratt (RIP)
- David Anderson
- Karl Scotland
- Alan Shalloway

These folks get together in conferences like "Lean Software and Systems" (in its website, you'll be able to find all the speakers that attended in 2011)

In the last few years, the Lean tribe grew incredibly fast. Particularly, a lean tool called Kanban has gained a great deal of attention and is becoming the new "Scrum" of the moment (or the new star of the moment).

How is the relationship between Agile and Lean? Well, I believe that although there have been some fights (or discussions should I say) because sometimes tribes defend their interests, the Agile community has a lot to learn from the Lean community. Although in the Agile world, Lean is new, it really is much older than Agile.