Agile Knowledge Repository

Summary

This area deals with knowledge acquisition and knowledge management in an organization. One of the main points of CMMI is to maintain a process repository (as expressed in the Organization Process Definition Area) that enables the organization to acquire and maintain its knowledge. Agile Organizations should maintain this repository, by creating an Agile Culture (I expressed my doubts about being able to create a documented knowledge repository in an agile software organization, as I think software development is a craft). Therefore, processes and techniques live inside the people's brains of the organization and is acquired and maintained by practicing and teaching those process and techniques. Knowledge is maintained using the model used in the past by guilds.

Goals

- There is a knowledge path that people follow inside the organization, starting from apprentice (someone that is learning, does exercises, pair programs with someone and has a lot of time to educate himself) to Journeyman (which is someone that knows the craft and is able to teach it).
- Knowledge is shared openly and collaboratively inside the organization. Culture is strong. More senior software craftsmen lead and teach the rest of the group on "how we do things inside here". Values float in the air.

Standard Practices

I have taken these practices from 8thlight, a company I visited a few months ago. In 8thlight:

- Employees start as apprentices. Their job at that moment is to learn how to develop software and how the company manages their projects. They do exercises and pair programming with more senior craftsmen.
- Software craftsmen have one or more apprentices in charge. So their responsibility is to teach apprentices the processes and techniques used inside the organization.
- Knowledge is shared and maintained by practicing together. Employees do code retreats or katas together. The philosophy is programmers need to practice their craft and this practice is performed on code that is not business oriented.

Presentation in Agiles 2012

Transforming Cultures on Prezi

Agile Collaboration Area

Summary

The Agile Manifesto says "Business people and developers must work together daily through the project". Everyone, within the organization and beyond it (i.e. clients) must work jointly towards a shared objective, help each other, have good intentions, share all the information and ultimately to do what's best for the everyone involved.

Goals

- Collaboration between the organization and its clients is enabled financially. Contracts should enforce collaboration.
- Collaboration among team members in a project and among different departments of the organization should be enforced. 
- All stakeholders should be involved throughout the duration of a project.

Standard Practices

Clients work with team members to specify the requirements and prioritize them in a backlog that will be used continually during the project. Features are presented to these clients early in the life of the project to be able to receive feedback and change according to it. This change should be enabled in the contract (if the contract penalizes delays, the team won't be willing to accept any changes during the life of the project). Everyone in the team should share the same objectives. Therefore, policies and rewards should be devised with this in mind.

Agile Communication Area

Summary

Communication is key in an Agile organization. Developing software is a communication problem. Communication needs to flow as fast and clear as possible among all members of the organization. An environment that enables and enforces communication should be built.

Goals

- There is an environment that allows interactions among all members in the organization freely.
- All members of the organization can easily  communicate with each other.
- There is complete transparency to everyone about the goals, status, direction of the project/organization.

Standard Practices

- The office is an open space, where all members share the same information. There are visual radiators in many places of the office that allow members of a project and all members in the organization to understand the status and direction of the different projects.
- Team members in a project interact as much as possible to synchronize and make sure there is a shared understanding of what needs to be built. In Scrum, there are different meetings that enable and enforce this communication: 1) Planning meeting at the beginning of each iteration 2) Daily Standup every day 3) Demo and Retrospection at the end of the iteration. Besides this, practices like pair program or code review sessions push the members to interact even more. Understanding the effectiveness of the different types of communication types helps devise a communication strategy. When is possible, face to face communication is the most effective type of communication but there are situations when the team is distributed geographically and therefore other communication methods should be used. Increasing the communication to its highest level should be the direction in these cases.

Self Organization Area

Summary

This area's goal is to create an environment where all members feel empowered to contribute all their knowledge and energy for the organization's benefit. Leaders and more experienced individuals should guide less experienced ones and set constraints and directions.

Goals

- Team members are empowered
- The team self organizes to create the best solution to the specified problems.
- Leaders create and maintain an environment where self-organization can thrive

Standard Practices

- Within a project, a cross functional team self organizes to develop the best solution to the problem.
- Managers are servant leader that teach through example and help other members to grow within the organization and to give the best of them.

Agile Process Development Lifecycle Area

Summary

The process should be an enabler of the objectives the organization needs to fulfill. It should allow the organization to deliver value fast and sustainably. It should be revised and improved continually (Kaizen).

Goals

- The process enables incremental delivery of features. In other words, the process should allow to slice the functionality to be delivered in multiple chunks, that when delivered allow the business to receive the most valuable/important functionality sooner.
- Process should be light and empiric. All artifacts in the process should give value to the product creation process.
- Process should be empiric. The team should continually introspect on the process and change it to function better within the specific context.

Standard Practices

- Common methodologies/frameworks like Scrum and Extreme Programming use iterations to deliver features incrementally and iteratively. Features should be potentially shippable at the end of each iteration. This doesn't mean that they could be shipped, as a feature could be part of a larget minimum viable feature.
- At the end of each iteration, the team can show to all business stakeholders the business functionality that was completed. If requirements are sliced vertically and provide business value, they can be shown.  Stakeholders can provide their feedback, which could be incorporated in the product backlog through the product owner.
- The team should inspect the process (among other things) at the end of each iteration. Usually, the team tries to find what is working with the current process and isn't, devising action items that can improve its performance.

Agile Project Monitoring Area

Summary

All members of the organization are in charge of monitoring the progress of the project, suggesting corrective actions and making sure they are completed. There is complete transparency and a lot of communication among everyone, thus ensuring that everyone understand the current status of the project.

Goals

- The team monitors the actual progress of the project at timeboxed intervals. It should be measured the actual completion of requirements, risks tackled and discovered, changes in business requirements (i.e. changes in scope), etc.
- The team monitors the progress daily, trying to discover blockers (things that prevent the team from doing progress) and synchronization points (e.g. among dependent requirements)
- The team and business stakeholders agree on corrective actions or adaptions to the plan and execute them.

Standard Practices

- After each iteration, there is a demo that allows the team to show the business side the actual progress made on requirements. There is a great deal of knowledge won in this meeting as the business people get a feeling of the actual product being constructed. This knowledge gained should be reflected in the plan, as the horizon should be clearer after each iteration. The team should have also gained a clearer understanding of what it takes to develop the requirements and the amount of work remaining.
- Daily, the team introspects on the progress (compared to the expect one) and blockers trying to complete all work they have committed to and remove all impediments as fast as possible. In Scrum, this meeting is called "daily scrum" and it is usually the ScrumMaster who is in charge of removing any impediment that might prevent the team from doing progress.
- Each action item detected during either the daily scrum or the review should be completed to end. It is a good practice for one of the team members to be hold accountable.
- Visual Radiators should be hanged in various places of the open space the team works on. One of the tenets of Agile is transparency, and therefore everyone should be as aware as possible of the actual progress.

Agile Project Planning Area

Summary

Initial planning sets up the initial direction, which is continually updated based on new discoveries (business and technical). Therefore, planning is a continuous work that endures the whole project.

Goals

- A plan is built when the project starts and updated it continually throughout the project.
- Estimations are performed for all requirements. The time invested in estimation should be directly proportional to the level of approval and understanding of the requirement (A greater lever of understanding and approval enables a greater level of detail on the estimation)
- All involved members should be involved in estimations, aware of the plan and committed to the success of the project.

Standard Practices

- In the 'Release Planning' meeting, a backlog that contains all stories known so far is constructed. As there is a great deal of uncertainty and unknowns, stories are estimated roughly, to have a first approximation to the dimension of the project.
- If the project is too big, it is a common practice to slice the project in releases. First release should contain the most important business functionality, what is needed more urgently or what lets the company enter a field.
- During the execution of the project and as more is learned about the project, estimations and forecasts are updated to reflect the gained knowledge in the plan.
- A common technique for doing the estimation is the Poker Planning, where all members of the project contribute their wisdom (this technique relies on the Wisdom of Crowds)
- Involvement and transparency enable full commitment of the team.

Agile Requirements Management Area

Summary

Requirements are built by business owners and technical members and refined through all the duration of the project.They should be prioritized based on their value and technical risk to maximize the probabilities of success of the project.

Goals

- At all times, a project backlog is maintained with all requirements prioritized by their business values and technical risk. This backlog might have different levels of understanding and approval. The highest priority requirements should have a greater level of detail, understanding and approval.
- The backlog should be easy to maintain and light. The product owner should be able to modify and re-prioritize it at all times.
- All requirements should have a set of acceptance automated tests up to date at all times. If there are changes to the already implemented requirements, the acceptance tests need to be modified to reflect that.

Standard Practices

- A product backlog is created at the beginning of the project and maintained through out it. The main input should come from the product owners, who understand the business needs and the value of the requirements and from the development team, who should advise about the technical risks.
- A common practice is to write the requirements as user stories. A user story is written using the business lingo, it has an actor (someone that will use the system) and it provides some (ideally measurable) business value.
- Acceptance tests should be defined and validated with the product owner and implemented for all stories. These acceptance tests act as an scaffold that assures all requirements are being met at all times and also as a traceability tool.
- An Agile Management Tool is needed in case the project is considerably big and/or geographically distributed. This tool should allow anyone to make modifications on the stories and to reprioritize them.

Agile Maturity Model Categories & Areas

A time ago I did a post about the Agile Architecture which contains what I think are the most important 'components' (architecturally speaking) of Agile:

I believe this is a good start to define the areas that will make up this maturity model.

Process

In the process category, I will group all areas related to the process development lifecycle. I am tempted to call this category the methodology one, but the methodology encompasses much more elements than just the process ones. The methodology is everything you do in your organization to develop software (with this definition, methodology and culture are almost the same thing?). Some of the areas in this category include

- Requirements Management
- Process Development Lifecycle Management
- Project Planning
- Project Monitoring (or execution)

Human

- Self-Organization and Self-Discipline (I moved this to the human category in the original diagram)
- Communication
- Colllaboration
- Develop Competence
- Creating an Agile Culture?

Technical

- Technical Excellence
- Iterative Architecture and Emergent Design
- Verification & Validation
- SCM

First steps towards an Agile Maturity Model

I wonder how a model for maturing in an Agile context should be thought. Let's see some examples of previous attempts:

Agile Process Maturity Model

"The goal of the Agile Process Maturity Model (APMM) is to provide a framework which provides context for the plethora of agile methodologies and practices out there today. The APMM defines three levels, each of which build upon each other, for agile processes and practices."

APMM Level 1: Core Agile Development

Level 1 agile processes address a portion of the system life cycle (SDLC). Examples of Level 1 agile processes include Scrum, Extreme Programming, Agile Modeling and Agile Data.

APMM Level 2: Disciplined Agile Delivery

Level 2 agile processes extend Level 1 to address the full system delivery life cycle (SDLC). As the 
criteria for disciplined agile development suggests they also tend to dial up certain aspects of agile development, such as testing, measurement, and process improvement. Furthermore, they include explicit mechanisms to support effective (ideally lean) governance. Examples of Level 2 agile processes include Hybrid Processes (Scrum@XP), Rational Unified Process, Open Unified Process, Harmony ESW and Dynamic System Development Method.

APMM Level 3: Agility at Scale

In the early days of agile, the applications where agile development was applied were smaller in scope and relatively straightforward. Today, the picture has changed significantly and organizations want to apply agile development to a broader set of projects. Agile hence needs to adapt to deal with the many business, organization, and technical complexities todays software development organizations are facing. This is what Level 3 of the APMM is all about explicitly addressing the complexities which disciplined agile delivery teams face in the real world.

The goal of the APMM is "to define a framework which can be used to put the myriad agile processes into context. 

Agile Maturity Model

The AMM, "tries to quantify degree of responsiveness of a team to an organization's business needs. The philosophy being that increasing the discipline and proficiency of executing some best practices will increase the degree of responsiveness."

The AMM primarily focuses on application development activities; the dimensions of the AMM are

• Testing
• Source code management
• Collective code ownership
• Collaboration
• Responsiveness to business
• Assurance and governance
• Story formation
• Design simplicity
• Build process

For each of the above dimensions, the AMM lists behavioral patterns exhibited by teams. Teams can use the AMM as a reference to identify and compare the behavior that they currently exhibit and plan to alter and tailor them.

Testing

1. Acceptance testing in lieu of all other tests
2. No unit testing framework, ad hoc testing (Manual testing of individual components/ controls)
3. Unit testing, manual functional testing; application has a testable architecture
4. Unit testing integrated into build process, testing automated as much as reasonable given application
5. Developers write unit tests before writing functional code
6. Test are identified and produced as part of a story creation
7. Automated functional testing (e.g.. GUI testing); stories remain in development until all bugs are fixed or deferred

Source Code Management

1. Traditional schemes based on fire-locking and time-sharing
2. SCM supports version of code
3. IDE(s) integrate with SCM; developers include meaningful comments in commits
4. SCM support merging; optimistic check-ins
5. SCM support atomic commits
6. All development collateral is in SCM
7. SCM is transparent to delivery team, behind build process

Collective code ownership

1. Knowledge held by specific team members; people work in isolation
2. No pairing, people work alone, some informal process for keeping people informed
3. Pairing; no code locking
4. Pairing scheme ensures rotation
5. Team signs up for functionality rather than assignment to individuals
6. Within an sprint, functionality delivery is signed up for just in time
7. Old (bugs) and new functionality is queued, development team pops the stack in real time

Collaboration

1. Regular progress updates from management to the delivery team (as opposed to the other way around); irregular team meetings
2. Project collaboration tools (wiki, mailing list, IM) in place and used throughout project team; project status published and visible to all
3. Daily stand-ups and sprint meetings; problem solving is bottom-up as opposed to top-down; team sets sprint objectives and agrees on estimates.
4. Integrated, continuous build process, with build status notification to the team and collective responsibility for state of the build
5. Business is part of the team, stakeholders accept working software at reviews in lieu of other tracking or progress metrics
6. Frequent (near-real time) prioritization of old and new functionality
7. Build automatically deploys to QA environment available to any interested party

Responsiveness to business

1. Frozen specification, unresponsive to business value
2. Team tries to accommodate new requirements by ad hoc change requests ("pile it on")
3. Iterative process with sprints of length short enough to respond to business change
4. Showcases per sprint; business prioritizes functionality per sprint
5. Continuous involvement of the business on the team: business certifies story complete out of dev; involved in (rather than approval of) story definition and acceptance tests
6. Business writes high-level stories for all requests; organizational story queue exists; prioritization decisions made from full story backlog
7. Development process is integral to business initiatives; development teams work as a part of the business unit rather than as a service to the business unit

Assurance and Governance 

1. Status reports document progress; schedule acts as plan
2. Concept of release planning is introduced
3. Sprint planning is introduced
4. Plan becomes communication and tracking tool rather than an exception report
5. Every sprint review examines value delivered and assesses next sprint by need, priorities (and delivery)
6. Organizational adoption: Introduction of Portfolio Planning and global optimization of resources (use of metric and standard measures)
7. Business review of value and return helps teams on regular basis based upon status

Story Formation (Requirements)

1. Development tasks extracted from voluminous, frozen requirements artifacts
2. Production of lightweight artifacts (e.g. panel flows) to drive high-level requirements development
3. Mapping granular requirements (e.g., use cases) in high level user requirements
4. Marshaling use cases into discreet statements of functionality that can be delivered in time boxed development sprints
5. Stories are an expression of end-to-end functionality to be developed, including testable acceptance criteria and a statements of value
6. Spontaneous story development provided by business for in-flight projects; stories not derived from extant sources but are immediate expressions of customer demand/need
7. Global repository of functional requirements for stories developed by business for any business requirements, including a formal measure of business value delivered

Simplicity

1. Big up-front design that attempts to accommodate all potential future needs
2. Application of fundamental design patterns
3. Structural refactoring to decouple application architecture
4. Doing only what needs to be done
5. Aggressive and constant refactoring to improve code quality/simplicity extant code
6. Spiking solution with each release to introduce new ideas and challenge architectural decision
7. Technical design decisions taken with each story

Build

1. Ad-hoc build requests; scripting and component marshalling performed manually
2. Consistent, repeatable build process with durable artifacts executed manually with each release
3. Build is automated, executed on a timed basis
4. Unit tests are integrated with the automated build; team is constantly notified of the status of the build; build triggered by SCM updates
5. Test and metrics (e.g., code quality, complexity, check style, etc.) are integrated as gatekeeper events; build data archived and reported in build portal/dashboard
6. All product components advertise dependencies; master repository established
7. Product integration tests included; build process executes automatic deployment to QA testing environment

This approach ressembles in a way to how CMMI is expressed as a model. It lists dimensions (key process areas) and then different maturity levels for each one. The model's scope is very narrow and well expressed (IMO) and it could be useful for an organization to define how responsive in agile terms is the development department to the business.

The 5 levels of Maturity

Martin Proulx sketch this agile maturity model:

Level 1 – Team Level Maturity

At this level, team members have decided to adopt Scrum and/or software engineering practices without asking for approval from their manager. Some of the well known practices are used but without consistency.

Level 2 – Department Level Maturity

At this level, the practices adopted by the team members have started to be imitated by other teams within the software development department. Some of the managers have noticed the positive results of adopting the Agile approach and are tempted to replicate what they observed.

Level 3 – Business Level Maturity

At this level, the solution teams have integrated the business people in the model. Collaboration (and trust) has increased and a partnership relationship is increasing.

Level 4 – Project Management Level Maturity

At this level, the project management approach is modified to include some of the Scrum practices. Although the department still mostly relies on the traditional PMBOK recommendations, Scrum has been integrated in the project management approach.

Level 5 – Management Level Maturity

At this level, managers have adapted their management style to support an Agile organization. Organizational structures and reporting mechanisms are better adapted for collaboration and improved for increased performance.

This model measures basically how spread are the Agile values throughout the organization, starting from the team level with certain practices until reaching the maximun level, where the whole organization shares the same values and behaviors. 

How to measure Agile maturity then?

It seems there are 2 dimensions to be included when measuring the Agile maturity:

• How mature are the Agile practices and process: this is basically the 2nd approach listed above and applies mainly at the project level (or team level using the 3rd model above). How mature are you to do testing? You do all testing manually (or not do it at all!) or you have a solid testing strategy that includes a great deal of automation. How mature are you programming? How mature are you managing your requirements? In all these dimensions, it can be defined an agile way of doing things. The 2nd approach uses behavioral patterns. Another option would be to use objectives and key practices. 
• How spread are these Agile practices and processes throughout the organization: Basically this point says that a mature Agile organization does business in an Agile way. It is fast to deliver business value, it adapts, there is a good level of communication and collaboration throughout the company, etc. A model that encompasses this dimension is much mode ambitious. But probably it is the only way of making Agile work in an organization. The way the software is constructed needs to fit with the way the organization does business around it. The structure of the organization should enable and encourage the right connections that make communication smart and effective throughout the organization. The organization needs to have also the structure to learn and maintain this Agile culture.

CMMI follows a similar approach as it starts with the practices dimensions (level 2) and follows towards a standarized process in the organization (level 3). This seems the natural projection to follow when seeking maturity. Starting with mastering Agility at the team level and then managing the whole organization with the same values and principles.

For both dimensions, a means of measuring maturity is needed. A first attempt using the Shu-Ha-Ri scale will be used.

A very inmature Agile organization would then be an organization where the practices are in the SHU-SHU level at the practices dimension and organization dimension. A RI-RI organization is one that has mastered the Agile practices at the team level and that manages its whole business with Agility.

Agile Capability Maturity Model

Agile Capability Maturity Model

After all the differences we have talked about, the question arises. Does it make sense to have a Capability Maturity Model for Agile? How would it be? Let’s analyze the semantics of the words again, but in an Agile context.

What does it means to be capable, in an agile context?

In the CMMI model, Capability was defined as “the inherent ability of a process to produced the planned results. As the capability of a process improves, the process becomes predictable and measurable, the quality of the result product augments and the productivity of the teams too”. Thus, an organization gets more capable when it has defined and implemented a set of processes throughout the organization that enables it to be predictable, measurable, and continuously improving.

Does the word have the same meaning in an agile context? Agility is more like a mindset or culture than a set of processes. Thus, in Agile you are capable when you have the mindset and the organization is capable when it has the mindset. The semantics of the word changes considerably and worst of all, it makes it unmeasurable. How would you measure how much of the mindset the organization has? Nonetheless, and perhaps very subjectively, one can recognize a team/organization that is more Agile than another. You do it because it adheres better to the Agile values (communication, collaboration, technical excellence, etc). Using this subjectivity, perhaps we can define the meaning of capable in an Agile context. One is capable when it has grasped the Agile values, when you are at the Ri level (in the Shu-Ha-Ri scale) or, as some people say, when you ARE agile. 

How do you get more mature?

Using CMMI, you get more mature as you implement the different processes that conform the model throughout the organization, you can measure them and draw conclusions out of the metrics that allows the organization to improve. 

If we follow the definition of capability in an Agile context given above, the definition of maturity falls in the subjective side as well. An organization is more mature when it can communicate better, it can collaborate better, is able to produce its products with the required quality sustainably, etc. 

Senge talks about the intelligent organization. A mature Agile organization is an intelligent organization. It’s an organization where its members are smart agents that collaborate among them to solve problems and improve continuously. Today’s knowledge organization need to mature in this direction.

The Schneider Culture Model

This diagram is taken from The Reengineering Alternative: A plan for making your current culture work, the book by William Schneider. In it, there are 4 cultures. Each has its name, picture and some words that characterized the quadrant.

There are also two axis that indicate where the focus or an organization is:

1. Horizontal: People Oriented (Personal) vs. Company Oriented (Impersonal)
2. Vertical: Reality Oriented (Actuality) vs. Possibility Oriented

And if we map the Agile values and principles to this model...

We can see how Agile mainly falls into the collaboration and cultivation cultures. I believe that the introduction of CMMI in an Agile Culture moves the culture to the “Control” quadrant. Many people believe this is the way to scale agile and therefore it could be argued that CMMI could be a good way to scale Agile. I don’t feel very comfortable with this idea, as I believe that a good maturity framework should allow the organization to mature without losing its values, its heart. In other words, I believe the maturity needs to happen in the collaboration/cultivation quadrant mainly and the competence one. 

Conclusion

A capability maturity framework for Agile needs to redefine the meanings of capability and maturity to suit the context as being capable and mature have very different semantics in the CMMI world and the Agile one. An Agile Capability Maturity Model should make the organization mature towards a collaboration/cultivation culture. The Agile definitions of capable and mature makes introduce a lot of subjectivity. Could we still measure the results?