Wednesday, December 29, 2010

Is the Process and Product Quality Assurance Process Area compatible with Agile?

This is a special post number 50!! So....

Enough, let's go back to Agile and CMMI compatibility..


The Process and Product Quality Assurance Process Area, a process area at maturity 2, aims at objectively evaluating projects for adherence to the standards processes and procedures. Objectivity is key. It can be achieved through an external group or through demonstration of objectivity.

Below are the specific goals and practices:

SG 1 Objectively Evaluate Processes and Work Products
SP 1.1 Objectively Evaluate Processes
SP 1.2 Objectively Evaluate Work Products and Services
SG 2 Provide Objective Insight
SP 2.1 Communicate and Ensure Resolution of Noncompliance Issues
SP 2.2 Establish Records

Is it compatible?

This process area aims at ensuring that each of the projects in the company follows the company’s defined processes and practices. Something that can make this process area difficult to apply in an Agile company is the empiricist nature of Agile methodologies. In one side, there’s a company defined methodology (and the willing to follow it represented in this process area) and on the other there’s the Agile encouragement to introspect on the methodology and modify things that are not working. Probably this goes back to the advantages and disadvantages that I mentioned in the introduction to this series of posts and also to the fact that Agile is a value system. Therefore (and again), probably it’s more important to ensure that the company’s values (which in an Agile company should be compliant with the Agile values) are followed than to ensure that a specific defined process is followed step by step.

Another thing I also wonder is what is the best mechanism to provide this insurance.  The CMMI reckons that “Traditionally, a quality assurance group that is independent of the project provides this objectivity.”. However, it also states that “It may be appropriate in some organizations, however, to implement the process and product quality assurance role without that kind of independence. For example, in an organization with an open, quality-oriented culture, the process and product quality assurance role may be performed, partially or completely, by peers; and the quality assurance function may be embedded in the process.”. I believe that the second mechanism should be the one recommended in Agile environments. A mature Agile team is one that doesn’t of any external group to follow the processes and practices and that realizes when it’s incurring in technical debt and can recover and pay it. How does it get there? Through a leader embedded in the project. A leader is a software craftsman that teaches the rest of the group the practices and ensures that they are always followed, even in tempestuous times (when the only thing that matters is handling some software by the door!)


Exact adherence to processes shouldn’t be encouraged in Agile environments, but exact adherence to values should. The best mechanism to ensure quality are through leaders embedded in the projects that act as quality and process keepers. 

Other Process Areas

- Is the Requirements Management Process Area compatible with Agile?
- Is the Project Monitoring and Control Process Area compatible with Agile?
- Is the Project Planning Process Area compatible with Agile?

Thursday, December 23, 2010

Is the Measurement and Analysis Process Area Compatible with Agile?


The measurement and Analysis process area, a process area at maturity 2, aims at developing a measurement capability that is used to support management information needs. Measurement objectives based on identified information needs are defined and using these objectives, objective and quantifiable measures are defined. Mechanisms to collect and store the data should also be defined.

Below are the specific goals and practices:

SG 1 Align Measurement and Analysis Activities
SP 1.1 Establish Measurement Objectives
SP 1.2 Specify Measures
SP 1.3 Specify Data Collection and Storage Procedures
SP 1.4 Specify Analysis Procedures
SG 2 Provide Measurement Results
SP 2.1 Collect Measurement Data
SP 2.2 Analyze Measurement Data
SP 2.3 Store Data and Results
SP 2.4 Communicate Results

Are they compatible?

Agile possess a culture of continuous improvement, through continual inspection and adaptation. The concept of Kaizen, that comes from the Lean world is also very popular. The question is ‘how is this improvement attained?’ and ‘is it always aligned with the wider company objectives?’.

How is this improvement attained? How should be the process to improve? The mechanism suggested by CMMI is the only one I know. The objective is identified, measurements that tell us if the objective has been reached as well as a plan to get there are defined and then successive measures are taken through the path. Having a company wide process improvement framework like CMMI can help identify these objectives in a project more easily and also assures that the objectives selected are aligned with the wider company objectives. It can also help define the measures, which should be “precise, quantifiable measures”.

However, I have a couple of concerns related with setting objectives and trying to reach these objectives:

1. Not weighting properly the importance of the objective within the whole context of the project

Let me picture this with an example, for which I will use Highsmith’s Agile triangle.

We may set up our objective as “Improve prior levels of quality” (taken from the CMMI examples). What is a quantifiable precise measure for this objective? Probably most will say counting the number of raised bugs. Having no bugs is really wonderful, but having no bugs on a software that doesn’t do what the customer needs is really valuable?

2. Creating dysfunctional behaviours

The human beings find ways to reach the objective that don’t match exactly what the objective was created for. Having raised less bugs assures us that the software is more robust? Are we really doing things better? The human/social aspects could be tricky.

Another thing to keep in mind when devising measurements and processes to be used inside a project is that all these processes and measures are “waste” (things that don’t add value to the product). Of course, this could all be good waste, because it will allow the company to improve, to do things better in the future. Nonetheless, they should be treated as such and designed to be as light as possible. Probably, automation (again) is the key in this regard.


I believe that creating more formal mechanisms to improve, that are aligned with company objectives, could be beneficial to Agile projects which in general focus their improvements efforts informally and with a project scope. The real objectives and the human aspects should be taken into consideration to avoid optimizing in the wrong side. The improvements processes should be light and agile.

Other Process Areas

- Is the Requirements Management Process Area compatible with Agile?
- Is the Project Monitoring and Control Process Area compatible with Agile?
- Is the Project Planning Process Area compatible with Agile?

Sunday, December 19, 2010

Is the Requirements Management Process Area compatible with Agile


The requirements management process area, a process area at maturity 2, deals with managing all requirements related to the project throughout its life.

Below are the specific goals and practices:

SG 1 Manage Requirements
SP 1.1 Obtain an Understanding of Requirements
SP 1.2 Obtain Commitment to Requirements
SP 1.3 Manage Requirements Changes
SP 1.4 Maintain Bidirectional Traceability of Requirements
SP 1.5 Identify Inconsistencies Between Project Work and Requirements

SG 1 Manage Requirements

The only objective in this process area states that requirements should be managed by doing the following:

  • Managing all changes to the requirements
  • Maintaining the relationships among the requirements, the project plans, and the work products
  • Identifying inconsistencies among the requirements, the project plans, and the work products
  • Taking corrective action

All of this is completely true in an agile environment. However, the practices recommended to get there is very different. Let’s analyze each of the practices.

CMMI lists the following as the recommended practices to manage requirements:

1. Obtain an understanding of the requirements. When and how complete? Well, the objective states that “The project maintains a current and approved set of requirements” so my assumption is this understanding is complete.
2. Obtaining commitment to Requirements. From those “who have to carry out the activities necessary to implement the requirements.”
3. Manage Requirements Changes. Analyze impact and document rationale.
4. Maintain Bidirectional Traceability of Requirements. From requirements to source code and vice-versa. This traceability helps determine requirements have been completely addressed.
5. Identify Inconsistencies Between Project Work and Requirements.

Requirements are managed differently in Agile. Let’s see how the practices would change:

1. The project backlog doesn’t represent a current approved set of requirements. Requirements in a backlog have different levels of understanding and approval. Requirements live their life in the backlog with different levels of understanding and approval. They are born as ideas, “topics of conversation” and grow their understanding and approval, even after they are implemented. The driver in this understanding is their priority. More important requirements need to be understood/approved first. There could be phases in the form of releases in the backlog that will determine the understanding and approval of a set of requirements.
2. It is important the Agile recommendation of having cross-functional teams. A jelled team which contains all the necessary expertise represents the easiest path for full commitment.
3. Changes in the backlog should be managed lightly, but continually. In fact, I am not even sure that they should be named changes. As previously mentioned in point 1, a requirement grows in the backlog, perhaps changing its status, form and priority. This doesn’t mean that Agile embraces high volatility or churn. All stakeholders should be aware of the impact of the changes. A change may be important, but is never free. The Product Owner, responsible for the ROI is the one to make the trade-off.
4. Maintaining traceability could be very time consuming and only helps determine that requirements have been addressed. Acceptance Tests are less time consuming are help determine that requirements have been fulfilled.
5. Again, automated acceptance tests are the recommended practice to detect and fix inconsistencies in an Agile project. Doing this manually would be too time and mind consuming. Computers should maintain all this knowledge gained and detect the inconsistencies for us. The second recommended practice is to involve the customer through all the project. Customers are the best persons to detect inconsistencies (should we call them inconsistencies?)

What is missing in the recommended practices?

Prioritization: One of the most important aspects in managing requirements in an agile environment is managing priorities. Prioritization works starts at project inception and follows throughout the project. Of course, this is a very difficult task as it involves not only understanding the business value that different requirements add but also making trade-offs based on technical feedback and measures of effort.

Requirements live and Requirements light: One of the most difficult and time consuming tasks during a project is to maintain requirements up to date. If doing this is too time consuming, agility suffers. Many agilists suggest that the best way of maintaining them is through maintaining them as executable requirements (or executable specifications, examples, etc.). An executable requirement must be maintained and it always up to date. Being able to execute/debug this requirements allows to easily trace requirements with source code.


I believe the only goal of this process area is entirely applicable in an agile environment, but the path to get there goes in a complete opposite direction. The CMMI recommendation is to get a complete approved set of requirements, which is maintained and traced to the source code (and viceversa) throughout the life of the project. The agile recommendation is to maintain a prioritized executable specification at all times. The later mechanism is lighter, assures is up to date and automated.

Other Process Areas

- Is the Project Monitoring and Control Process Area compatible with Agile
- Is the Project Planning Process Area compatible with Agile?

Saturday, December 4, 2010

Is the Project Monitoring and Control Process Ares compatible with Agile?


The Project Monitoring and Control area, a process area at maturity level 2, aims at understanding the project’s progress and taking corrective action when the project’s performance deviates significantly from the plan.

Below are the specific goals and practices:

SG 1 Monitor Project Against Plan
SP 1.1 Monitor Project Planning Parameters
SP 1.2 Monitor Commitments
SP 1.3 Monitor Project Risks
SP 1.4 Monitor Data Management
SP 1.5 Monitor Stakeholder Involvement
SP 1.6 Conduct Progress Reviews
SP 1.7 Conduct Milestone Reviews
SG 2 Manage Corrective Action to Closure
SP 2.1 Analyze Issues
SP 2.2 Take Corrective Action
SP 2.3 Manage Corrective Action

SG 1 Monitor Project Against Plan

This goal states that there should be a way to compare actual performance, against what was established in the plan. Among the things that are measured are project planning parameters,  commitments, risks, stakeholder’s involvement, progress reviews and milestone reviews.

I believe in Agile, there is a shift from traditional management in who does the monitoring, how it is performed, what is measured and also in the objectives of the monitoring. These changes should be reflected in the process maturity framework.

First, who does the monitoring and and how it is performed? In Agile, this is the not the responsibility of the Project Manager, but of the entire team. Agile has 2 main practices that enable monitoring that are very effective because they encourage self-organizing teams (increasing accountability and enthusiasm as a side effect). The first enables day to day monitoring and is called Scrum daily meeting if you come from the Scrum world or standup meeting if you come from the XP world (though in Argentina, most of the people call it “the scrum”). In this meeting, the team as a whole “monitors” how it is doing with the tasks, which are the blocking issues and how is the iteration progressing. It is important to stress that there isn’t a central authority to report to in this meeting. It is strictly a team’s meeting. The second monitoring meeting practice is the retrospective. This meeting happens after each iteration and enables to monitor how the team performed in the iteration and so far in the project. This meeting allows for a more thorough “monitoring” or introspection as we like to call it and bigger corrective actions can be taken. Another practice suggested for monitoring is the use of Visual Radiators which allow anyone to constantly monitor the project’s progress.

What is measured? As the Project Planning process area suggests an activity based project plan, the project monitoring suggests to monitor “the activities and milestones”. As previously said, Agile does not encourage activity-based plans. Basically, work products that don’t deliver business value aren’t really considered progress in Agile terms. So what should be measured? Certainly, the best measure of progress is to be able to measure how much business value has been delivered. Although this is really difficult (and I must confess I’ve never done it), there are techniques that allow to monetize the value of the features (see Value Point Analysis in Highsmith's Agile Project Management book). Another measure is effort, but it should be stressed that progress is made when running, tested and customer accepted features that deliver value are completed. Progress should be tangible. Intermediate work products don’t represent progress.

Finally, what about the objectives? A successful project is not one whose measures match the plan, but the one that delivers the value expected by the customer. Bjarte Bosgnes, in Implementing Beyond Budgeting asks a simple question, “What is the best performance: delivering 100 against a target of 100, or delivering 105 against a target of 110?”. Most people would agree that 105 is better than 100, however most performance measurements systems would call 105 against 110 target a failure [from Agile Project Management]. As Highsmith states in his book, I believe an agile process maturity framework should shift its objectives from the constraints (scope, schedule, cost) to delivering value.

There is another topic within this goal that I would like to tackle. CMMI suggests to record actual parameters to be used as historical data. Something I am afraid of when measuring actual parameters (actual times) is to use them as a pressure mechanism. I believe that measuring actual times of taks and comparing them to estimated times lead to dysfunctional behaviours that damage the project (like overestimation or cutting corners - dropping quality - to achieve those times)

SG 2 Manage Corrective Action to Closure

The second objective states that corrective actions should be managed to closure when project’s performance or results deviate significantly from  the plan.

What does it mean that corrective actions are “managed” to closure? It means there is a manager that assures that the actions take place. How do we get that assurance in Agile, inside a self-organizing team without a manager per se? People inside the team should be hold accountable for leading the corrective actions to closure. This, in my experience, is sometimes difficult as the informality of the process sometimes leads some people to not take the corrective actions seriously (specially in less mature teams, previously used to command and control structures). However, as CMMI states, corrective actions should be closed and there should be a mechanism that forces the team to do so.


Monitoring and taking corrective actions are necessary within any project. The way CMMI suggests to do it seems very oriented to traditional project management. Agile project management takes a big shift in what is measured, how it is measured and in the objectives of measuring. This should be reflected in the process maturity framework to be compatible with Agile.