Software Development Methodologies: Lean vs Agile Principles

Published Aug 13, 2017Last updated Sep 10, 2017
Software Development Methodologies: Lean vs Agile Principles

“Lean” and “Agile” are terms that have been thrown around a lot recently, often in reference to software development methodologies, project management, or organizational styles.

But do you ever find yourself wondering:

What is Lean? What is Agile? Is there a difference?

Merriam-Webster Dictionary defines Agile as “having quick resourceful and adaptable character, or marked by ready ability to move with quick easy grace.”

Lean is defined simply as “thin and healthy, or containing little or no fat.”

Based on these definitions, we can assume that someone who is lean and someone who is agile could have many shared characteristics. The same is true in the context of software development.

What is Agile? What is Lean?

Agile is now widely known in the technology world as a set of values and principles to guide the development of software. “The Agile Manifesto” lays out a set of four values and 12 principles. Distilled to its core, Agile is exactly what you think it might be. It’s Agile. The methodology favors flexibility, communication, collaboration, and simplicity.

Lean is less understood and lacks a clear cut definition supported by a professional consensus.The term “Lean” was originally coined to describe a manufacturing organization model based on the Toyota Production System, but is commonly considered a sub framework within the Agile umbrella of software development.

Today, there is much confusion about what is Lean, what is Agile, if they are one and the same, and which should be used.

The Birth of New Software Development Methodologies

Both Lean and Agile were developed in response to the shortcomings of existing plan-driven methods like Waterfall. Used since the 1970s, developers and software engineering managers began to notice the inefficiencies of Waterfall by the 90s. With more dynamic markets and tech savvy consumers, Waterfall was unable to respond quickly enough to market demands, changing technology, or deliver bug-free software on a consistent basis.

In pursuit of a better model, the creators of Lean and Agile sought to develop methodologies with a more customer-focused approach. The new methodologies embraced the ability to adapt as a competitive advantage, favored early and continued testing, and brought a human element into project management and execution.

Lean vs. Agile principles

Lean Software Development (LSD) was first proposed by Dr. Robert Charette as a way to build change-tolerant organizations that were becoming increasingly dependent on software.

Next came “The Agile Manifesto” which enshrined the 12 principles of Agile Software Development.

The other authoritative work on software development methodologies is credited to Mary and Tom Poppendieck, who published Lean Software Development: An Agile Toolkit.

Here is a side-by-side comparison of the values and principles of each:

Methods Chart

Comparing the 12 principles of Dr. Charette’s LSD and the 12 principles of Agile, you can see that they are strikingly similar. The seven Lean principles proposed by the Poppendiecks are less targeted, but nevertheless overlap with “The Agile Manifesto” and Charette’s Lean Software Development.

Here are more elements they all have in common:

  • The value of responding to customer needs quickly
  • Early testing
  • An iterative approach to development
  • MVP (Minimum Viable Product) style of development over feature heavy
  • Cooperation both within the company and with outside stakeholders

Timeline

We’ve investigated the watershed events and publications that gave birth to these terminologies to see how they became popular. Check out our timeline below detailing the progression, and add these to your reading list if you are so inclined!

Timeline
[1]

Origins of Lean and Agile

As you can see from the timeline, the term Lean was first used by Womack et. al. in 1990 to describe the Toyota Production System in their book, The Machine That Changed The World. Dr. Robert Charette later adapted Lean ideas described in earlier publications to create his “Lean Software Development”.

The term Agile was not was not widely adopted until the publishing of “The Agile Manifesto” in 2001. The terms Agile and Lean were both coined by western technology professionals or academics who were referencing the Toyota Production System (more on this later).

We might catch some flak for saying so, but with this in mind, the terms Lean and Agile are not actually that important. Having two terms stemming from the same principles actually contributes to confusion on the subject. That being said, this is the terminology the industry has adopted, so moving forward we will continue to use them.

The timeline is also another source of confusion. Dr. Robert Charette introduced his ideas on Lean Software Development in the early and mid-90s. The tactical purpose and 12 principles of his Lean Development approach were described in 1998 in an article titled, “Lean Development,” nearly three years before the “The Agile Manifesto.” In a testament to the overlap between Lean and Agile, this article was written by Jim Highsmith, who later became a core founder of the Agile movement. However, Highsmith’s article was not widely circulated, and it wasn’t until 2003 that Charette himself wrote a more in-depth explanation behind his lean development approach in the research report, “Challenging the Fundamental Notions of Software Development.”

In an email exchange with Dr. Charette, he was quick point out that his conception of Lean Development was intended for the organizational level within the software field:

Mine was borne out of a strategic as well as operational need to improve IT’s business/mission value to the organization, and I approached this from a management of risk perspective.
-Dr. Robert N. Charette

This differs slightly from Agile, which was first proposed strictly as a “better way of developing software,” but now is applied to various projects and management methods.

2003 was also the year the Poppendiecks published Lean Software Development: An Agile Toolkit. This book detailed seven principles of Lean Development, which correlates directly to the seven forms of waste in Lean Manufacturing.

The Poppendiecks’ book simultaneously bolstered Lean as a software development methodology and blurred the distinction between Lean and Agile, by proposing Lean as a complementary method within Agile. In fact, at the time of publishing, the book was sold as the latest publication within The Agile Software Development Series.

Today, the Poppendiecks’ multiple works on the subject are considered essential reading for Lean, and “aspiring-lean” software development practitioners.

Divergence in Application

According to Dr. Charette, one of the primary differences between Lean and Agile is that Agile is bottom up, while Lean is top down. This is evident in Lean’s end-to-end (E2E) structure and the principle of See the Whole proposed by the Poppendiecks. Below we explain these principles at work in the practice of value stream mapping.

Value Stream Mapping

To realize the principle of See the Whole, the Poppendiecks detail a method of value stream mapping to reveal the value-added activities versus the non-value added activities throughout the end-to-end development process.

Value stream mapping analyzes the development cycle from the time a requirement is received to the time it is delivered to the customer. The goal is to identify the wastes of sitting inventory and waiting (delays in production), and explore new practices to reduce Work-in-Progress (WIP) and lead time.

According to the Poppendieck’s, mapping your value stream is a simple exercise that only requires a pencil and a piece of paper. The process is as follows:

  1. Map your current value stream (beginning with a requirement and a timeline of actions on the journey to delivery)
  2. Analyze the biggest cause of waste (What is holding up Work-in-Progress?)
  3. Develop a plan to cut this waste in half
  4. Create a future value stream map

The Agile paradigm as laid out in “The Agile Manifesto” favors short iteration cycles and frequent deliveries over a holistic end-to-end view. The E2E focus is therefore unique to Lean. In fact, it is because of the E2E construct that Lean (rather than Agile) is more often applied as an organizational structure and management style.

The end-to-end view necessitates that the whole organization takes part in order to eliminate waste. This echoes Dr. Charette’s stated purpose for his original Lean Development concept:

Lean Development is a philosophy, a way of seeing and thinking about IT and its relationship to an organization, as much as it is a development approach.

Lean Kanban Vs. Agile Kanban

Both Lean and Agile have adopted the TPS Kanban system with slight variations. Toyota’s Kanban system was designed to limit Work-in-Progress.

As opposed to traditional “push manufacturing,” which pushes inventory to the next step in the process, Kanban only pulls new material into production once the current piece has been processed and components need to be replenished.

Kanban cards become resupply orders and are sent back to the previous step in production. As a result, Work-in-Progress is minimized, and idle inventory is reduced.

In software development, instead of passing Kanban cards from one manufacturing step back to the previous one, a Kanban board is used. Sticky notes are used, most often to represent requirements in progress.

kanban board.jpg

According to the chapter contributed by Kai Petersen in Modern Software Engineering Concepts and Practices: Advanced Approaches, both Agile and Lean use a prioritized list of requirements to pull tasks from.

Unlike Agile, which uses fixed duration iteration cycles to limit the time of development and govern the Kanban board, Lean limits the number of tasks allowed at any given time. This allows Lean to fulfill its primary goal of limiting WIP, while more accurately measuring lead-time, and identifying waste in production. Once a task is completed, a new one can be pulled from the prioritized list. While Lean uses the concept of continuous flow, Agile begins each new iteration with a fresh board.

Some teams recognize the benefits of both approaches, and are beginning to use a hybrid method known as scrumban.

These are just two of the subtle differences in approach Lean and Agile take to achieve common goals. Another article published on Codementor explores more of the uses and applications of Lean and Agile.

In practice, whether your team takes a so called “Agile” approach or a “Lean” approach is unimportant. What’s important is finding the right practices to optimize your workflows and consistently deliver value to your customers, which both methodologies see as the ultimate purpose.

The TPS Connection

So what does this all have to do with the Toyota Production System? Pretty much everything. The creators of both Agile and Lean were heavily influenced by TPS, as Womack et. al. describe in The Machine That Changed the World. To better understand the inspiration for Lean and Agile methodologies, we will take a look the manufacturing system developed in Japan between the 1950s-70s, specifically:

  • Just-in-Time Manufacturing
  • Jidoka (intelligent automation)
  • Eight Forms of Waste
  • TPS Values

Warning:

The rest of this article contains jargon that you can use to sound scholarly after reading.

Just-in-Time Manufacturing

Following WWII, Toyota was on the brink of collapse with a damaged supply chain and depression-level demand for their automobiles. The company could not hope to follow a Detroit model of mass production and survive.

Under these conditions, Taiichi Ohno and Kiichiro Toyoda set out to remain profitable by eliminating waste in production, reducing lead time, and only producing what customers needed, also known as Just-in-Time (JIT) manufacturing.

To eliminate waste, Ohno resolved to make only what was needed, when it was needed, and only in the amount it was needed in.

The iterative process of Lean and Agile which focuses on minimizing feature development, and maximizing delivery of updates directly mirrors Toyota’s Just-in-Time manufacturing.

Jidoka

Jidoka (自働化) can be translated as automation with a human touch, sometimes referred to as “intelligent automation.” Jidoka plays a major role in eliminating waste in production by making machines more independent which frees up people to play a more active role in production and unlocks human creativity.

Jidoka relies on intelligent machines that stop automatically when there is an irregularity. Human workers can then go and fix the problem, stopping defects from being passed down the production process.

Jidoka also empowers every employee to stop the production line when a problem is detected. The principle includes a four step process to eliminate the waste of defective products:

  1. Detect the abnormality
  2. Stop production
  3. Fix or correct the immediate condition
  4. Investigate the root cause and remedy the situation

You can see the Jidoka concept in the Lean and Agile focus on early and frequent testing to root out bugs, and emphasis on customer consultation to pinpoint user pains.

Wastes of TPS in a Software Development Context

To achieve JIT manufacturing, Taiichi Ohno outlined seven forms of waste to be eliminated. Number eight was added later. If you take a closer look at Agile and Lean’s values, goals, and principles, you can see that they are designed to guard against the eight wastes of TPS. In their 2003 book Lean Software Development: An Agile Toolkit, the Poppendiecks presented TPS wastes in a software development context.

1. The Waste of Overproduction. The waste of overproduction is one of the biggest reasons the Waterfall method has been abandoned. Overproduction is considered extra coding for features that weren’t requested and that the customer may not want.

This is reflected in the following principles:

Agile ⟶ simplicity
Charette’s Lean⟶ minimalism
Poppendiecks’ Lean ⟶ eliminate waste

2. The Waste of Waiting. In JIT manufacturing, waiting on an idle machine or worker is wasteful. In software development, waste is waiting on a team with excess capacity. If there are delays in production that cause a team to be on standby, or cause the customer to wait for delivery, there is waste.

Matching principles are as follows:

Agile ⟶ frequent cycles
Charette’s Lean ⟶ ⅓ of the time (goal of LSD), 80% solution today
Poppendieck’s Lean⟶ deliver as fast as possible

3. The Waste of Transportation. In software development, transportation is translated as “task switching”. Too many handovers or employees assigned to multiple teams with a demand for excessive multitasking is inefficient and a waste.

Matching principles:

Agile ⟶ stakeholder collaboration, self-organizing teams, culture of
trust, support, and motivation

Charette’s Lean ⟶ team effort
Poppendiecks’ Lean ⟶ empower the team

4. The Waste of Overprocessing. This is simply extra processes that aren’t really needed to deliver value to the customer. The primary example is documentation. Excessive documentation for inflexible processes will not be valuable, nor are overly detailed user manuals which could be summed up in a page or two. Documentation is time-consuming yet offers limited value to the end-user.

Matching principles:

Agile ⟶ simplicity, working software as the measure
Charette’s Lean ⟶ minimalism
Poppendiecks’ Lean ⟶ eliminate waste

5. The Waste of Inventory. Inventory waste is Work-in-Progress for which an investment has been made, but holds no value until completion. In other words, incomplete software provides no value to the user. Lean and Agile value fast and frequent deliveries, with an emphasis on frequent iterative cycles and delivery of working software.

Matching principles:

Agile ⟶ customer satisfaction (through early and frequent customer
delivery)
Charette’s Lean ⟶ 80% solution today
Poppendieck’s Lean ⟶ deliver as fast as possible

6. The Waste of Movement. Waste of movement is excess effort required to get information or answer questions. This is common when teams are not co-located, if tasks are completed and results are not made available immediately to all relevant parties, or when stakeholders are not readily available for consultation.

Matching principles:

Agile ⟶ stakeholder collaboration, face-to-face communication
Charette’s Lean ⟶ customer participation, team effort
Poppendiecks’ Lean ⟶ eliminate waste

7. The Waste of Defects. Like in manufacturing, producing defective or buggy software represents a wasted investment by the company. The quicker a defect is detected, the more likely waste is mitigated. Many Agile and Lean principles seek to halt the waste of defects. To reduce defects, all three methodologies place a premium on early and frequent testing.

Matching principles:

Agile ⟶ technical excellence, working software as a measure
Charette’s Lean ⟶ customer satisfaction
Poppendiecks’ Lean⟶ amplify learning

8. The Waste of Unused Employee Creativity. In software development, unused creativity results from a rigid roadmap and lack of human collaboration. Just like Jidoka (自働化), Agile and Lean seek to maximize human cooperation and innovation to get the best results out of technology.

Matching principles:

Agile ⟶ stakeholder collaboration, team reflection
Charette’s Lean ⟶ “unwritten” 13th principle of satisfaction through
work

Poppendiecks’ Lean ⟶ amplify learning

TPS Values in Software Development Methodology

Many of the core values that make up TPS are also reflected in Agile and Lean software development methodologies.

Kaizen (改善) translates as continuous improvement. With flexible, iterative, customer focused models, continuous improvement is perhaps the most important value of Agile and Lean software development methodologies.

Hansei (反省) means self-reflection. As a means to improve efficiency, The Agile Manifesto directly adopts team self-reflection as its 12th principle. In introducing his Lean Development model, Dr. Charette challenges readers to reflect on all of the current assumptions that dictate the processes they use as an initial step to finding better ones. The Poppendiecks’ amplify learning principle can be considered Hansei as well.

Respect for People (尊重) is central to all three paradigms. The first value of “The Agile Manifesto” is to “value individuals and interactions over processes and tools.”

Respect for people also appears in Dr. Charette’s assertion of the importance for individuals to experience satisfaction through work, and the Poppendiecks’ Lean principle of empowerment of teams. This value recognizes that when individuals are involved in decision making and improving their work environment, they are more innovative and efficient workers.

Seiri (整理) is the principle that mirrors waste. Seiri dictates that what is unnecessary should be removed. This encompasses all of the original seven wastes of TPS, for which we have already identified the parallel waste in the software development environment.

Gengchi Genbutsu (現地現物) literally translates to “actual place, actual thing.” In practice, this means “inspect to understand.” When there is a problem in the production process, the manager should go to the source to understand the problem and determine how to solve it.

In software development, this value is manifested in the emphasis on short iteration cycles, early testing, and customer collaboration so that software engineers can build working software that users actually want.

Conclusion

As we mentioned in the opening paragraphs of this post, Lean methodology is less well understood and is often considered to be an Agile practice. Lean is perhaps less well-defined simply because of its broader applications.

Lean has a more direct relationship with the Toyota Production System and was first proposed as an organizational set of methods and practices for business management, and only later applied to software development. Agile, on the other hand, was developed specifically for software development by dedicated professionals in the field.

After detailing the shared background and general principles of these two methodologies, you can see that these two paradigms have more in common than they have differences.

One of the primary authors of “The Agile Manifesto,” Martin Fowler, who has also worked closely with the Poppendiecks, has pointed out that Lean and Agile are not mutually exclusive:

Lean and Agile are deeply intertwined in the software world. You can’t really talk about them being alternatives….you don’t do Agile or Lean, you do Agile and Lean.


  1. The 12 principles of Charette's Lean Software Development were actually first described in Jim Highsmith's article "Lean Development" in 1998. This was later elaborated in Dr.Charette's own article "Challenging the Fundamental Notions of Software Development" ↩︎

Hire a top developer for your team.