A Django site.
November 17, 2008

Peter Abilla
no nic
shmula
» Law of Instinct

time.jpgI love data, but not much credit is given to hunch; gut, instinct.  Colin Powell, in his Laws of Leadership, shares what he calls his Law of Instinct.  He claims the following:

Part I:

Use the formula P@40-to-70, in which P stands for the probability of success and the numbers indicate the percentage of information obtained.

Part II:

Once the information is in the 40 to 70 range, go with your gut.  Don’t take action if you have only enough information to give you less than a 40 percent chance of being right, but don’t wait until you have enough facts to be 100 percent sure, because by then it is almost always too late.

Today, excessive delays in the name of information-gathering breeds “analysis paralysis.”  Procrastination in the name of reducing risk actually increases risk.

Visually, what Colin Powell is describing is the Law of Diminishing Returns where, over time, the value of information diminishes.  Perhaps that relationship might look like this:

(click image for a larger picture)

time.jpg

The circle represents an inflection point at which for each marginal unit of information gathered, the value starts to go down or the cost-effort-benefit tradeoff for the next marginal unit of information is less than the cost to obtain it.

Instinct in Product Development

In product development — any type of product, software, material, or otherwise — there is often a discovery process at the outset.  The trap that most companies fall into is excessive market research, thinking that the more we know, the less risk we’ll face.  As General Colin Powell points out, that type of thinking is flawed.  The truth is that the more delays there are, the risk just increases: knowing more doesn’t translate to less risk or higher probability of success.

Big-Design-Up-Front Design is the poster child for Analysis Paralysis in product development.  In product development, that typically takes the form of “Requirements Gathering Ad-Infinitum” — which is a term I use to indicate incessant requirements gathering with the aim of exhaustively gathering customer requirements, but the activity itself takes a form of its own and — often — it leads to documenting a lot of stuff, but nothing tangible has been produced that brings value to the customer.

Little’s Law is your Friend

A queueing system is a model with the following structure: customers arrive and join a queue to wait for service given by n servers. After receiving service, the customer exits the system. A fundamental result of queueing theory is little’s law.

Theorem: for a queueing system in steady state, the average length of the queue is equivalent to the average arrival rate multiplied by the average waiting time. in other words,

L = λW

Little’s Law is a fundamental principle in business, mathematics, and has applications to many real-world problems. One of those real-world problems is in product development.

First, a definition:

WIP/TIP: Work-in-process of Things-in-process. For the purposes of this article, they are synonymous. Being “in-process” means the work or things have entered a state-of-affairs but have not yet exited. The “work” can be anything: materials, components, sales orders, software code, software testing, projects, customer inquiries, checks, phone calls to return reports suppliers to qualify, repair orders, or emails waiting to be answered, etc.

For product development, we can use a transformation of Little’s Law, like the following:

[(Throughput) = (Things-in-Process) / (Average Completion Rate)]

What this equation tells us and what experience has shown time-after-time, is that the number one driver of Product Development Cycle Time are the “things-in-process”. There is no quicker way to reduce the cycle time by which your company can get a product from concept-to-delivery than through first prioritizing all the projects or products and focusing on the ones that make strategic and tactical sense, and killing the lower priority projects.

You might be thinking: “True, but couldn’t we also increase the average completion rate”? You’re right, but the impact of doing that is much lower than reducing the TIP — that is, influencing the average completion rate is rather difficult and is often a function of available resources, scope creep, market demands and changes, etc. Here’s the bottom line: the number one driver for shipping products quicker is by focusing on the important ones and killing the unimportant ones.

How Batchy Are You?

From a Lean Thinking perspective, Powell is really advocating for a less batchy approach and one that obeys the one-piece flow principle.  Gathering a lot market research and a lot of data is really a batchy approach, whereas the one-piece flow approach from Lean is one for which Colin Powell is arguing.

Specifically, Powell — whether he knew it or not — is really arguing for “What is the right batch size?”  In the case of information, Powell is arguing for ~40% to 70% relative to success as the batch size.  Since one-piece flow is not possible in some cases, then asking the right batch size is the better approach.  An approach that looked at the right batch size and also followed an iterative model — that would be an approach that is more customer-facing and will have a higher likelihood of success (or lower likelihood of failure).

Little’s Law, The Law of Diminishing Marginal Returns & Powell’s Law of Instinct

There are several principles at play: Little’s Law is true — as Work-in-Process grows and the Cycle Time to complete each unit increases, Throughput decreases.  The end result is that products aren’t shipped on-time or at all and the customer loses.

The Law of Diminishing Marginal Returns teaches us that there is a point at which obtaininig the next marginal unit — of any type of unit — might not be worth the cost to obtain it.

Colin Powell’s Law of Instinct teaches us how to reconcile the Little’s Law and the Law of Diminishing Marginal Returns.  He advocates that we rely on hunch, gut, and instinct.  Once we have enough information to be within the probability of success of 40% to 70%, then go with your gut.

Related Posts:

  1. Little’s Law for Product Development This post is part of a series on Queueing Theory....
  2. Multi-Tasking Leads to Lower Productivity There is a predisposition for firms and people to think...
  3. queueing theory: part 1 This post is part of a series on Queueing Theory....

August 9, 2008

Peter Abilla
no nic
shmula
» Not Accountable, Not Responsible

Team size can make a big difference in the success of your service or product. What is counterintuitive for most people is that the larger the team size, the lower the likelihood of success for your service or product.  Why? Entropy can set in and large teams are inherently bad vehicles for communication. More insipid, however, is that the larger the team, there is a higher likelihood of accountability and responsibility being diffused across the team.

When accountability and responsibility is massively diffused, then the mantra holds: if everyone is charge, then nobody is in charge.  In this article, I quantitatively show the inverse relationship between team size and productivity & how team size does impact the effectiveness of communication and accountability & the eventual success of the service or product.

I’ve written about efficient teams before here and here. When I was at Amazon, teams were organized into small, delta teams called “2-pizza teams”: no team should be larger than 2 pizzas can feed. It’s a great approach to team size. In my short career, I’ve learned how true that rule is. Here’s another thing I’ve learned –

  • 2 people are smarter than one
  • 3, 4, 5, 6, 7, 8 people are smarter than 2
  • a team larger than 9 people is just a big dumb gelatinous blob (acronym: BDGB)

Okay, that’s not true at a wholesale level, but it sure feels like it. A small team with highly smart and capable team members can do much more than 10 mediocre team members. The Wisdom of Crowds mentality doesn’t work that well when it comes to efficiency in teams.

A more quantitative explanation is as follows:

One of the root causes of failure in projects is communication — either a lack thereof, miscommunication, or hand-off’s.  Large teams are inherently vehicles for bad communication. This is basic combinatorics — for a given project, suppose there are persons A and B. In this scenario there is only 1 communication link. Add person C, now we have 3 communication links, A-B, B-C, C-A. Add person D, then we have 6; Add person E, then we have 10 communication links. Inductively, as team size grows, the raw combinatoric communication link counts grows geometrically, not linearly. To demonstrate this, we use basic statistics of the form n-choose-r, where !, such as n!, is equivalent to n factorial, to arrive at the formula for how many pairs we can choose from n items:

shmula.com, combinatorics

For the number of pairs, we can reduce the above formula to the following:

shmula.com, combinatorics

Visually, as team size grows, the communication links grows non-linearly, but exponentially:

shmula.com, combinatorics

A Rejoinder

Do not let the above dissuade you from large teams; if the product requires a large team, then that is what is needed. Caution, is what I am arguing here. The facts are that the larger the team, the more communication channels there are and the entire process then becomes more error-prone. If the product requires a large team, then expect the above challenge and manage it.

A Conclusion

There is wisdom in Bezos’ notion of the 2-Pizza Team. Small teams — provided you have the right people — work incredibly well. Also, there is wisdom in Toyota’s usage of Obeya or “The Big Room” as a way to mitigate defects caused by large teams. A combining of the two will most likely make for a great team and a successful product.


Articles on Ethnography and Design:

  1. Feature? What Feature?
  2. Simplify The Product
  3. Ask Aza Raskin
  4. Aza Raskin on Poka-Yoke & The Humane Interface
  5. Aza Raskin on Quasimodal Design and The ATM
  6. Aza on Feature-Bloat and Site Clutter
  7. Aza on Google Search Results Page
  8. Aza on Cooperation and Team Size
  9. Design Thinking in Medicine
  10. On Designing a Watering Can for Little Hands
  11. Queueing Theory and Visual Management
  12. An Interview with the Inventor of “Clocky”
  13. Bad Breath but Good Design
  14. What is Ethnography

Articles on Leadership:

  1. Overmanaged and Underled
  2. Colin Powell on Leadership
  3. Team or Staff?
  4. Tipping-Point Leadership
  5. Abraham Lincoln on Leadership
  6. How to transform an Organization: Chime-in Before Buy-in

Articles on Queueing Theory:

Articles on Operations, lean and six sigma:

December 12, 2007

Peter Abilla
no nic
shmula
» Agile, Lean, and the Silver Bullet

Today, Mary Poppendieck responds to Corey Ladas‘ question on the relationship between Agile and Lean and what to make of all the methodologies in software engineering.

Corey Ladas said,
November 28, 2007 @ 4:23 pm

Lean would seem to allow for a broader set of ideas and practices than some Agile adherents would find acceptable. For example, SEI Team Software Process seems closer to Lean both in spirit and in pedigree than much of the Agile body of practice, yet many Agilists regard Watts Humphrey as a villain. 

Has Agile become the new orthodoxy? How does Agile add value to Lean? Will the prejudices of the Agile community limit the progress of Lean ideas within software development? When do we get to drop the Lean/Agile qualifier and just be Lean?

MBP >> 

Hi Corey.  I have seen and admire your work in Lean Software Engineering, and I often refer people to your excellent website.  I recently had the opportunity to work with a superb open source team that had a much deeper understanding of top notch software engineering practices than I find in most agile environments.  I observed that “by-the-book” agile practices are hardly the only way to develop great software.  As you know, when the lean principles of pull and flow are combined with good software engineering practices, the results might not fit the agile mold, but they can be quite dramatic.

Agile and lean are just the latest in a series of good software development approaches that I have seen rise in popularity over the years.  They join the ranks of software engineering, structured development, JAD, rapid prototyping, CMM and others….  Every one of these approaches has been used to great effect in some companies, while others adopted it in search of the elusive silver bullet.  It seems that every seven years or so, our industry has to learn one more time that software development is complex, challenging work that takes skill and discipline to do well.  What changes over time are the tools available to do the job well; what never changes is the simple fact that there is no such thing as a silver bullet.

+++++

If you are interested, go here for a previous interview with Mary and Tom Poppendieck

Please find originally-written articles on Queueing Theory below:

For a few articles on Operations, lean and six sigma, please visit the links below:

December 5, 2007

Peter Abilla
no nic
shmula
» Poppendieck: Should Lean be top-down or bottom-up?

Last week, I invited the readers of shmula to pose questions to Mary and Tom Poppendieck, the authors of Lean Software Development: An Agile Toolkit for Software Development Managers (Paperback), which won the Software Development Productivity Award in 2004 and, the sequel Implementing Lean Software Development: From Concept to Cash (Paperback).  Several questions were submitted and, over the next several weeks, I’ll be posting Mary (MBP) and Tom’s (TDP) responses.

Earlier, Mary and Tom responded to the question of Waste and the Handoff in software development.  Today, they respond to two questions from Ursula Rutherford:

Ursula Rutherford said,
November 16, 2007 @ 3:55 am

The term “Lean” is very little known in the software development and system delivery industries. Do you expect there to be more explicit use of it in the future? 

It seems IS people are treading their own path towards quality management and capability maturity without reference to other industries. Perhaps if Lean were a more well-known term in IS, we could benefit from several decades of experience and from training in lean principles.

MBP >>

I have to agree, there is a lot to be learned from good old “low-tech” industries.  What will drive improvement into software development is competition - in highly competitive industries, there is no alternative but to find the approach that has the least waste, lowest cost, and highest quality.  As software becomes more and more pervasive, those who develop systems wisely will end up at the top of the competitive hill.

TDP >>

Lean looks at the whole value stream from concept to cash or order to payment.  It delivers more value by avoiding sub-optimizing choices and handoffs.

About 1/2 hour later, Ursula poses another question to Mary and Tom Poppendieck:

Ursula Rutherford said,
November 16, 2007 @ 4:22 am

Making a broad generalization, it seems lean initiatives in many industries need top-management support to have a chance of success. In contrast, there are many accounts of agile   programming techniques being adopted at team level with good results.

At what level of organizations do you recommend lean software development be introduced?

Imagine an organization with high-level sponsorship and a generous budget to make software development lean… Should change begin with education or practice; in the development team or the boardroom; slow or fast; a pilot project or ‘for real’; imposed by decree or encouraged by incentives?


MBP >>

Where to start is very dependent on the context and on the location of the biggest constraint.  If the biggest constraint is a huge end-of-cycle test and merge bottleneck, then you can make huge strides by getting stop-the-line testing disciplines in place.  This involves integration of testing with development, but that might be possible at a rather low level.

On the other hand, if the biggest constraint is caused by thrashing because more work is being dumped on the development organization than it can hope to handle, then higher level of management involvement is usually necessary to make a big difference.   If the approval process is insisting that a host of low priority features be developed, you have yet another problem that needs to be addressed from the perspective of a broader organization. 

I generally recommend that you do a rough value stream map in order to find the biggest constraint in your system.  Actually, people probably already know what the biggest constraint is, but are reluctant to confront it, and a value stream map might just help to put difficult issues on the table for discussion.  Once you have found and are ready to confront the biggest constraint in your development process, then you don’t need a value stream map for a while, instead you need a top notch, team-based process improvement process that addresses and gradually removes the key constraint.  Since the constraint generally occurs at organizational boundaries, it usually helps to have senior management involvement at this point.

TDP >>

At its heart, Lean is a management philosophy based on deep respect for people and relentless elimination of waste from the delivery of value to customers to return sustainable prosperity for the organization.  Sustainable deployment of Lean (or Agile) must reach high enough in an organization to control the entire value stream and to control how people are treated.  In some cases, this may only extend to departmental or divisional level management, in other cases, it may need to extend beyond organizational boundaries to an entire supply chain.  How people are measured and rewarded determines how they will behave in the long run.

+++++

If you are interested, go here for a previous interview with Mary and Tom Poppendieck

Please find originally-written articles on Queueing Theory below:

For a few articles on Operations, lean and six sigma, please visit the links below:

December 3, 2007

Peter Abilla
no nic
shmula
» Poppendieck on Waste & The Handoff

Last week, I invited the readers of shmula to pose questions to Mary and Tom Poppendieck, the authors of Lean Software Development: An Agile Toolkit for Software Development Managers (Paperback), which won the Software Development Productivity Award in 2004 and, the sequel Implementing Lean Software Development: From Concept to Cash (Paperback).  Several questions were submitted and, over the next several weeks, I’ll be posting Mary and Tom’s responses.

A quick note on convention:

MBP: This is Mary Poppendiek’s response.
TDP: This is Tom Poppendiek’s response.

Below is the first question-and-answer in the series:

John D. Heintz said,
November 15, 2007 @ 1:43 pm

Many Agile methods suggest a division of responsibility between the Customer and development team. The Customer is responsible for understanding and prioritizing the business needs and the dev team is responsible for estimating and implementing solutions.  This business/technology division of labor seems simple, visible and I can understand the daily responsibilities of each side.  The downside to this is (paraphrasing you from the lean development list) a we/they divide leading to local optimizations.  An alternate structure is the “Chief Engineer” who is responsible for the total business success of a product and leading the engineering as well.

My question: How does the division and delegation of responsibility differ when a Chief Engineer is responsible for the business success?  A Chief Engineer must have broad business and technical experience, but this person can’t be responsible for everything, all the time.  The best guess I can think of is to characterize the two styles into:

  • Vertical Style: business/technical sides with a communication contract.
  • Horizontal Style: A central figure that uses set-based design to constrain the next levels of work.

Interested in your experience and thoughts,

John

MBP >>

I recommend that you check out the book Lean Product and Process Development by Allen Ward (Lean Enterprise Institute, March 2007).  Allen Ward spent years leading studies of Toyota’s product development process.

He claims in the book that the biggest waste in product development is found at hand-offs.  A handoff occurs whenever you separate responsibility (what to do), knowledge (how to do it), action (actually doing it), and feedback (learning from results).  The problem with a we-they relationship between “the business” and “the development team” is that it creates a huge handoff, and embedded in that handoff is significant lost knowledge, and hence serious waste.  The development team should not be separate from the business, it should be a part of the business.  In this book Ward also discusses the role of what he calls the “entrepreneurial systems designer” (ESD), and he gives a good explanation of the role. 

But to address your question directly, in a culture with product champions (as we had at 3M) or Chief Engineers (as at Toyota), the focus of activity is on a specific value stream (product & its support structure), while the focus of knowledge creation is on the horizontal (functional) organization.  This is to say if your company distinguishes itself by being really good at User Interaction Design, or test frameworks for hardware, or highly resilient transaction processing, or whatever, you had better develop and protect an unassailable technical capability in the areas that constitute your competitive advantage.  This technical capability will be applied to products (led by a product champion), but needs to be protected as an organizational core competence (perhaps under the guidance of a functional manager).

TDP >>

When a new engineer joins Toyota, they spend their first six months working in a factory building cars.  They spend the next few months working in a dealership selling cars.  Toyota considers well worthwhile this investment in providing the people who design their products with a deep, first hand understanding of the human consequences of their design decisions.  There are no strictly technical or strictly business decisions.  Each area both enables and constrains the other.  The chief engineer holds the vision of how the customers can be profitably served, and collaborates with the team members to iteratively refine the vision into a collection of features and implementations.  Each team member ensures that from their specific functional perspective the result will deliver sustainable value. Tradeoffs often cross functional boundaries.

+++++

If you are interested, go here for a previous interview with Mary and Tom Poppendieck

Please find originally-written articles on Queueing Theory below:

For a few articles on Operations, lean and six sigma, please visit the links below:

November 12, 2007

Peter Abilla
no nic
shmula
» Ask Mary Poppendieck Anything!

In August 2006, Mary Poppendieck was nice enough to entertain questions from my readers on the topic of Lean for Software.  Some great questions were submitted and Mary answered them. 

Well, she’s willing to do that again, so please submit your questions for Mary and she will answer some of those questions.  I will then post her responses on future posts.  Here’s the process:

  1. Submit your questions on Lean for Software or Agile in the comments below.
  2. I will close comments on November 25.
  3. I will begin posting Mary’s answers after November 25, 2007.

Here is Mary’s Biography:

Mary Poppendieck has been in the Information Technology industry for thirty years.  She has managed solutions in software development, supply chain management, manufacturing operations, and new product development.

A popular writer and speaker, Mary’s classes apply lean principles to Software Development problems and offer a fresh perspective on software development processes.  She is the author of Lean Software Development: An Agile Toolkit for Software Development Managers (Paperback) and Implementing Lean Software Development: From Concept to Cash (Paperback).

Please ask your tough questions — Mary will most likely have something to teach us all.

+++++

Please find originally-written articles on Queueing Theory below:

For a few articles on Operations, lean and six sigma, please visit the links below: