Lean is more of a set of general operational management principles. Although it was originally developed for manufacturing it can be applied to other contexts, including services, or software development. As mentioned in my earlier posts on this subject, in IT Lean is probably best suited to live service support, although under certain circumstance it probably could be used for solutions delivery.

Agile on the other hand is a set of techniques that comes specifically from software development. Lean is more of a manager’s toolbox; agile is more of a maker’s toolbox. There are people out there who are trying to take Agile concept further, but I am personally too convinced about this as circumstances under which Agile works is much more specific.

Agile is often getting bad reputation. Some of that is caused by too much preaching that can be very offputting to the audience (including myself). Other times, Agile is often an excuse for shoddy delivery. Despite these negatives, my view is that using Agile can be very beneficial as can give you:

• Quick delivery of software features … due to trading off a size of the delivery unit for elapsed time to deliver it

• Solutions that are more fit for purpose… thanks to an increased quality of requirements achieved via disciplined involvement of the user

• More flexible incorporation of changes in requirements … due to generally short timescales

• Better quality of the constructed product … due to continuous integration

However, to get these benefits, one has to leave behind wide-sweeping generalisations Agile evangelists annoy the rest of the world with so often and define the right circumstances under which the approach can work. I would argue that as far as the right context, there are roughly three points that need to be thought of:

1. It needs to be used for the right type of problems

2. Delivery team needs to be very clear about the scope of the agile processes and where they integrate with non-agile ones

3. Customer and the delivery team need to find working arrangements to help deal with uncertainty in regards to budgeting

I will look at these three problems separately in the next parts of this series.

I like Lean’s coherence and its focus on methods of achieving business change,  but I have some doubts about applicability of its mainstream form to IT. This post describes why and sketches out some speculations about alternative models to use for IT.

What is waste
To help you understand why I think the mainstream Lean may not work that well for IT, I would like to revisit Lean’s concept of waste. Because systematic and continuous elimination of waste is how Lean achieves its results, the decision on what counts as waste is the key to results you get. Yet, considering how important the concept of waste is,  its definition as applied to IT seems to be rather dubious.  There are the seven types of waste, but those apply to manufacturing and its translation to IT services seem rather questionable. You even can’t compare categories of waste between different authors. Go and have a look at Poppendiecks and John Bicheno’s definitions (quoted in Lean Services). You will realise they are so different you even can’t even put them in the same table for comparison purposes!

The difficulty is that that the definition of waste is context specific, and therefore varies across industries, companies and possibly across functions. For a start, it is quite obvious that types of waste will be different between manufacturing and service industries. Inventory is, for instance, one of the biggest types of waste in manufacturing; unfinished code and documents may be a nuisance, but their cost is nowhere near a courtyard full of half finished products. But the types of waste will also differ even between different types of service organisations. What can be thought of as waste in transaction-oriented businesses (think haircuts, Matalan suits, Wagamama meals, … or IT live service support) will be quite different from tailor-made business  (e.g. interior design, bespoke shirts service,… or many IT projects).

To illustrate my point, let me define a number of examples of waste in IT, all of which will be real and yet none of them will fit the categories of Lean Software Development: Loss of credibility due to incapability to deliver projects predictably,  delivery of projects that are not needed, use of inadequately skilled people, people on standby waiting for their turn on the project, solutions that do exactly what business users ‘want’, yet fail to meet the real business need, solutions that do exactly what is the business need, but are not supportable, etc. etc.

It may seem a list like this is arbitrary… that’s it… until we revisit the first Lean principle.

Customer value

My main point of contention with mainstream lean is its preoccupation with process, and its effficiency, which are to me only one of the many possible customer values. If you want to disagree with efficiency as a central point, there is a need putting up some framework around customer value to help deal with its seeming randomness.

There are undoubtedly several ways how to do this; the framework that can be useful in our case is Treacy & Wiersema’s value disciplines. The idea is simple: every organisation has three choices as to where to create long-lasting value: customer intimacy, product leadership or operational excellence. According to Treacy and Wiersema, every organisation should aim to be excellent in one of these disciplines and aim at doing OK at others.

I would argue that what is considered waste will differ depending on which value discipline you will see as core:
- Operational excellence - losses in productivity
- Customer intimacy - any issues affecting customer relationship
- Product leadership - failure to use creativity, loss of innovative ideas

This makes clear (at least to me) that mainstream Lean, including its techniques, is most suited to operational excellence problems. I have not thought about this too much but I suspect that other value disciplines will need rather different methods to address their ‘waste’ (probably more in a category of  relationship building, consulting skills, communities, individual skill, though there is undoubtedly going to be some process too). Having said that, there is still a space to use Lean to get to an acceptable level of efficiency, at which it is invaluable.

Please add your comments if you disagree or have additional insights.

Aral, Brynjolfsson & Van Alstyne found out that “asynchronous information seeking such as email and database use promotes multitasking while synchronous information seeking over the phone shows a negative correlation.” Add to that the conclusions of the paper by Nick Bloom from LSE that shows that “technologies that reduce information costs enable agents to acquire more knowledge and ‘empower’ lower level agents. Conversely, technologies reducing communication costs substitute agent’s knowledge for directions from their managers, and lead to centralization”.

Now substitute ‘communication technology’ with ‘twitter’ and ‘facebook’ and watch the results…

Rather than the empowerment of workers, you may get the exact opposite… Maybe it is the time for social networking advocates to make social computing improve the quality of information, or they risk ending up with the results that are exact opposite of what was intended.

• Idea management (collecting ideas on process innovation, product innovation)
• Idea crowdsourcing (Dell)
• Customer forums
• Knowledge sharing / management: Rio Tinto,
• Collaborative planning, forecasting & replenishment: Aerochain
• Customer outreach
• Customer communities - Nike, Avon
• Understanding customers (social networking analytics)
• Viral marketing: Quicksilver

An important point is that these tools need to be used in to enrich/enhance/change existing business processes rather than as a standalone technical solution. As Sameer Patel puts it, “… data, and intelligence normally buried in closed process centric activity and systems were pushed into people centric social realms for improvement, only then to be put back into process systems in their newer highly optimized forms.” A brilliant example of how this could work is German Fidor Bank.

Sources: Beyond enthusiasm, CIMA Report on e2.0, Sameer Patel: Why Process Barfs on Social, Rio Tinto videos on Communities of Practice and Denis Howlett’s Glimpse to the Future.

I spent two years working for a extremely cost-sensitive customer, followed by a year in business development, so I am now rather sensitised to the whole cost/value discussion. Considering I am interested in the practical applications of social computing, I find the ongoing discussion about business value of e2.0 rather interesting and almost personal subject. Today I came across  Aral, Brynjolfsson & Van Alstyne’s paper called Information, Technology and Information Worker Productivity that is interesting by a virtue of being one of the few fact-based contributions to the discussion.

One of the key conclusion of the paper is that “the structure and size of workers’ communication networks are highly correlated with performance.” This seems to be a proof of the ultimate impact of one’s social network on company performance (in hard numbers sense, rather than in woolly e2.0 will make you more in tune with the universe, which must lead to better business value sense).

The only thing that makes me wary is the fact that the research was done using data from a recruitment agency. I  be still careful about generalising this conclusion beyond the knowledge-heavy services industries / functions. We now seem to know how w2.0 fits into a broader enterprise technology landscape and there seems to be case studies from knowledge heavy industries. The question is, do we have any from process-focused and less knowledge-heavy functions?

Starting Point
Lean is more of an operations philosophy rather than set of techniques and that’s why it requires looking first at its principles before dwelling on techniques. Let’s start with the principles set by Womack & Jones’ in the book that started it all:

• Specify value (or know what your customers want & need);
• Identify the value stream (or understand the sequence of steps leading to delivery of the end result within and without your organisation);
• Flow (make sure there are no stop-gaps in the process);
• Pull (reconfigure the process to be driven by customer demand as much as possible);
• Perfection (which seems to me very similar to continuous improvement).

Process Improvement & Its Costs

A quick search will show that the core of the techniques originating in Lean, that are actually used, focus on process improvement (principles 2,3).  It is hard to argue against process improvement, but the way that it is supposed to be applied to IT as part of Lean poses some interesting challenges.

Identification of the value stream is something popular and achievable even if it is not not exactly easy. Unless your operation is CMMI level 3 or above, your existing process are probably blurry or missing in places. To sort out the holes or blurry patches mean formalising, rationalising and agreeing the end-to-end process:

• This is possible for repetitive activities, such as live service support. ITIL gives a reasonable reference process framework to start with this. It is not the easiest process but certainly can be done;
• Solution delivery processes are harder as they fall largely into the Barely Repeatable Processes category. Firstly, projects are rather varied when seen from the viewpoint of an enterprise Process to design a new website is rather different from a process to decide how to upgrade your desktop, which is different from a process to improve your data quality). Secondly, big projects are not repeated that frequently. Considering large projects of each type happen perhaps once every 5 years, what benefit does a detailed process definition gives;
• Front-end process activities are also hard. For tasks such as identification of potential projects or initial project scoping, lack of detailed process is a benefit rather than drawback as it enables creativity and innovation.

If mapping the value stream is not easy, reconfiguring it for flow is even harder:

• Firstly, making IT requires more than just agile process (that is the boilerplate answer from Lean proponents in IT). There is a number of other things that you don’t need to care about as a developer, but that are really important from a management perspective, e.g. how capacity management is,  staff cross-training, how much spare staff do you want to pay whilst they aren’t doing anything useful and others;
• Secondly,  some of the biggest issues IT faces are upstream - facing off to the customer - rather than downstream. Making the upstream process flow hints on business-IT alignment, which is a subject that has been one of the consultants and academics’ favourite for almost 20 years, but still remains a problem.
• Thirdly, I would claim that as long as the IT organisation provides a good service to its customers (e.g. does not discourage them), it will be facing demand that is much less predictable and homogeneous that that of manufacturers. Under certain conditions, loss of flexibility due to over-optimisation may become a bigger issue than the perceived waste in the process.

Questioning the Benefit

Let’s now move to benefit side of the equation, starting with Womack & Jones explanation of the implications of the Flow principle:

when we start thinking about ways to line up all of the essential steps needed to get the job done into a steady, continuous, flow with no wasted motions, no interuptions, no batches and no queues, it changes everything: how we work together, the kind of tools we devise to help with our work, the organisations we create to facilitate the flow, the kinds of carreers we pursue, the nature of business firms (including non profit service providers) and their linkages to each other and society.

From the quote it is clear that the expected business benefit of Lean comes from eliminating waste. The problem is that waste in IT is much harder to identify and quantify, especially when compared to manufacturing which has a pretty large, single and visible source of waste called inventory. The people who interpreted Lean into IT  domain will tell you that IT process waste consists of unfinished, unneeded functionality, processes, multi-tasking, delays and handoffs etc. The issue with this list is that is rather long, full of small and often hard to quantify inefficiencies, rather than a large, single and visible one, which makes reducing IT process waste harder and its payoff smaller.

Considering the level of disruption when adopting Lean, lack of a single large category of benefit greatly erodes the benefit case and the support you can expect from the CIO or the board for an enterprise scope implementation.

In Summary

The implementation challenges combined with difficulty of developing a robust enterprise-wide benefits case, means Lean is not as groundbreaking as some of its advocate think. It still makes useful operational improvement approach that can deliver a reasonable business benefit. When take and go with it, it is important to consider the type of operation you want to apply it to and start where the cost/benefit ratio is the best. If you have a good relationship with business,  it may be worth addressing  your business architecture and portfolio management as  enablers of a wider application of Lean in IT delivery.

There are few more implications that I won’t go into here as I’m running out of space & time, so you can look forward for one more post on the subject.

Most of what I read about Agile and Lean is coherent and relatively believable, but I still can’t get rid of a feeling that there is something wrong. When I dug deeper, I then realised that the key sources of my cognitive dissonance (apart from over-generalisation and over-zealous marketing) is the fact that

1. Their benefits of these methods are poorly formulated and
2. The benefits of across-the-board implementation probably do not stack up if you judge it by implementation difficulty

But I am getting ahead of myself. To understand whether Lean and Agile are worth the effort, means understanding cost, benefits and limitations.

Case 1: Agile
To understand how is the business benefit a problem in the case of Agile, let me play out a scenario of me going to one of my senior IT execs asking for sponsorship for an internal project to drive Agile into some parts of our business. The situation is imaginary, but is likely to have happened if I tried to do something like this in one of my gigs during last 2-3 years.

Sponsor: “So what is this Agile thing about?”

Me: “It’s a reshuffle in our delivery process. We’d be basically using a kind of a  ‘Just in Time’ and make our delivery process smooth and produce a lot less documentation”.

Sponsor (looking at me with expectations): “and that would give us?”

Me (channelling David Nicolette): “Our customer will like the incremental delivery and demonstrations of working, value-add software, starting with the most valuable features as defined by the customer. From the earliest iterations we will demonstrate working software to their customer. Our customers have never received results that good from traditional method.”

Sponsor: “That sounds good in theory and I can see how architects would like that, but I am not entirely convinced if business would really see this as a major breakthrough. Sure, the responsiveness is an issue, but when push comes to shove, getting the new customer management system and cleaning out the duplicates in asset data is likely going to bea much higher priority. Money speaks you know and these two things will generate lots of revenue and lower costs.”

Me: “Ehm, um”

The problem is that if you prioritise a wholesale internal process change against delivery of new business capabilities (or an improvement of existing ones), it is typically business capability that wins over internal process. Having said that, there are definite situations where quick and adaptive delivery of new business or IT capabilities is critical. The only question is which one are these. And this will be constrained by the fact that some activities will be inherently difficult to make agile because of capacity, contractual, commercial or plain physical limitations. This means that the right questions relative to Agile arenot if it is worth it but,

• What are the niches in which delivery of small and frequent iterations are indeed business critical rather than desirable
• Which IT delivery activities can be realistically turned agile without impacting dependent activities

Case 2: Lean

Lean is a slightly different story. When compared with Agile, adoption of Lean in manufacturing industry shows  business benefits that would make my execs listen rather intently. For instance if you interpreted some of metrics quoted in Cliff Ransom’s A Wall Street View of Lean Transformation, you could surmise that in the context of an in-house IT organisation it could help to increase total output by 12% to 15% without related increase in cost.

That looks appealing, at least until you find out this  issue of Strategy+Business with Booz&Co consultants saying:

Lean manufacturing may not be rocket science, but implementing it is like advanced rocket science. There has been no shortage of initiatives intended to revive established manufacturing locations in North America and Western Europe. But the failure rates for plant turnaround in these areas are striking. Few manufacturing professionals today have any difficulty describing their vision of how excellent factories should operate. They are highly knowledgeable about the leading manufacturing techniques. Knowing the theory is one thing, of course; making it work in practice is quite another. Despite the fact that practically all of the relevant elements have been public knowledge for nearly 20 years, few brownfield plants have successfully made the transition to lean manufacturing.

So in case of Lean, the question worth looking at are:

• Do the factors that generate the kind of benefits Lean manufacturing companies benefit from apply to IT?
• Would implementation of Lean in IT be more or less difficult than in manufacturing?
• Is partial implementation of Lean in IT feasible and what kind of benefits it would give you?

If you find the above remotely interesting, stay tuned as the follow-up posts that I have in progress.

What people proclaim and what they really mean

At the same time … originality remains highly acceptable in the abstract. Here again the conventional wisdom makes vigorous advocacy of originality a substitute for originality itself.

Why people don’t get tired of conventional wisdom

There are many reasons why people like to hear articulated that which they approve. It serves their ego: the individual has the satisfaction of knowing that other and more famous people share his conclusions. To hear what he believes is also a source of reassurance… Further to hear what one approves serves the evangelizing instinct.

In some measure, the articulation of the conventional wisdom is a religious rite. It is an act of affirmation like reading aloud from the Scriptures or going to church.

Becoming a pointy hairy boss

The high public official is expected, and indeed is to some extent required to expound the conventional wisdom…Expounding of the conventional wisdom is the prerogative of business success.

How conventional wisdom dies

The enemy of the conventional wisdom is not ideas but the march of events. The fatal blow to the conventional wisdom comes when the conventional ideas fail signally to deal with some contingency to which obsolescence has made them palpably inapplicable… At this stage, the irrelevance will often be dramatized by some individual. To him will accrue the credit for overthrowing the conventional wisdom and for installing the new ideas. In fact, he will have only crystalized in words what the events have made clear.

Why is conventional wisdom to the society

Few men are ever unuseful and the man of conventional wisdom is not. Every society must be protected from a too facile flow of ideas. In the communist countries, stability of ideas and social purpose is achieved by formal adherence to officially proclaimed doctrine. In our society, a similar stability is enforced far more informally by the conventional wisdom.

Benefits of peddling conventional wisdom

Nor is it supposed that the man of conventional wisdom is an object of pity. Apart from his socially useful role, he has come to good terms with life. He can think of himself with justice as socialy elect, for society in fact accords him the applause which his ideas are so arranged to evoke. Secure in this applause, he is well armed against the annoyance of dissent. His bargain is to exchange a strong and even lofty position in the present for a weak one in the future.

Images courtesy of Wikipedia & Ol.v!er

If you measure the quality of a book by how much it changes the way you think or do things, I was lucky to find some truly excellent books. Deciding which one were the best ones was not easy as there were quite a few that had a bit of the ‘my life will not be the same again’ quality, but here it is.

Stephen Clarke: A Year in the Merde

This is a hilarious, and politically incorrect view on French (and British) cultural stereotypes. If you live(d) in France, work with French or even like the country, read the book for your amusement and education. Everyone I spoke about it (regardless their nationality) agreed that it is both extremely funny and at the same time a factual description of many cultural quirks of a Parisian life.

Nassim Taleb: The Black Swan
It would have been enough to read the Black Swan if only for the fact that it foretold the last year’s financial crisis. Yet, Nassim Taleb went a step further and this book gives  a treatment of the subject of predictability (or a lack of thereof) in everyday life and puts up a straw man of a philosophy enabling you to act in a positive manner in a world where ‘noone knows nothin’. Whereas the Black Swan is slightly more applicable to business and investment domain, it is worth checking out his previous book, Fooled by Randomness, which deals more with personal implications of the uncertainty and over-abundance of data.

Michael Pollan: The Omnivore’s Dilemma

What Taleb does for finance and unpredictability, Michael Pollan does for food. Whether you are vegetarian or eat burgers, you will find this book fascinating. The book will make you pause, more than once, to reflect about what you eat and how.

Samuel Hayakawa: Language in Thought And Action

Despite a book on language may seem boring, its subject is something extremely important. Words give us a filter through which we see the world around us. They colour perception with emotions. They enable us to distinguish lie from the truth and give us a windows into the inner world of others’. Their skilful use makes some people charismatic and turns others into spin-doctors. The book paints you the basics of what Samuel Hayakawa describes as ’science of how not to be a damn fool’, explaining the basics of what different usage of words do to you. Overall it is a great self-defence manual to all kind of mischief in the media and a guidebook to a better and fairer use of words.

Other noteworthy books I read in 2008:

• Tom Peters: Brand You 50:
• Darren Brown: Tricks of the Mind
• Andrew Austin: The Rainbow Machine
• Keith Johnstone: Impro
• Stevens, Brook, Jackson & Arnold: Systems Engineering: Coping with Complexity
• Womack & Jones: Lean Thinking