Category: design

On Cause-and-Effect Trees

Let’s get back to the Herald ferry accident for a moment. Its investigation was a remarkably curious – and quite unusual at the time – example of an analysis technique that sometimes goes under the name of the cause-and-effect tree.

The cause-and-effect tree is an iterative reasoning exercise intended to explore the cause-and-effect relationships behind a problem. Its primary purpose is to help the investigator resist the temptation to stop at the first plausible explanation and avoid jumping to superficial conclusions by forcing him or her to keep asking the same question (the ‘why’) over and over again, even if the answer has seemingly been found.

Cause-and-effect trees belong to a wider family of causal-analysis tools, ranging from the simple Five Whys game often attributed to Sakichi Toyoda, to highly formalized Fault Tree Analysis. The precise technique matters less here than the habit of thinking they encourage: resisting the temptation to stop, and looking one step further, one chapter earlier, one level deeper. The tree structure also helps us realise that a specific failure may have more than one cause. The captain, for example, remained unaware that the ship was leaving with its bow doors open both because there was no warning light and because the company relied on negative reporting. You begin building your tree by placing the problem at its root, and start growing it upwards by asking ‘why’ at every stop. Every ‘because’ adds another branch at your current stop. Once you have added branches for all the plausible “becauses” you can identify, you move upwards and ask ‘why?’ of each one in turn. The more answers you get for a why, the wider your tree becomes. The more ‘why’s you ask in any one subtree, the taller it grows.

Let’s consider the Herald accident. Leaving aside broader aspects of the vessel’s design, we can end up with the following tree (click to enlarge):

A cause-and-effect tree of the Herald accident investigation

It is very helpful to build this diagram collaboratively, where one subgroup of the team provides their ‘because’ while the other provides counterarguments: “Why didn’t the assistant bosun close the doors? Because he slept through! But he wasn’t the first person to do so. Doesn’t that suggest there may be something else worth investigating?” Team members taking sides can make the exercise considerably more productive as it facilitates the search for less obvious causal explanations. It also helps keep the blame game under control and reminds everyone that humans are, well, humans.

Looking at our simplified tree, we can conclude that many of the deepest actionable causes lead back to management decisions: the company’s failure to create a safe working environment through clear, workable procedures and appropriate safeguards, and its failure to establish a safe and productive work culture.

Note that the tree doesn’t stop on “human trait” or “human weakness” causes. Fatigue, forgetfulness, confirmation bias, complacency and distraction are all part of what makes us human. We cannot eliminate those qualities from people. They are not moral failings in themselves, nor useful places to stop an investigation. They are predictable characteristics of human beings – and therefore risks that a well-designed system should anticipate and mitigate.

Only when we start looking at the problem from that angle can we do what we ultimately want to do – make our systems and our world safer. Humans get tired. Humans make mistakes. Humans underestimate their role in the overall safety of the system. Replace one bosun without changing anything else, and another may make exactly the same mistake – as had already happened on another company ferry. Add redundancy, positive confirmation and independent checks, and you remove important routes by which that mistake can turn into a disaster.

There’s another curious thing about that tree. Nearly every leaf that is not a human trait involves a human decision – but those humans occupy very different positions in the system. This brings us back to the earlier point: the accident was undoubtedly a product of human error. And yet saying that tells us surprisingly little until we ask which human, making what decision, under what conditions, and how far upstream.

And one more thing.

It is vitally important to resist the urge to be dismissive of people when analysing causes. Kindness and curiosity are unexpectedly close relatives. Try your best to be kind. Try to be understanding. Try to postpone blame until you understand what happened. Accountability can come afterwards; explanation has to come first.

Most mistakes are not acts of malice. People make poor decisions, overlook things, and sometimes break rules, but there is often context that helps explain why their actions made sense to them at the time.

You may eventually conclude that the person genuinely behaved negligently. But you arrive there after investigation rather than before it. The greatest value of causal-analysis tools – whether Five Whys, fishbone diagrams or formal fault-tree analysis – is that they cultivate the habit of asking one more question after we think we already know the answer.

Sometimes the next “why?” finds a broken procedure. Sometimes it finds a bad decision. And sometimes it merely finds a tired human being. But it is usually worth asking.

The Law vs. the Beaten Path: And, Not Or

Last time, while talking about the golden principle of good design, we mentioned one little detail that I would like to look at more closely.

I’m talking about this:

More than half of motorists would fail a theory test if they retook it today.

That is actually true: according to official figures, 55% of motorists would fail their theory test if they took it today.

Let’s resist the temptation to judge them immediately and instead focus on what this tells us in the context of design. (And if you are strongly in favour of judging, remember that risky behaviour is an incomparably greater contributor to serious and fatal road accidents than poor knowledge of the Highway Code.)

One thing we can conclude is that it would not be sensible to design our roads based solely on what the Highway Code says. If we know that the majority of drivers are somewhat shaky on the rules, then building a straightforward junction that relies entirely on those rules — for example, by painting the necessary road markings and putting up priority signs — is obviously not enough to make it safe.

Such a junction might comply perfectly with traffic law, but that does not guarantee that it will serve road users well by being efficient and safe.

This simple fact was understood many years ago, which is why today’s roads are much more than strips of tarmac accompanied by traffic signs. Modern road infrastructure has increasingly shifted from simply expecting drivers to follow rules written for them towards nudging them in the right direction through carefully designed road layouts.

Consider this road junction:

From the Highway Code perspective, it is a perfectly constructed junction. Well-equipped with road markings and traffic signs, it looks quite on par with your average rural junction, and certainly beats a lot of junctions in less prosperous neighbourhoods and countries, where road markings are often a luxury.

Thing is, junctions like the one above are responsible for enormous amounts of deadly crashes. The labels tell why.

A way better equivalent is shown below:

It still perfectly complies with the Highway Code, but it also does something else — it treats humans as humans, and not undistractable, observant, flawless law-abiding robots. This junction is a holistic structure, every piece of which – from the safety islands in the middle of the carriageway, to repeated give way signs, to intentional narrowing and curving, force the driver into slowing down and navigating the intersection with care.

Safety-focused road infrastructure is one example of an effective resolution to the Law vs. the Beaten Path dilemma.

Most parts of our gigantic system — driving included — are governed by laws or sets of written rules designed to organise and clarify how that part of the system should work. What is important to understand is that those rules do not always support reality as effectively as they could. The Highway Code creates rules that are supposed to make our road safe, but are they really that efficient if less than a half of road users can use them confidently enough?

Blaming an accident on a driver who failed to navigate a junction safely because of gaps in her knowledge of the Highway Code is easy. Legally, that assessment may be perfectly correct. But in the long run, it does little to make driving easier or our roads safer.

It is the designer’s job to identify situations in which formal rules are not doing enough to support what actually happens on the ground. As we learned in the previous post, the designer’s ultimate goal is to create things that increase the overall value of the environment we live in, rather than merely address the specific problem immediately in front of them.

That includes the willingness and ability to identify rules that are not working well enough and, when necessary, to shape the environment so that users are naturally encouraged to follow those rules — rather than simply punishing them when they do not.

The Golden Principle

The case of the Herald of Free Enterprise can teach us one curious investigative technique that can be particularly helpful in identifying culprits in cases of human error. But let’s first take a short break from human fallibility. Picking apart someone’s bad design decisions is as easy as it is untimely: what’s done is done, and the best we can do once the flaws reveal themselves is learn from mistakes that have already happened.

Let’s instead ask ourselves this: what distinguishes a well-designed system or product from a bad one? Why do some products become known as models of reliability and efficiency, whereas others — often quite similar at first glance — seem almost built to attract problems? What can we do at the planning stage to ensure that whatever comes off our drawing board becomes a safe and efficient product or workflow?

If I had to pick one key principle of good design above all others, it would be this:

Good design fits the system instead of forcing the system to fit it.

Look around. Our world consists of a multitude of processes that operate reliably and sustainably around us every single day:

The Sun rises in the east and sets in the west.

Gravity pulls objects towards the ground.

Weather follows recurring patterns, both short-term (clear weather gives way to overcast conditions and then clears again), long-term (warmer in summer, cooler in winter), and very long-term (El Niño and La Niña).

Plants and animals grow and thrive in favourable conditions and decline in unfavourable ones, both as individual organisms and as species.

Social life is subject to similar patterns:

People wake up in the morning and go to bed in the evening, becoming increasingly tired as the day progresses.

Most people in WEIRD countries are motivated by status and recognition, social affiliation, and comfort and security.

Most Westerners form nuclear households with one or two children and live independently of their own parents.

In certain societies, families extend beyond nuclear boundaries, with multiple generations and more distant relatives heavily involved in one another’s lives.

Most people navigate supermarkets counterclockwise.

More than half of motorists would fail a theory test if they retook it today.

Learning new things requires mental effort. Using familiar things doesn’t.

Most individuals, groups, organisations, towns, and countries develop sets of rules and patterns of behaviour that they follow in day-to-day life. Whether you call them laws, policies, cultural norms, routines, or rules of thumb, they are mechanisms people rely on to function efficiently and contentedly.

All these processes — physical, biological, social, and many others — come together to form one enormous system. Every one of us operates both with and within that system, relying on mechanisms that are already part of it.

Whenever you set out to design an object, a process, or a system, you are effectively inserting a new element into that colossal arrangement of gears, levers, and wheels. The element you introduce will inevitably change the system in some way: it may squeeze between gears, make some levers move more smoothly, or cause some wheels to turn faster. Or perhaps it will cause gears to lose contact with one another or make the levers too heavy. It may even produce excess heat, making neighbouring components too hot to touch.

The effect will vary, but there will always be one. A poorly engineered element, or one inserted in the wrong place, may cause wheels to spin aimlessly, wasting energy. It may introduce friction between gears that previously turned smoothly. Another may make no observable difference at all, simply sitting there, consuming resources and complicating maintenance.

A well-designed and carefully positioned element, on the other hand, can make the whole system work better. It might replace a dozen pairs of gears with one cleverly positioned pair, reducing both time and energy costs. It might adjust a faulty wheel ever so slightly so that it no longer throws sparks against the wheel beside it. Or it might make use of heat already produced by a steam engine, using that heat both to power itself and to dispose of excess thermal energy.

The point is that a well-designed object, system, or workflow increases the overall value of the larger system by integrating smoothly into it rather than demanding that the system make allowances for it. It takes something from the system and gives something back, but the net result of that interaction is positive.

That positive net result is the value your product introduces: it improves the system and, as a consequence, creates a sustainable reward.

Poorly designed objects, by contrast, obstruct the system’s existing processes and take from it without giving enough back. Even if they generate some value in isolation, that value may be offset by making the wider system slower, more expensive, or less reliable. Such objects can produce short-term benefits for some people, but they are unlikely to be sustainable in the long run.

Compare these two automotive designs:

An internal combustion engine produces hot exhaust gas. The manufacturer installs a cooling fan next to the exhaust manifold to cool the gas and prevent damage to nearby plastic components. The fan consumes electrical energy supplied by the alternator, which in turn receives mechanical energy from the engine through a belt.

Now imagine instead that the manufacturer installs a turbocharger that extracts energy from the exhaust to produce additional power, or channels that energy through a thermoelectric generator to produce electricity for the vehicle’s electronics. In the process, the exhaust gas gives up some of its energy and cools naturally.

The first design spends energy to fight an undesirable aspect of the system. The second design adopts and embraces that aspect, getting rid of it through a natural process and acquiring energy as a reward.

Build a habit of asking yourself: will the system benefit from my invention, policy, or protocol? Or will it rather be a burden for the system, even if providing limited local benefits for a narrow set of users?

Inventions that stick with the former are known to lead a long and happy life.

Because they proudly follow the golden principle of good design.

A few more words on human error: an illustrative example

On the evening of 6 March 1987, the Herald of Free Enterprise, one of three Spirit-class ferries operated by Townsend Thoresen, left Zeebrugge fully loaded and bound for Dover. Just over twenty minutes later, shortly after passing the harbour’s outer mole, she capsized, eventually claiming the lives of 193 passengers and crew.

The immediate cause could hardly have been more prosaic—or more merciless. The crew had failed to close the enormous bow doors through which vehicles entered and left the car deck. Once the ship cleared the harbour and began gaining speed, vast quantities of water poured onto the deck. Within minutes, the ferry was lying on her side. Only the shallow water and a fortunate sandbank prevented her from disappearing completely beneath the surface.

There was little dispute about the immediate cause: the bow doors had been left open. Identifying who was responsible, however, turned out to be a much more complicated exercise—one that would ultimately lead to important changes in maritime safety and, more broadly, to the way human error is understood.

The first suspect appeared rather quickly.

Closing the bow doors before departure was the responsibility of the assistant bosun, Mark Stanley. Shortly before sailing, Stanley went down to his cabin for a short break. He fell asleep and was still asleep when the ship dropped her moorings.

So the investigation was clearly dealing with human error, and the human in question seemed obvious: Stanley had overslept an important duty.

Case closed?

Happily, the investigators did not stop there.

They discovered that this was not the first time something remarkably similar had happened. In 1983, another ferry belonging to the same company—the Pride of Free Enterprise—had sailed with its doors open after another assistant bosun had fallen asleep. On that occasion the mistake was noticed in time and disaster was avoided. There had, in fact, been several previous occasions on which company vessels had gone to sea with bow or stern doors open.

At that point, blaming one sleepy junior crew member began to look less satisfactory. Passenger ferries should not be capable of sinking simply because one tired person fails to wake up at the right moment.

There is another detail worth remembering about Stanley. After the capsize, despite being injured, he returned to help rescue passengers trapped inside the vessel until cold and blood loss forced him to stop. The man whose mistake helped trigger the disaster was also capable, minutes later, of considerable courage.

Human beings are inconveniently complicated like that.

So the investigation moved one step further up the chain of command, to Stanley’s superior: Chief Officer Leslie Sabel.

Sabel had responsibility for ensuring that the bow doors were closed. He had been on the vehicle deck, but left shortly before departure without actually seeing them shut.

Surely that qualified as serious negligence and a clear failure of duty?

It did. But again, the answer was not quite complete.

Another set of company instructions could require the same officer to be on the bridge before departure while his loading duties still kept him on the vehicle deck. The inquiry itself recognised that this created a conflict between his responsibilities. At the same time, officers were under considerable pressure to get the ferries away promptly once loading was finished.

Most of the time, of course, everything worked. The officer left the deck, somebody closed the doors, and the ship sailed safely. Nothing terrible happened.

And repetition has a remarkable ability to make an unsafe practice feel perfectly normal.

So responsibility moved another step upwards.

The captain was ultimately responsible for the safe departure of the vessel. Surely it was his duty to recognise the dangerous situation and stop the ship before it left harbour.

Sadly, not this time.

The captain could not see the bow doors from the bridge. Nor was there any indicator or other visual cue telling him whether they were open or closed.

There was something else too.

Townsend Thoresen’s standing orders effectively operated on a system of negative reporting: if nobody reported a deficiency, the Master could assume that the vessel was ready for sea.

And that was exactly what he did.

The inquiry still found the captain negligent. But it also noted that other masters were using essentially the same defective system, and that previous incidents involving open doors had not been communicated adequately to them.

So the investigation moved further again—to the people who had designed and managed that system.

Why should a safety-critical operation rely on the principle that “no news means everything is safe”? Why wasn’t there a simple “DOOR OPEN” light on the bridge? The problem hardly called for cutting-edge technology.

And why had no effective lesson been learned from earlier incidents?

The lessons certainly had opportunities to be learned.

As early as 1985, one of the company’s captains had specifically proposed fitting indicator lights on the bridge so that officers could see whether the bow doors were closed. The suggestion was circulated within management, but dismissed. One response even questioned whether an indicator was really necessary when someone was already being paid to close the doors.

The inquiry later concluded that, had the proposal received proper consideration, the disaster might well have been prevented.

By now, the picture looked very different from the one we started with.

Stanley had made a mistake. Sabel had failed in his responsibilities. The captain had departed without knowing for certain that the vessel was secured. Procedures had allowed conflicting duties and unsafe assumptions to become routine. Management had received warning signs and opportunities for improvement but had failed to act on them.

None of those discoveries made the previous failures disappear. They simply showed that stopping at any one of them would have produced an incomplete explanation.

The official inquiry ultimately reached much the same conclusion. Having examined the actions of the crew, it found itself led inexorably further up the organisation, eventually describing the company as suffering, “from top to bottom”, from a “disease of sloppiness”.

The subsequent criminal proceedings revealed another interesting problem.

Company managers were prosecuted for gross negligence manslaughter, and the operating company itself was charged with corporate manslaughter. The prosecution ultimately failed. At the time, English law struggled to deal with exactly this kind of distributed organisational responsibility: to convict a company, the prosecution effectively needed to identify a sufficiently senior individual whose personal gross negligence could be treated as the negligence of the company itself.

The evidence pointed towards failures spread throughout an organisation, while the law was still searching for one sufficiently important human being to pin them on.

The Herald of Free Enterprise teaches many lessons, but perhaps one of the most important is about the way we think about human fallibility.

Human beings make mistakes. We become tired. We lose concentration. We overlook things, make poor judgements, misunderstand instructions and occasionally behave negligently. These weaknesses are as inseparable from us as the better aspects of human nature—kindness, courage, compassion, self-sacrifice and love.

Accepting our fallibility, studying it and designing the way we live and work around it will take us towards a safer world far more effectively than treating every mistake as an opportunity for blame or humiliation.

That does not mean removing personal responsibility. Stanley was responsible for failing to close the doors. Sabel was responsible for leaving without ensuring that they were closed. The captain carried his own responsibility, and management carried theirs.

Understanding why somebody made a mistake does not make the mistake disappear.

But neither should identifying one mistake bring the investigation to an end.

Thanks to investigators who resisted the temptation to stop when they reached an easy answer—and then resisted it again when they reached the next one—the Herald of Free Enterprise became an important case in the development of maritime safety and in our understanding of organisational failure.

Would any of that have happened if the inquiry had assigned one hundred percent of the blame to the sleeping assistant bosun?

A few words on human error

It is widely assumed that between 70% and 90% of serious accidents across all industries can be attributed to human error. While this is probably about right—you can’t argue with statistics—few people understand what kind of error lies behind that incredible number.

Upon hearing the words ‘human error’, most people think of the person or people directly involved in the accident—the ‘operator’. Was it a road traffic collision? Probably the driver who misjudged the situation. An aeroplane crash? The pilot pulled too hard on the controls. An explosion at a power plant? Must have been some young and clumsy shift operator.

While any of these may, of course, be the case, reality is rarely that straightforward. Many industries—particularly high-risk ones such as aviation, traffic management, construction and nuclear energy—have long been interested in reducing their accident rates. They have worked hard to introduce safety controls that reduce the chance of human error, or at least make its consequences more manageable. Did you know, for example, that the white swirl painted on an aircraft engine is not just a funny bit of styling, but is there to help ground crews see that the engine is running when they may not be able to hear it at a busy airport?

Those efforts have paid off. Commercial aviation, for example, has become one of the safest methods of transport available to human beings, with air travel around 170 times safer per mile than travelling by car, and orders of magnitude safer than walking. In 2023, the industry could boast zero fatalities, despite air traffic reaching a record-breaking 32 million flights.

This increase in safety has had a curious side effect, though. While the proportion of accidents attributed to human error has remained somewhere around that 70–90% mark—of a much smaller number of accidents—the source of human error has propagated well up the chain of command.

A modern human error, particularly in a high-risk industry, is rarely just an operator error. Everything possible has been done to reduce the chances of basic ‘human fallibility’ errors. Most of the improvement in safety records has been achieved not by carefully selecting only responsible, prudent, well-disciplined and positive people for safety-critical positions (there are only so many of us 😉), but by designing systems in such a way that their susceptibility to human fallibility is low—and their tolerance of it is high.

It is highly unlikely that a poorly qualified, exhausted or drunk pilot will make it into the cockpit—and even if one somehow does, there is another qualified pilot sitting right beside them. It is next to impossible for an unqualified person to enter the control room of a nuclear power plant, just as it is impossible to gain that qualification without rigorous training and having your knowledge verified by a diligent examiner. Critical jobs are rarely assigned to a single person without some form of supervision, checking or independent control.

Today, traditional operator failings—fatigue, drink-driving, lack of discipline, negligence, poor training and the like—rarely become the sole root cause of a high-severity accident. Multiple things usually have to go wrong at the same time.

More importantly, alongside increased attention to system resilience, there has also been a qualitative shift in the way we think about human behaviour and workplace failures. Gone are the days of imposing unrealistic expectations on frontline workers and then simply blaming them when they fail to cope. Today, the UK Health and Safety Executive puts the following statement at the heart of its approach:

“Human failure is normal and predictable. It can be identified and managed.”

That is why modern ‘human errors’ are often quite different from what we instinctively imagine them to be. They are no longer solely operator errors. They can be failures to recognise a flaw in the system design that leaves a task beyond the operator’s reasonable capabilities. They can be failures of management to anticipate honest mistakes and mitigate their consequences. They can be failures by executives to keep up with modern working practices and adopt them within their organisations.

None of this means that the operator is now implicitly cleared of all responsibility. Negligence, lack of focus and deliberate violations of rules cannot—and should not—be tolerated. But the operator no longer carries the entire burden of responsibility for an accident and the damage it causes. They remain accountable within the scope of their responsibilities, their abilities, and the balance between the two.

Responsibility for deficiencies in that balance—and for a system’s inability to protect itself against annoying but entirely predictable human slip-ups—belongs to the system and to the people who designed, organised and managed it.

And that is, without doubt, a good thing.