Looking in the Wrong Place: the Streetlight Effect, Survivorship Bias, the Cobra Effect, Broken Windows and the Law of Triviality

The problem is not the answer but where we look

What undermines good judgement is often not a wrong calculation but looking in the wrong place. When our gaze is drawn to where the data is, what stands out, or numbers that are easy to measure, we miss what matters. This article covers five traps common in individuals and organisations.

The streetlight effect: searching where it is easy

A man is looking for his keys under a streetlight at night. A passer-by asks, “Did you lose them here?” “No, over in the park. But the light is better here.” Philosopher Abraham Kaplan (1964) used this story to warn that researchers tend to choose questions that are easy to measure rather than those that matter.

It is common in everyday life too: when sales fall, staring only at the metrics already on the dashboard whatever the cause, or tracking step counts because they are easy to measure while calling it “looking after your health.”

Check question: “Where is the answer likely to be — and is that where I am looking?”

Survivorship bias: look only at survivors and the conclusion flips

During the Second World War, statistician Abraham Wald analysed bullet holes in bombers returning from missions. Some argued for armouring the wings and fuselage, which had the most holes. Wald argued the opposite: armour the places with no holes, such as the engines, because planes hit there did not come back (his analysis is described by Mangel & Samaniego, 1984).

Stories of “the college dropout who became a billionaire” or “how I got rich doing this” fall into the same trap. The people who did the same things and failed do not become stories.

Check question: “Who is invisible in this conclusion — where are the failures that disappeared?”

The cobra effect: reward the number and you get the number

In 1902 the French colonial government in Hanoi paid a bounty for rat tails to reduce the rat population. Soon, living rats without tails were being found in the streets: cutting off the tail and letting the rat breed kept the bounties coming (Vann, 2003). When an incentive produces the opposite of its purpose, it is often called the cobra effect (Siebert, 2001).

Economist Charles Goodhart (1975) made the observation now summarised as “when a measure becomes a target, it ceases to be a good measure,” and social scientist Donald Campbell (1979) warned that the more a quantitative indicator is used for decisions, the more it is subject to corruption and distortion pressures.

Check question: “Does the easiest way to raise this number harm the real goal?”

Broken windows: does disorder breed disorder?

Wilson and Kelling (1982) argued that leaving one broken window unrepaired signals that “nobody cares,” inviting greater disorder and crime. In field experiments by Keizer and colleagues (2008), people in an alley covered with graffiti littered more, supporting the idea that signs of disorder increase rule-breaking.

Whether aggressive enforcement of minor offences based on this theory reduced crime, however, is hotly debated. Harcourt and Ludwig (2006) found it hard to credit New York’s crime decline to the policy, and a meta-analysis of multiple studies (Braga et al., 2015) found that community problem-solving that improved places reduced crime, while aggressive arrest-focused tactics showed no clear effect.

Check question: “Which actually worked — fixing small signals, or punishing people?”

The law of triviality: the easier the item, the longer the debate

C. Northcote Parkinson (1957) satirised a committee that approved the budget for a nuclear reactor in minutes, then argued at length about the roofing material for a staff bicycle shed. The reactor was too complex for anyone to comment on, but everyone could say something about a bike shed. This is the law of triviality, often called “bikeshedding.”

Check question: “Is the item we are discussing longest really the most important one?” Setting importance and time for each agenda item in advance helps.

Checklist at a glance

Trap	Check question
Streetlight effect	Am I looking where the answer is likely to be?
Survivorship bias	Where are the failures that vanished?
Cobra effect	Does the easy way to raise the number harm the goal?
Broken windows	Which approach actually worked?
Law of triviality	Is the longest discussion the most important item?

All five arise when what is easy to see differs from what matters. Asking “where am I looking right now?” before deciding can save a great deal of wasted effort.

---

References

• Kaplan, A. (1964). The Conduct of Inquiry: Methodology for Behavioral Science. Chandler.
• Mangel, M., & Samaniego, F. J. (1984). Abraham Wald’s work on aircraft survivability. Journal of the American Statistical Association, 79(386), 259–267.
• Vann, M. G. (2003). Of rats, rice, and race: The Great Hanoi Rat Massacre, an episode in French colonial history. French Colonial History, 4, 191–203.
• Siebert, H. (2001). Der Kobra-Effekt: Wie man Irrwege der Wirtschaftspolitik vermeidet. Deutsche Verlags-Anstalt.
• Goodhart, C. A. E. (1975). Problems of monetary management: The U.K. experience. Papers in Monetary Economics, Reserve Bank of Australia.
• Campbell, D. T. (1979). Assessing the impact of planned social change. Evaluation and Program Planning, 2(1), 67–90.
• Wilson, J. Q., & Kelling, G. L. (1982). Broken windows: The police and neighborhood safety. The Atlantic Monthly, 249(3), 29–38.
• Keizer, K., Lindenberg, S., & Steg, L. (2008). The spreading of disorder. Science, 322(5908), 1681–1685.
• Harcourt, B. E., & Ludwig, J. (2006). Broken windows: New evidence from New York City and a five-city social experiment. University of Chicago Law Review, 73(1), 271–320.
• Braga, A. A., Welsh, B. C., & Schnell, C. (2015). Can policing disorder reduce crime? A systematic review and meta-analysis. Journal of Research in Crime and Delinquency, 52(4), 567–588.
• Parkinson, C. N. (1957). Parkinson’s Law, and Other Studies in Administration. Houghton Mifflin.