Tales of serendipitous discovery crop up again and again within the history of science. From the apple that inspired Newton to formulate his law of universal gravitation, to Fleming’s accidental discovery of the antibacterial effects of penicillin, to the strange ‘bit of scruff’ on Dame Jocelyn Bell Burnell’s chart recorder that led her to describe the first pulsar, the ‘scientist stumbles across major breakthrough’ trope has become a classic. No doubt part of the reason these tales have persisted is that they make for such good stories, but good fortune does seem to play more of a role in scientific discovery than one might predict. It’s difficult to quantify the exact contribution of serendipity to progress within research – studies have made estimates as low as 8.3% and as high as 35% – but appreciating its role is crucial if scientific progress is to continue.
To attribute all scientific discovery to luck would be foolish, ignoring the years of hard work and trial and error necessary to generate an environment that will benefit from serendipity. Louis Pasteur, the French microbiologist responsible for a slew of developments during the 19th century (he most famously invented the process of pasteurisation, but also played a role in the development of rabies and anthrax vaccines, as well as making progress in the study of molecular asymmetry and fermentation) could certainly be considered to have been smiled upon by the gods of scientific fortune. In fact, Pasteur was careful to emphasise the relative insignificance of chance in his success, maintaining that “when you work fourteen hours a day, seven days a week, you get lucky”.
“When you work fourteen hours a day, seven days a week, you get lucky”
I’d argue chance is just a small component of these serendipitous discoveries. Although many ‘happy accident’ discoveries arise through unforeseen experimental error, it is only with the implementation of careful control conditions that such unexpected findings can be identified. Albert Hofmann, while admitting his synthesis of LSD while researching ergot derivatives was a “chance discovery”, nevertheless insisted it was the outcome of “planned experiments” taking place “in the framework of systematic pharmaceutical, chemical research”.
In addition, it is one thing to identify these anomalies, and quite another to recognise their utility. The word serendipity itself was coined in 1754 by Horace Walpole, taking inspiration from the Persian fairy tale The Three Princes of Serendip, in which the heroes “were always making discoveries, by accidents and sagacity, of things they were not in quest of”. It’s that sagacity, arguably, that is the more important ingredient in serendipity in science. It is only possible to identify a solution when equipped with a full understanding of the problem, meaning a thorough knowledge of the field of research is crucial. This requires years of preparation, immersion and experience – it’s no coincidence that scientific genius tends not to peak until middle age. It was only because of Fleming’s experience in the Royal Army Medical Corps, where he witnessed the death of many soldiers from sepsis following wound infection, that he was able to identify the possible value of penicillin for treating bacterial infection. To quote Pasteur again, “chance favours the prepared mind”.
Call me cynical, but maybe we’re reaching a point where we can’t rely on serendipity to drive progress. The middle portion of the 20th century is often hailed as having been the Golden Age of scientific discovery, especially in medical research – from the discovery of penicillin, the development of vaccines against polio, diphtheria and tetanus, the first successful organ transplants, renal dialysis, the contraceptive pill and the discovery of the link between smoking and lung cancer. Perhaps serendipity was a solid foundation for scientific advancement, because everywhere you looked there was a new discovery waiting to be stumbled upon; proverbial Newton’s apples raining down from the sky. These days, it seems the possibilities, or at least the low-hanging fruit, are being rapidly exhausted, with the increments by which the frontiers of science are being pushed forwards becoming smaller and smaller in many fields.
“We’re reaching a point where we can’t rely on serendipity to drive progress”
These days, a far more methodical strategy is often necessary. This is particularly true in the field of antibiotic development, for example, where the looming threat of antimicrobial resistance means we can’t afford to wait to get lucky.
The systematic approach has its limitations, though: the very nature of experimentation means it isn’t possible to predict the outcome of a project at the outset. Baruch S Blumberg was studying differences in blood antigens in different populations in Australia when he discovered the HBsAg antigen; an “apparently esoteric” project which led to the discovery of the hepatitis B virus. Blumberg concluded his 1997 paper with a reminder of the importance of this freedom of direction with scientific investigation: “funding for basic research is based on the faith that it will, in due course, lead to an outcome that benefits society”.
The value of such apparently oblique research is repeatedly underestimated. Katalin Karikó was refused tenure by the University of Pennsylvania and discouraged from pursuing her work on mRNA, being told she was “not of faculty quality”. This work that “nobody believed” in ended up being foundational to the rapid development of vaccines against Covid-19. Although Karikó and her research partner Drew Weissman did eventually receive recognition – they were jointly awarded the 2023 Nobel Prize in Physiology or Medicine – this is just one example of the importance of seemingly tangential research leading to a major success.
“Breakthroughs happen when scientists are permitted the freedom to muck around and get lucky”
Of course we can’t rely on luck, but equally it isn’t always possible to target important findings – often breakthroughs happen when scientists are permitted the freedom to muck around and get lucky. As National Institutes of Health (NIH) Director Harold Varmus wrote in 1997, “research aimed in one direction frequently provides benefits in an unexpected direction”.
With the Trump administration intent on slashing grant funding in favour of targeting scientific progress towards the “industries of the future”, it seems esoteric research will continue to be devalued, with scientific research suffering as a result.
To put all scientific development down to serendipity would be reductive and insulting, dismissing the labour of those behind the research. However, to ignore the importance of chance entirely could be far more dangerous.
Want to share your thoughts on this article? Send us a letter to letters@varsity.co.uk or by using this form.
