In scientific research, new ideas have become harder to find. Innovation has fallen compared to 50 years ago. Rather than a fear of “too much change,” many researchers worry about stagnation.
One argument suggests that the low-hanging fruit of scientific research has already been picked. Older scientists made the major breakthroughs, and younger scientists now focus on the minor work that comes from major revelations.
Another argument, however, suggests that the problem isn’t a lack of fruit. Instead, scientific research that focuses on old ideas is rewarded, not research on new ones. Jay Bhattacharya of Stanford University and Mikko Packalen of the University of Waterloo in Canada make that argument in a recent working paper published by the National Bureau of Economic Research.
“Over the last five decades, citations have become the dominant way to evaluate scientific contributions and scientists,” Bhattacharya and Packalen wrote. “This emphasis on citations in the measurement of scientific productivity shifted scientist rewards and behavior on the margin toward incremental science and away from exploratory projects that are more likely to fail, but which are the fuel for future breakthroughs…As attention given to new ideas decreased, science stagnated.”
“Even baseball players are evaluated on many more dimensions than scientists,” they wrote. To re-align incentives and encourage scientific breakthroughs, Bhattacharya and Packalen suggest creating a model that rewards scientists based on multiple dimensions, such as measuring novelty and work based on new ideas.
The Martin Center spoke with Mikko Packalen via email to discuss scientific incentives and how to reverse stagnation.
A main theme of your paper is that scientific research faces an incentive problem. Are there other incentive problems in science beyond publishing?
First, I want to say that right now scientists around the world are doing important work attempting to respond to the Sars-Cov-2 pandemic. I very much hope that they are successful in their efforts. Our views on the slowdown of scientific progress in recent decades and on how incentives in science are distorted should in no way be construed as discouraging these ongoing efforts. The unusually collaborative nature of how the scientific community has responded to the pandemic does hint that broader institutional reforms in science are possible and worth exploring.
As for your question, beyond publishing, in the scientific community I worry about the incentives created by new communication tools such as Twitter.
On social media, your value is mainly measured by your follower count. Academics thus today have an increased incentive to say popular things and enter areas of investigation where there are already many others. These platforms also increase the incentive to pursue research topics that are currently the most newsworthy, as news coverage exposes you to more potential followers.
But popularity and timeliness are anathema to most scientific progress. The most successful scholars uncover secrets that are at first very unexpected and unpopular—almost nobody believes them. The scientists are only proven to be right through relentless revision, testing, and validation of these ideas. This work can take decades.
I thus worry that social media tempts especially young scientists to pursue topics that lead to quick popularity at the expense of engaging in exploration that might lead to uncovering more important secrets.
Do some areas of scientific research have a bigger problem than others? For instance, I imagine that some areas with the potential for profit, such as pharmaceuticals or the tech sector, may reward risk and the pursuit of new ideas.
I think it is true that the rate of progress has not been uniform across scientific areas, and I wouldn’t expect it to be. There will always be random fluctuations in progress in almost any activity. Sometimes the efforts are successful, other times not. It is because the slowdown has persisted for decades now that gives the most reason to suspect that scientist incentives have become distorted.
Computer science has indeed had some very visible successes in recent decades, as has biomedicine. But in terms of broader societal benefits, even these advances have thus far fallen well short of the expectations that we had based on comparable past breakthroughs. The relatively strong for-profit motive thus has not inoculated these scientific areas from concerns about stagnation.
In general, the for-profit motive works quite poorly in terms of encouraging risky exploration that may lead to breakthroughs in the very distant future. Companies have a strong incentive to fund work done during the last mile of the marathon. They have a relatively small incentive to fund runners on early parts of the marathon when we know that only a few of them will finish. Thus, to the extent that companies now increasingly set the agenda in academic computer science research, it is not clear this is entirely a good thing in the long run.
In biomedicine the situation is similar. We have some important advances such as CRISPR, but pharmaceutical companies have little incentive to fund the kind of exploration of the unknown that made CRISPR possible decades later.
With academia struggling to reward risk, is there a connection with the average professor getting older? With professors getting older, should we expect the institutions they control to become more risk-averse?
I think so. In our past work, we calculated how career stage influences the willingness to try out new ideas in research. Perhaps unsurprisingly, early-stage scientists were found to be more likely to pursue novel work than their seasoned colleagues. I would certainly expect that the relative hostility to risky work on the part of aging scientists will be increasingly reflected also in how scientific institutions operate.
This is a difficult issue to overcome but the rise of China to the scientific frontier gives me some hope. In some areas, China is now ahead of the U.S. in terms of applying new ideas. The competition with China will force the U.S. scientific institutions to implement some reforms. Thus, while the U.S. already greatly benefits from China’s scientific investments through new knowledge, the competition aspect too can have great benefits.
What about the idea of awarding grant funding by lottery? Or would more grants and prizes created for young academics be another possibility to reduce risk aversion?
Giving grants directly to young scholars would help produce more novel work. While many funding agencies have taken steps in this direction, we have yet to see a large-scale shift in science funding. For a long time, the Nobel-winning economist Paul Romer has raised concerns about the graying of the scientific workforce and how this will impact progress. As a solution, he has advocated giving grants directly to young students and scholars instead of older professors. Students would then have more freedom to choose what to study and with whom. Unfortunately, thus far the impact of this proposal has been limited.
I am skeptical, however, that more generous funding to young people would by itself solve the incentive problem that we identify in the paper. As long as career incentives are dominated by citation counts, also young scientists have too strong an incentive to work on well-established research areas in favor of risky exploration of new areas of investigation. Thus, I prefer that we give more funding to young scientists directly and—at the same time—start measuring and rewarding scientific novelty.
As long as career incentives are dominated by citation counts, also young scientists have too strong an incentive to work on well-established research areas in favor of risky exploration of new areas of investigation.As for giving money by lotteries, I think it is an interesting experiment and worth exploring further. However, I think that the career incentives of scientists—as reflected by their salaries, promotions, and prestige—will still be strongly driven by citations counts, even if all funding were allocated by lotteries. Thus, in my view, for lotteries to generate more true exploration of the unknown, we first need to measure and reward also the scientific novelty of contributions.
Furthermore, I suspect that, ultimately, we will get a better return on limited research dollars if we continue to allocate them based on merit. But, of course, we should broaden how we measure merit.
Are government entities more likely to be risk-averse than private foundations? Are any funders taking action to reward risky research?
From biomedicine there is some evidence that private foundations, specifically the Howard Hughes Medical Institute, have succeeded in supporting riskier research than public funding agencies. But it is not clear that their approach, which is based on long-term funding guarantees, would work on a larger scale. On the government side, DARPA has long been hailed as exemplary in terms of funding innovative work.
The problem with public funding agencies is that they inevitably face pressure from taxpayers to produce visible results fast. In theory, we would expect private foundations to be less responsive to this kind of pressure. But I suspect that their funding sources—particularly the ones with living benefactors—would also rather see successes sooner rather than later. It is thus not clear that private foundations have necessarily better incentives than public foundations in terms of funding risky exploratory work that might lead to breakthroughs decades later.
Furthermore, even public funders such as the NIH have tried for years to find ways to encourage more innovative science. But these efforts inevitably run into two key obstacles. First, the public perception remains that its reviewers are very conservative in their funding decisions. Second, while a funding agency can influence the direction of research somewhat, research choices are still mostly determined by general career incentives.
Accordingly, as long as scientist prestige and salaries largely depend on citations and the ability to publish in high-impact journals, the incentive to pursue risky exploratory research paths will remain small no matter how much funding agencies emphasize innovativeness and risk in their funding announcements.
Looking toward the future, how can professors train students to pursue risky ideas instead of following academic trends? How can they attract students who are risk-takers?
I certainly think that professors play a key role in shaping how their students approach scientific work.
If we express openness to new ideas and appreciate above all else those scientists who persevered in the face of early skepticism about their work and later succeeded, our students are more likely to engage in research paths that may uncover important secrets.
Similarly, if we express our appreciation for exploration that results in work that is not yet popular in any way (and may never be, or may only look like a failure for a long time), our students will be more likely to pursue high-risk high-reward research directions.
If we express our appreciation for exploration that results in work that is not yet popular in any way (and may never be, or may only look like a failure for a long time), our students will be more likely to pursue high-risk high-reward research directions.By contrast, if we obsess over quick-hit papers, top journals, impact factors, and citation counts, our students likewise will develop a fixation with incremental science. Similarly, if we keep telling our students that the reason science has become stagnant is that most of the good ideas have been exhausted already, our students will not dare to pursue the kind of exploration that might lead them to uncover important secrets.
We should tell our students that they should strive to uncover something that is deeply unpopular at first and that finding such important secrets is still possible. Of course, not everyone will succeed in finding those secrets, but that should be the goal.
Beyond our proposal, which involves measuring and rewarding novelty, and which in part aims to encourage more adventurous people to become scientists once again, another potential approach for attracting more risk-takers to science involves providing scientists attractive careers that do not involve tenure. For example, we should experiment with contracts that offer more salary, generous research funding, and longer, 15-year, contracts but no tenure.
How can higher education honor and encourage what you call “fruitful scientific failure?”
One concrete thing we can and should do is to educate scientists, students, and analysts more about the natural history of ideas—how ideas develop from the germ of an idea to transformative ideas. We should always keep in mind the skepticism that many great discoveries faced initially, and even for decades, before—after relentless revision, testing, and validation—the ideas finally won the battle. Similarly, we should keep in mind all the apparent failures that these breakthroughs were built upon, and how these apparent failures were the result of exploring the unknown. If we were more cognizant of how science progresses, we would be less tempted to pursue incremental work over risky exploration.
A better understanding of the natural history of ideas would also make peer review more tolerant of new ideas.
Today peer review is often vilified, but I think peer review can serve a useful purpose. The problem today is not with peer review per se but with the fact that scientists have come to see the value of scientific contributions and scientists only through the lens of citations.
If scientists had a better understanding of what citations measure and how ideas develop—in particular, how crucial risky exploration is for scientific progress—I think reviewers would be much more tolerant of exploration. They would then be much more likely to give a green light to exploratory papers than they currently do. Simply asking peer reviewers to rate papers or research proposals in terms of their innovativeness, as the NIH currently does, does not accomplish anything useful when reviewers do not understand how crucial innovativeness and exploration are to how ideas develop and how science progresses.
How else could we reward edge science outside of publishing metrics?
In addition to refocusing existing journals to publish more exploratory work, or starting new journals that would publish exploratory work, we could have prizes that celebrate exploration. Some prizes could reward exploration done in recent years, other prizes could reward exploration that was done in the more distant past but only recently proved to be useful in any way. Certainly, every year after the Nobel-prizes are announced, we should carefully examine what concurrent and prior exploration were underpinning those discoveries. If we gave prizes also for such visibly fruitful exploration, it would quickly draw more attention to the collaborative nature of science, and the fact that work on different stages of the lifecycle of ideas is valuable.
The main problem with relying only on prizes or new journals to encourage exploration is that they are probably hard to scale to an extent so that they would have a significant enough impact on scientists’ career incentives.
Your paper argues that the problem in science is incentives, not that we’ve picked all the “low-hanging fruit” for scientific progress. What would a counterfactual need to look like to make you believe in the “low-hanging fruit” argument?
If we saw a scientific field try many different approaches over decades both in terms of career incentives and in terms of research directions, and the field still remained stagnant, then I would be more likely to believe that there are no more important secrets left in that particular area. The same goes for overall scientific progress.
Given the stagnation, we should try new approaches both in terms of what career incentives we give to scientists and in terms of diversity of research approaches. Relying on the “science has just become harder” explanation for the stagnation is convenient as it absolves everyone—scientists, administrators, and analysts—alike from any blame for the slowdown. But we shouldn’t give up on trying to create a better future that easily.
Anthony Hennen is managing editor of the James G. Martin Center for Academic Renewal.