“This Research has Important Policy Implications…”

1 The Push for Demonstrating Research “Policy Relevance” and “Policy Implications”

The COVID 19 pandemic has generated much interest in the relationship between research and policy, and it is clear that there is often no simple linear path from prior medical research to policy responses to COVID 19. The call for “policy to follow the science” sound compelling, but often “the science” itself is disputed, and there is a lack of clarity or agreement on policy objectives and priorities. In many cases, people start with strong prior assumptions or preferences for specific policies and then look selectively for evidence that appear to support positions already taken. Observing the experiences during the pandemic provides an important opportunity to reflect on the relationship between research and policy in conflict research. It is common among conflict researchers to claim that research “has important policy implications”. Such statements are often added tacked onto research articles, possibly as a way to either underscore the importance of research projects or to try respond to calls by funders and home institutions for research to be “policy relevant”. In this article I examine common problems in claims about policy implications following from research. Many research articles often make it seem as if stated policy implications arise directly from the research presented. Yet, claims about policy relevance are often at best incomplete and entail a number of common problems, including confusing outcomes and policies, or stating implications claims that do not follow in any direct way from the research itself. My argument is not that research cannot speak to policy or that researchers should not be interested in policy. However, if researchers wish to speak to policy questions and dilemmas, then their comparative advantage is precisely in research and description rather than prescription, and one would often need to ask very different questions or do different analyses to speak more directly to policy discussions and decisions. The demands that researchers face when seeking to get manuscripts accepted for publication or achieving academic success often do not incentivize the type of research and analysis that could be most helpful to evaluate policy proposals or policy decisions. But the potential role of research for policy is arguably too important to be treated as an afterthought in academic research.

In Gleditsch (2022) I examine the relationship between policy and prediction in international studies, and I propose a simple four-item typology for the key elements that should underlie policy decisions. First, we would need to clarify policy objectives, or what we wish to achieve. Second, we would need to identify policy alternatives, or what we think we can do to achieve these objectives. Third, we would need to examine likely policy consequences, or what we think would happen under different alternative policies. Fourth, we need to do cost-benefit analysis of proposals. In many cases the objectives targeted by one alternative course of action may be in conflict with other objectives, or the likely consequences of one alternative could entail unintended consequences that are detrimental to other key objectives. Gleditsch (2022) primarily seeks to highlight item 3, and underscore how any statements about future consequences are in effect predictions. As such, it is hard to see how we can avoid predictions in policy proposals, and we should look at what we know about making predictions and how to predict better to do as well as we can in informing policy discussion and debate. However, the other points are also important in their own right, and claims about policy implications in conflict research often fail to engage with points 1–2 and 4.

2 The Dark Side of Stock Phrases in Research

“This research has important policy implications” is a very common stock phrase often tacked onto articles, even if not as common as the cliché that “more research is needed”.^[1] A generous interpretation might see these type of stock phrases in research as largely innocuous conventions. Claims about how “more research is needed”, for example, can arguably provide an opportunity to provide an informal discussion of possible new directions in research and things that other researchers might consider. Graduate students are sometimes encouraged to look in the concluding sections of articles for ideas on potential novel contributions when planning their dissertations and research projects. By the same token, one might perhaps argue that laying out an informal discussion of policy in a research article could help draw attention to why someone should care about the research topic in the first place and the potential relevance for policy debates.

However, claims about needs and implications are also directive, and many have pointed to how the statement that “more research is needed” (MRIN) also has a darker side, with potentially negative consequences for research. Greenlaugh argues that indiscriminate MRIN statements often become a way to save a null hypothesis from actual empirical scrutiny in medical research.^[2] It can become a defense of pursuing an existing research program even when the results are largely negative, on the premise that stronger results or confirmation are just around the next corner, with more time or money. A commitment to science should also entail a commitment to abandon theories and propositions if we fail to find support. Standard tenets of philosophy of science tell us that if the results of an experiment do not come out as expected after trying more than once, then we should be prepared to reject the theory, or at least identify what premises or potential auxiliary assumptions may not hold (e.g. Hempel 1966). Resources are invariably limited, and throwing more good money after bad is not just wasteful, but could deprive funding from other more useful projects. Of course, not all research can be experimental, and observational studies present additional layers of complexity (e.g. Morgan and Winship 2014; Rosenbaum 2002). But even so, if we have only week empirical evidence for propositions, if we wish to justify further research then we would at least need to be more precise on what may have been wrong in efforts to evaluate the implications of a proposition, or be prepared to look for incorrect assumptions in theory itself. In short, following Greenlaugh, what we need is not more open-ended research, but “more thinking” and directed research.

3 Pathologies of Claims About Policy Implications

There is an instructive parallel here to the common and often lose claims about policy implications following from research. These often seem to be added essentially as a marketing ploy or afterthought to an article, possibly as a device to motivate the importance of the research agenda or entice more interest by suggesting potential utility of the findings for policy. However, there is rarely much systematic discussion of policy objectives, policy alternatives, cost-benefit analysis, and how research can speak to these in order to inform policy debate and decisions.

The ultimate goal of research should be science, and there is nothing inherently wrong about academic research not having much to say about policy. However, researchers face many incentives to try to claim “policy relevance” for their research, either because reviewers ask for this or because institutions or research funders increasingly emphasize non-academic impacts.^[3] Incentives tend to influence behavior, and researchers are often tempted to make claims about research having policy relevance that are at best incomplete. There are a number of common problems plaguing claims about “policy relevance”. There is a tendency to present empirical findings or variables reflecting particular outcomes as if they were by themselves policies (i.e. we should “reduce conflict”). In some cases, researchers suggest that we should have policies targeting some factor X based on its relationship with some outcome Y (i.e. “reduce conflict by promoting democracy”). But evidence about a relationship between two factors X and Y do not by themselves provide clear evidence of our ability to change outcomes Y through changing X. Discussions often bypass or downplay important debates about objectives and preferences that are essential for a meaningful discussion about policy decisions. Researchers sometimes treat their own preferences and objectives as if they are inherently reasonable and knowledge based, even when they are clearly not universally shared and possibly highly contentious. Boulding (1977: 77) argued that peace research ought to be a “normative science” (which he defined as “the serious study of what we mean by saying that the state of the world goes from bad to better or from bad to worse”, see also Regan 2013), but at the same time noted that this was a “dangerous occupation … [since] [t]here is always a danger that our norms act as a filter which leads to a perversion of our image of reality”.^[4] Moreover, discussions of policy often assume that decision makers or politicians only care about stated outcomes and invariably seek to identify efficient policies for reaching these objectives. In reality, however, politicians can often have perverse incentives. There is often uncertainty often about basic facts and the attributable effects of policies. Krugman (1994) argues that the most influential “policy entrepreneurs” in economic policy peddle politically popular ideas that often lack support in academic research, and that their success is in least part due to the unwillingness or lack of effectiveness of academic economists to engage with their proposals. There is also often a clear bias towards policies that are more visible or help “signal determination” rather than the effectiveness of policies per se. Although many emphasize how “bad policies” may be “good politics” for dictators (e.g. Bueno de Mesquita et al. 2003), leaders in democracies also often face perverse incentives or may act rationally from their point of view yet make decisions that are counterproductive from the point of view of social welfare or stated objectives (see, e.g. Caplan 2022). Finally, interventions or efforts to address one concern can have important conflicts with other or unintended effects.

4 Research and Policy on Migration and Terrorism

As an illustration of the more general common problems in claims about policy implications I look in more detail at an example from an article focusing on the relationship between migration and terrorism by my occasional coauthors Bove and Böhmelt (2016). This is a very solid piece of empirical research, and my issue is not with the analysis itself reported in the article – it is a helpful example precisely because both the empirical findings are clearly presented and the alleged implications for policy are stated explicitly.^[5] The article shows that migration appears to be linked to an increase in the risk of terrorism only in cases where migrants come from locations with active ongoing conflict and political violence, and there is no general impact of immigration on terrorism. The authors argue that this has “critical implications” for “immigration policies” (p. 572) – again, perhaps because this is what we are expected to do, or because a reviewer asked for this. They endorse statements made by then EU commission head Juncker, aspiring to “a well-designed legal migration package”, which should consider both economic benefits and potential risks (p. 586). They also recommend “serious efforts to fight terrorism abroad and reduce the incidence of political violence in immigrants’ countries of origin” (p. 586).

This seems like aspirations that everyone could agree with, so what is the problem here? I see at least two. First, the idea of well-balanced migration policy sounds like a useful objective, but attaining something like this would also require much more explicit detail in identifying costs and benefits and how to trade off one against the other. The economic benefits from migration and the potential security risks from migration have no intrinsic common metric. Leaving aside uncertainty over likely costs and benefits, we would need to price one relative to the other. What would be considered “well-balanced” could differ dramatically if people assign different rates of one to the other. If one assigns a very high price for security relative to economic benefits one might conclude that “well-balanced” would imply be next to no migration, while others would argue that the cost of any security risk pale in comparison to the expected benefits from increased migration, or that these findings at most would support reducing migrants from countries with conflict but allowing more migrants from countries without conflict. I will return to this issue and provide more examples of divergent assessments later.

Second, a common and arguably even more fundamental problem in claims about policy implications is that they in essence amount to statements about objectives we wish to achieve. Less political violence in other countries may also be a laudable objective in its own right (irrespective of any impact on migration or risk of terrorism), but it is precisely an objective or outcome that we seek to achieve, not a policy to achieve the objective. Presenting objectives as policy is also common beyond academia. For example, while Prime Minister of the UK, Elizabeth Truss insisted that she wanted “higher economic growth”.^[6] However, growth is an outcome, and simply stating a wish for higher growth is not by itself a policy to achieve the outcome. The policy proposals offered by her government to boost growth in terms of tax cuts without a clear plan for financing did not produce the intended outcome in the short-term – if anything, they created negative growth expectations – and Truss resigned after 44 days in office, following increased government borrowing costs and currency depreciation.

This underscores how “policy” cannot simply be about stating objectives alone, even when these are largely uncontroversial. Rather, we need to think about choosing specific policies or actions among possible alternatives that we think may be helpful to achieve target objectives. Detailing policy objectives is not trivial, and it is not always obvious or easy to reach agreement on what they ought to be (more on this later). But even reaching agreement on objectives is not enough to proceed to “policy” – we will still need to consider policy alternatives and their consequences.

In many cases, research will uncover or find evidence of associations between specific independent variables and key outcomes, leading researchers to proceed to say that we should have “policies” focused on a key independent variable. For example, if democracy is plausibly associated with less political violence (e.g. Davenport 1999; Rummel 1997), then one might argue that we should try to reduce political violence by “promoting democracy”. The problem here is that establishing that a variable X is associated with differences in Y by itself does not tell us much about our ability to change Y through changing X. Do we have clear ways to promote democracy, for example, and what do we know about the effectiveness of alternative strategies in inducing democracy? For efforts to study such initiatives, see e.g. Bollen, Paxton, and Morishima (2016), Carnegie and Marinov (2017), and Finkel, Pérez-Liñán, and Seligson (2007). Effectiveness aside, could such strategies have unintended consequences that may exacerbate the risk of political violence?^[7] To evaluate such questions we need evidence on interventions and changes.

Bove and Böhmelt (2016) do not discuss the effectiveness of different immigration policies or strategies for reducing political violence in other countries, and this is not the purpose of the article in the first place. To be clear, the statements are not inherently wrong or raising concerns that are not laudable or important, rather the problem is that the research presented does not allow us to say much about how such policies could be designed or pursued, and if specific proposed actions could have the intended consequences.

Research can in principle have a lot to say about the consequences of specific policy alternatives. But we need to look at very different bodies of research, moving into the domain of “effects of causes” or interventions rather than the attributable “causes of effects” producing observed outcomes, which tends to be the focus of academic research (e.g. Dawid and Musio 2022). In short, for evaluating policy alternatives for a specific problem we would typically need an entirely different research program.

Differences in objectives are also usually not a trivial issue, and we are unlikely to have any agreement on cost-benefit analyses without some agreement on the objectives in the first place. Even a cursory review of existing work on migration and policy reveals that there are major disagreements in work on costs and benefits on immigration, in part reflecting differences in initial assumptions or priority assigned to specific concerns or objectives. Some such as Caplan and Weinersmith (2019) and Norberg (2020) argue that the economic case for the benefits of immigration is overwhelming, pointing to plausible studies indicating benefits for economic growth from increasing labor mobility. It is likely correct that more migration will tend to increase individual welfare of migrants and probably also raise global income. However, many skeptics of immigration simply point to other objectives or issues, arguing that immigration undermines social cohesion or can have other negative consequences, perhaps pointing to potential subsequent negative economic effects of reduced social cohesion, weaker social institutions, or negative implications for specific individual actors or coups in receiving countries. More immigration could imply lower GDP per capita even if total GDP grows. For example, Jones (2022) that argues that work on the “deep roots” of economic development should make us concerned about the consequences of immigration. Others such as Collier (2015) stress plausible negative impacts of migration on social cohesion. Others again stress security above all, possibly to the point where no economic benefits could ever compensate for the potential risks (e.g. Bawer 2006; Huntington 2004). Research alone cannot tell you how you to balance these concerns.

More generally, if people have different preferences or objectives in the first place, then we have no reason to expect that people will converge or agree on policies through more research or better information. Indeed, research on cognitive dissonance indicates that contradictory information and challenges to beliefs tend to lead to hardened beliefs among more committed individuals (Festinger 1957; see also Acharya, Blackwell, and Sen 2018; Harmon-Jones, Harmon-Jones, and Levy 2015; Mullainathan and Washington 2009). One of the first noted examples of cognitive dissonance was a study of the coping mechanisms arising among members of a UFO group after the predicted end of the world failed to materialize (Festinger, Riecken, and Seekers 1956). To use an extreme example, people have different views on abortion because they hold fundamentally different values at the outset (e.g. DiMaggio, Evans, and Bryson 1996), not because they disagree on medical research or scientific uncertainty about issues such as when a fetus is viable. It is ultimately a political a political question how societies chose to balance divergent objectives and preferences. This does not mean that there can be no role for research – research could possibly tell you much about existing empirical findings, the bases for the claims in individual studies, divergent conclusion, or even the distribution of popular views and preferences, and if nothing else such information could at least make cost-benefit analysis more explicit and provide for more informed debate. But it is clearly not the case that doing more research will always yield convergence or allow us to conclude on policy without at least first discussing what objectives should be and how to balance potentially competing concerns.

I am by no means an expert on migration, but know a bit more about terrorism and how international factors may influence political violence. In the case of terrorism we have some efforts to conduct more formal cost-benefit analyses of counterterrorism policies. For example, Mueller and Stewart (2011) argue that it is very unlikely that current counterterrorist spending in the US and other countries could be cost effective. This is in part because the direct costs of terrorism are estimated to be low. One might of course argue that the observed risk seems lower because counterterrorism policy might have prevented or deterred costly attacks that otherwise would have occurred. However, the only way to evaluate plausible gains in security is to try to engage explicitly with the possible reduction in risk. Mueller and Stewart calculate how many attacks with an estimated cost of $100 billion would have had to be deterred or averted for US homeland security spending after 9/11 to be cost-effective. They find that there would have to be at least two credible attacks per year averted as a result. If we lower the cost threshold to a more realistic $100 million – the plausible magnitude of the 2010 Times Square attack if it had succeeded – we would need to avert an astonishing 1667 attacks per year. They argue that what we know about terrorist planning and competence makes this difficult to justify. One might contend that Mueller and Stewart have had limited influence on counterterrorism policy, but it is hard to see how policy can benefit from avoiding cost-benefit analysis. Indeed, we could have had better debates if critics engaged with their analyses and tried to point out specially what they disagree with.

Mueller and Stewart (2011) do not provide a direct answer to questions about efforts to reduce terrorism abroad, since they consider only effects of policies at home.^[8] However, much research on terrorism has substantiated potential problems of transference, for example that one might inadvertently raise the risk of other types of terrorism by making it more difficult to carry a specific type of terrorist attacks. For example, efforts to better protect US targets abroad from attacks by Islamic groups after the US embassy bombing in Kenya and the attack on USS Cole of the cost of Yemen plausibly increased the risk of domestic attacks in the US such as 9/11, by lowering the costs of these attacks relative to the costs of targets abroad (Enders and Sandler 2012; Gaibulloev and Sandler 2019). If so, we cannot simply assume all else is equal if we seek to “fight terrorism abroad”.

When it comes to efforts to reduce violence abroad, we have evidence suggesting that international peacekeeping works in the sense that it can prevent civil conflict recurrence (e.g. Walter, Howard, and Fortna 2021). However, good news about peacekeeping and civil war does not necessarily translate to less terrorism by implication. Di Salvatore, Polo, and Ruggeri (2022), for example, argue that UN peacekeeping in civil wars can lead to a shift towards more irregular attacks such as terrorism even if it reduces conventional attacks. In short, it is a valuable idea that we should consider investing in efforts to reduce political violence abroad to offset potential risk of terrorism from migration, but there is clearly much that we do not yet know or research that needs to be done to evaluate properly how actual strategies for this could or are likely to work as well as possible undesired side effects.

5 Prediction can be Helpful for Policy, but Predictive Ability Does Not Imply Policy Relevance

Researchers can in principle have a lot to contribute to debates about policy, but our best bet for doing so is to use our comparative advantage in research and engage systematically in modelling and prediction (e.g. Gleditsch 2022). If policy is about future consequences, then it is hard to see how you can claim to be policy relevant without engaging in prediction and explicit modelling. It is easy to claim that something is predictable after the fact, but unless we actually make predictions in advance what we have is post-diction, with the benefit of hindsight. Prediction is useful in part because we need to be precise about outcomes, timing, and quantifying likelihood, and how we would score if the prediction ultimately was correct or not.

Advances in work on forecasting has taught us a great deal about the constraints on prediction in the social sciences as well as how we can predict better (see Gleditsch 2022; Tetlock 2006; Tetlock and Gardner 2016). First, prediction tends to work well when grounded in clear theory and more explicit propositions. The weather is very complex, for example, and Popper actually cited clouds as less predictable systems than mechanical clocks. Yet, weather forecasting is a clear success story, and advances in computing power has made it possible to apply Lewis Fry Richardson equations for atmospheric flow to data to forecast weather ahead. Second, comparing models is usually more informative than focusing on a single prediction. In the social sciences, we now know much more about what approaches work relatively better for predicting elections and conflict, in part because we have comparisons and debate. In conflict prediction, projects such as the Political Instability Task Force have emphasized comparing alternative predictions on common data sources in ways that have helped us understand what we can do relatively better as well as what we are less likely to do well. Finally, we have a better understanding of the traits and types of reason that allow some “superforecasters” to predict better than others. Tetlock and collaborators argue that forecasting is improved when we break up problems into smaller parts and reason separately about these, think about future events in terms of scenarios instead of single outcomes, and use Bayesian updating to adjust initial predictions as we learn more information (e.g. Tetlock and Gardner 2016). In sum, predicting political events remains difficult, but clearly some approaches are better than others, and more likely to be helpful.

Although systematic efforts at prediction can be helpful for policy, better prediction by itself does not lead to inherently better policy proposals or resolve the ambiguities in other empirical studies. My own prior work looking at evaluation through prediction helps highlight both the promise and limitations of predictive modelling to someone interested in policy. Cederman, Gleditsch, and Wucherpfennig (2017) try to evaluate whether the decline in ethnic civil could plausibly be related to changes in grievances, and how much of observed decline of ethnic conflict could be attributed to changes in factors that might induce grievances such as greater ethnic inclusion and accommodation.^[9] This is not a prediction about the future per se, but we can think of it is a predictive problem where we try to avoid overfitting statistical models to the data and look at ability of models estimated on training data to predict to new data, out-of-sample. More specifically, we trained models on for ethnic groups up to 2003, and then applied the estimated results to evaluate the predicted impact over the next 10 years. We compare the predicted impact in cases where we see changes to the implied predictions in the absence of changes (a counterfactual which avoids some of the problems in drawing inferences based on comparing levels of particular covariates across observations). The results suggest notable predicted reductions in the risk of onset and higher termination rates where we observed changes toward accommodation. Moreover, we show at the aggregate level that a model incorporating grievances and accommodation predicted global trends out-of-sample better than a purely autoregressive model based on observed trends. If our variables related to accommodation only added to overfitting the model we would expect to see worse predictions out-of-sample, yet the model with accommodation performs better and captures better the observed trend.

This helps underscore the difference between studying levels and changes, and how prediction can help us assess the consequences of changes in better ways than analyses focusing exclusively on levels or observed data. For informing policy, it is often more useful to have information on the consequences of changes than simply uncovering associations. On the positive side, our analyses show that where inequalities have been reduced we tend to see less conflict. This is useful and good to know. But we are still looking at ability to predict outcomes given observed changes rather than our ability to influence such changes. Our analysis did not consider actual interventions experiments to introduce inclusion as a policy – is this feasible and does it have different effects than where inclusion emerges organically among local actors? A brief look at other cases suggests that efforts to introduce interventions to reduce exclusion or inequality often fail, even if well-intended. The US invested a large amount of resources to broker power-sharing arrangements in Afghanistan, which ultimately failed (e.g. Coyne 2022). Likewise, the military in Sudan agreed to a transition framework following protest in 2019, but were very reluctant to implement this after the acute crisis abated, and have subsequently tried to reassert control through a new coup in 2021.^[10] Finally, the EU has spent a great deal of resources on democracy aid in neighboring countries, and the European Instrument for Democracy and Human Rights had a €1.3 billion budget over the period 2014–2020.^[11] Yet, despite all this investment, the impact in terms of observed change in neighboring countries seems rather modest and clearly falls short of expectations. In sum, I think it probably is that case that we could affect the risk of conflict through various efforts to reduce grievances, but at the same time a stretch to say that we have fully worked out proposals and impact analysis on policies to achieve this. We could probably learn a great deal more if we invest more time in directed research. This would be valuable, if not necessarily a path to academic success. However, in the absence of this we should be cautious in overstating implications of our current knowledge.

6 How Research can be Helpful for Policy Debate

I stated at the outset that policy implications rarely “follow” directly from research. Claims that research “was helpful” or “relevant” for a chosen policy are often chronologically questionable, as people start out with specific views and look selectively at research to find studies that appear to provide support for actions already chosen. Many funders are often particularly interested in evidence-based research when the evidence happens to fit their existing policy initiatives or approaches. Most conflict researchers are primarily scientists, and we should focus on our comparative advantage in research rather than claim to be experts in policy. And ultimately it is unlikely that offering specific policy prescriptions is what decision makers or policy audiences seek from academics.

Beyond being careful in confusing outcomes with policy overstating policy implications from regression tables without thinking of policy alternatives, are there more productive approaches to engage research and policy? One possible answer is to try to do science as well as possible and focus on our collective contributions rather than to emphasize narrow individual contributions. Pielke (2012) suggests four ideal types of science advice, based on different models of science and models of democracy. One is the standard linear model, where scientist do their research, others then look to their results, and conclude on what this could tell policy. Alternatively, scientists can acts as arbiters – i.e. evaluate policy proposals and comment on science, but as bystanders, without making active advice. Obviously, scientists often have many views of their own on policy, and this is not inherently a bad thing per se. The issue advocate model resembles Becker’s (1983) theory of interest groups – different stakeholders conduct their own science and try to win out in public debate. This is arguably a useful pluralist perspective, recognizing how many claims over policy are not disinterested scientific advice, but very much part of efforts to influence political processes and decisions. Pielke’s final model is scientist serving as honest brokers, who comment on what policy could seek to target, try to lay out different alternatives and map likely consequences, thereby hopefully contributing to better interaction between science and policy.

How might conflict researchers be better honest brokers? First, researchers can play a useful role in laying out or reviewing what we already know about a topic as a starting point. The research frontier is a very noisy place, and new findings are likely to be erratic and more often misleading or wrong (Ioannidis 2005). For an outsider it is usually more helpful to have someone detail a field more broadly and convey key results and findings, rather than to have someone focusing narrowly on their individual contribution, recent articles, and cutting-edge manuscripts most likely to get published in an academic journal. A broader collective focus on what we know first also provides a better basis for conveying what our new or individual research might add to this. Although researchers often tend to disparage more descriptive research, one of the most useful contributions is often better data or more accurate descriptive data and material on problems of interest. Indeed, better description of a problem is often far more useful than unsolicited policy advice, and it is hard to think of cases where policy decisions cannot benefit from better data.

Instead of suggesting that implications “follow” from our research, researchers could try to talk more systematically about possible concerns and clarify objectives that might guide policies and how findings could speak to this, rather than to try to suggest specific policies or initiatives. After highlighting sets of plausible assumptions and likely key objectives, researchers could then discuss what are the alternative factors or features that could be targeted, and what we know about likely consequences of efforts to do this. For example, how direct is the evidence that we have? How much is uncertainty is there about relationships or likely effect sizes? Might there be possible tension between objectives, or could actions to achieve specific objections undermine others? Again, when we do this, we predict. The more explicit we are about stating premises and how we get from A to B, the better the basis for evaluating policy proposals and claims about consequences. In addition, researchers could benefit from greater attention to communication and how to engage with non-academic audiences (see Meyer, De Franco, and Otto 2019). Non-academic audiences are often unfamiliar with academic jargon and prior research, and presenting results and conclusion in a clear and transparent manner intelligible to non-experts is more likely to make your work helpful or useful to others. My own limited experience suggests that non-academic audiences are quite willing to consider relatively complex or technical analyses, but they would like you to be able to convey how you get from A to B in a clear manner.

7 Policy is Too Important to be Treated as an Afterthought

In this article I have tried to show some common problems in claims about policy implications “following from research” and to offer some suggestions on how research may be presented in ways that can be more useful to inform policy, even if you do not have clear suggestions to offer or can claim direct “implications”. There are many reasons for researchers to be interested in policy and contributing to policy debate, and policy is if anything too important to be left to cliches, afterthoughts, and loose claims about implications. It is easy to stay within our comfort zone, follow standard conventions in research articles, and keep making the usual claims that our research “has important policy implications” in a casual manner. But both policy and research can be improved by thinking more systematically about this.

There is growing awareness of the problems with many conventions in research and how cliches like “more research is needed” can have potential problematic consequences. Some journals have apparently banned use of the phrase “more/further research is needed” (see Maldonado and Poole 1999; Phillips 2001). I am in favor of free speech, and blanket bans does not seem a useful approach to guide better scientific practices. Yet, if you find yourself at the point of writing that your research “has important policy implications”, then I hope you may recognize good reasons to pause and at least try to ensure that you are precise, say what the policy objectives might be, and how your research can speak to possible alternatives and their likely consequences. And although this alone should not be grounds for rejecting a submission, its entirely fair for editors to give a yellow card when a manuscript attempts to claim policy relevance and presents outcomes as if they were policies. Many researchers appear to be afraid of not having policy suggestions to offer, or perhaps concerned about discussing limitations in the evidence for claims out of fear that this will attract more scrutiny or criticism. However, it is much better to explicit and upfront than vague, overconfident, or understating limitations. If we do yet have much evidence on the effectiveness of possible strategies, then that is simply the current state of our knowledge. It is often more useful to know what we do not know than to pretend that we know more than we do. Communicating uncertainty and what we do not yet know can help set a new research agenda, and your current research may even be helpful for this. If prediction is hard, then policy must also be hard. But more thinking and predictive analysis are most likely to yield more productive input and be helpful for policy.