The posting below is a June 2016 note by the co-chairs of the Committee on Preparing the Next Generation of Policy Makers for Science-Based Decisions, an ad hoc committee under the auspices of the National Academies of Sciences, Engineering, and Medicine’s Committee on Science, Technology, and Law (CSTL), that has overseen the development of educational modules http://sites.nationalacademies.org/pga/scipol_ed_modules/index.htm to elucidate the role of science in decision-making for professional school students. The co-chairs are Paul Brest, former dean and professor emeritus (active), Stanford Law School, and Saul Perlmutter, Franklin W. and Karen Weber Dabby professor, University of California, Berkeley and senior scientist, E. O. Lawrence Berkeley National Laboratory. Further information can be found at: www.nationalacademies.org/stl or by contacting Dr. Steven Kendall firstname.lastname@example.org. Copyright 2016 The National Academy of Sciences. All rights reserved. Reprinted with permission.
UP NEXT: “Mentoring UP” Learning to Manage Your Mentoring Relationships
---------- 2,186 words ----------
Science Policy Decision-Making: Educational Modules
A Note from Committee Co-Chairs Paul Brest and Saul Perlmutter
During the course of their careers, most professional school students will be required to make decisions in which scientific and technical knowledge play a critical role. Yet many of these students will graduate without an adequate understanding of fundamental principles of scientific methodology or causal inference, and hence without a basis for understanding and assessing particular scientific claims. This failure to comprehend basic scientific research and reasoning may lead some policy professionals to be overly deferential to, or overly skeptical of, scientific claims, or leave them unable to distinguish claims based on scientific evidence from those claims based on political or other forms of interest. Such ignorance also may lead policy professionals to ignore or actively avoid engagement with scientific knowledge even when it is directly relevant to their work.
Students of law, public policy, medicine, journalism and business would benefit from early exposure to materials that elucidate the role that scientific information and scientific processes play in decision-making across diverse venues. In their professional lives, these individuals will invariably encounter scientific issues in legislation, regulation, standard-setting, litigation, and business decisions in subject matter areas ranging from energy and the environment to health and national security.
In light of the ever-increasing role that science plays in policy decisions, our committee, the Committee on Preparing the Next Generation of Policy Makers for Science-Based Decisions, an ad hoc committee under the auspices of the National Academies of Sciences, Engineering, and Medicine’s Committee on Science, Technology, and Law (CSTL), has overseen the development of educational modules to elucidate the role of science in decision-making for professional school students.
With these modules, we are not seeking to replicate graduate science programs in schools offering public policy training. It is neither feasible nor desirable to try to turn most professional school students into amateur scientists. It is possible, however, to present scientific principles in the context of public policy debates that involve issues that professional school students may encounter so they may become informed consumers of scientific research and information. We seek to empower students by providing access to materials that:
• Expose them to basic scientific principles and approaches so that they understand the frameworks of scientific inference.
• Enable them to understand the methodology of a scientific study and to ask questions that will inform an assessment of its evidentiary underpinnings and the validity of its claims.
• Provide them with materials that elucidate the scientific process, illustrate how scientific communities operate and the qualifications of those who conduct scientific research, and provide insights into
interactions between members of the scientific community and other communities.
• Provide materials that illustrate how science is a community process that both builds on and challenges existing knowledge, and subjects empirical claims to evidence derived
from observation and experiment.
• Allow students to distinguish when there is genuine scientific consensus on an issue and when there are legitimate differences – and how to best make decisions on the spectrum from consensus to serious
disagreement, and identify a path to a more informed understanding of relevant policy issues.
This project is a product of CSTL’s longstanding investment in public policy formation, which includes looking at the uses and misuses of science in shaping public policy at the confluence of the scientific, engineering, medical, and legal arenas. It is a direct follow-on to CSTL’s development, in collaboration with the Federal Judicial Center, of the third edition of the Reference Manual on Scientific Evidence. The Reference Manual assists judges in managing cases involving complex scientific and technical evidence by describing the basic tenets of key fields from which scientific evidence is typically derived.
The current project began in January 2012 when CSTL assembled twenty individuals from foundations, the judiciary, academic law, public policy, and science for a roundtable discussion on the feasibility of a project to increase the:
science literacy and sophistication of law and public policy students by designing modules that could be used by faculty to create integrative, overview courses that address the nature of science, methods
of scientific inquiry, standards of hypothesis testing and proof, and the uses and misuses of the natural and social sciences in both private and public decision-making.
The attendees agreed that such an effort would be valuable and, in the summer of 2012, a follow-up meeting was held to refine the parameters of the project. This website and the materials contained herein are
the end result of these discussions.
Professionals and educators across disciplines widely acknowledge that an understanding of fundamental scientific principles and techniques is crucial. In response to concerns about the science content in the
current premedical and medical education curricula, for instance, the American Association of Medical Colleges (AAMC) and the Howard Hughes Medical Institute (HHMI), recently convened a group of
scientists, physicians, and science educators from around the United States to determine the most important scientific competencies required of medical school graduates as they enter postgraduate
training. The panel developed overarching principles which emphasized that “the acquisition, understanding, and application of scientific knowledge and scientific reasoning based on evidence” is essential to
“the ability to evaluate competing claims in the medical literature.”1
In law, influential jurists, such as Justice Harlan Fiske Stone, have long highlighted the importance of interdisciplinary education. Stone’s ideas about the need to integrate economics, statistics, and social
sciences into the study of law led him to lament the failure of law schools to pursue such interdisciplinary studies.2 As one scholar recently put it, “The typical J.D. curriculum provides no preparation in the
skills needed to determine substantive facts beyond legal doctrine, nor in the scientific content and analytic methods necessary to assess and support factual conclusions.”3 Another scholar has noted that:
Many legal decisions cannot be made apart from their economic, social, historical and political contexts, and are often dependent upon business, scientific, medical, psychological, and technological
information. Nonlegal but legally relevant information is commonly used, among other reasons, to generate inferential support for factual premises, to support policy arguments, and to support or
challenge legal rules.4
“If law is ever to fully incorporate science into its decision making,” writes law professor David L. Faigman, “law schools will need to provide the basic training.” “Law schools,” he continues, “are in the
business of teaching critical thinking.” “At the beginning of the twenty-first century, it is no longer possible to be a critical thinking generalist if you cannot think critically about science.”5
Writing in the Harvard Business Review, Johan Roos, Dean and Managing Director of Jönköping International Business School, calls for a business school curriculum that is more STEM-driven. “We need,” he
writes, “to increase business students’ knowledge about science, technology, engineering, and math.” “Given that so much innovation happens in STEM-oriented industries, business education needs to help
students bridge the natural and social sciences.”6
The current project represents a continuation of the Academies’ commitment to the effective incorporation of the products of scientific knowledge in the development of public policy and a response to repeated\
calls for increased scientific literacy among non scientists. A 1978 NRC report, Knowledge and Policy: The Uncertain Connection, examined, for instance, how the fruits of the scientific study of social issues
might bear on the decisions of policy makers.7 More recently, in 2012, the Committee on the Use of Social Science Knowledge in Public Policy noted that:
policy education should equip its graduates to promote the use of science in policy-making settings…Policy students can be taught to appreciate policy making through policy argument or practical
reasoning and to understand that the relevance of and weight given to science depends on the policy context. They can recognize the limits of the persuasive power of scientific reasoning, the
substantial institutional barriers and cultural resistance to new scientific knowledge, and the role of moral and ethical beliefs.8
Our committee reaffirms the importance of the distinctive contribution of scientific knowledge to public policy debates and seeks to aid decision makers in government, business, medicine, and law in
understanding the strengths and weaknesses of scientific information that seeks to inform policy debates. Scientific research offers a unique perspective on policy issues, derived from systemic methods intended
to guard against self-deception and bias confirming a desired outcome. While other forms of knowledge also provide useful information, science provides a quantifiable method for assessing both the nature and
the magnitude of problems and the consequences (both intended and unintended) of different forms of interventions. The committee hopes that the materials on this website will be used to strengthen the
curriculum of professional schools and facilitate better understanding and collaboration amongst scientists conducting studies relevant to decision makers who must consider such findings, select a course of
action, and guide its implementation.
The committee developed a set of guidelines to facilitate the development of the educational modules and supplementary materials. It then asked prominent members of the legal and scientific communities to develop proposals for a range of topics where scientific information and understanding play or played a critical role in decision making. The committee met with module authors to discuss their proposals and, on the basis of these discussions, contracts were awarded.
In the end, nine proposals were developed into modules. The modules are:
• The Interpretation of DNA Evidence: A Case Study in Probabilities by David H. Kaye,The Pennsylvania State University
• Models: Scientific Practice in Context by Elizabeth Fisher, Oxford University; Pasky Pascual, U.S. Environmental Protection Agency; and Wendy Wagner, University of
Texas at Austin
• Scientific Evidence of Factual Causation by Steve C. Gold, Rutgers University, Michael D. Green, Wake Forest University; and Joseph Sanders, University of Houston
• Shale Gas Development by John D. Graham, John A. Rupp, and Adam V. Maltese, Indiana University
• Drug-Induced Birth Defects: Exploring the Intersection of Regulation, Medicine, Science, and Law by Nathan A. Schachtman, Columbia University
• Forensic Pattern Recognition Evidence by Simon A. Cole and Alyse Berthenal, University of California, Irvine and Jaclyn Seelagy, University of California, Los Angeles
• Placing a Bet: A New Therapy for Parkinson’s Disease by Kevin W. Sharer, Harvard University
• Translating Science into Policy: The Role of Decision Science by Paul Brest, Stanford University
• Vaccines by Arturo Casadevall, Johns Hopkins University
The committee reviewed module drafts over the course of fifteen months. The committee had ultimate responsibility for approving the content of modules. For the duration of the project, the committee corresponded with module authors and suggested refinements and enhancements to submissions.
Materials were designed for use by faculty seeking to improve scientific understanding among legal, medical, policy, and business students, either as parts of courses or as a the basis for a course designed by individual faculty members. They have been structured to explicate core competencies9 in science and technology through illustrative case studies. While the modules are currently posted on this Academies’ website, the modules will ultimately be moved to a permanent external website that will, it is hoped, serve as a locus wherein additional educational materials may be discussed, developed, and housed.
By making exemplary educational resources readily accessible, faculty from various disciplines might easily design coursework to meet any or all of the goals stated above. The materials currently on the website have been designed to be sufficiently modular so as to allow faculty to incorporate specific components into a larger course – e.g., an evidence professor might make use of the “DNA” module within the basic evidence course, or an environmental policy professor might incorporate the “fracking” module into his or her seminar. Alternatively, multiple modules might be combined to form the basis for an integrated course on science, law, and public policy. The existence of a variety of materials in distinct topic areas provides faculty with a unique resource from which to draw materials to increase professional school students’ sophistication when confronting issues involving science.
The typical module is centered on a case or problem that elucidates some of these competencies and provides text, ancillary materials, and a guide for instructors. The modules may be used subject to the terms of the Creative Commons Attribution-NonCommercial 4.0 International Public License. As envisioned, the materials will serve as core reference material to guide developers of future modules.
This website housing the modules is not intended as a static collection of material. Rather, it is expected that the website and collection of materials will grow over time, and this is exactly the outcome that the committee ultimately seeks. We encourage professional school faculty and others to revise the educational materials on this website to fit their particular needs and to share this resource widely.
As policy makers confront new policy issues and as scientific and technical understanding continues to evolve, educational materials in professional schools must evolve to meet the needs of a changing scientific and policy landscape.
We wish to express our deep appreciation to the authors of the modules for their responsiveness to the suggestions of the committee and to the committee for their reviews of numerous module drafts and for their input on our project from conception to completion.
Finally, we would like to thank the staff of the committee; Steven Kendall, Charles Stotler, Karolina Konarzewska, and Anne-Marie Mazza.
1American Association of Medical Colleges and the Howard Hughes Medical Institute, “Scientific Foundations for Future Physicians: Report of the AAMC-HHMI Committee,” 2009, p 5.
2See Miriam Galston, Activism and Restraint: The Evolution of Harlan Fiske Stone’s Judicial Philosophy, 70 TUL. L. REV. 137, 172-73 (1995).
3Carl N. Edwards, In Search of Legal Scholarship: Strategies for the Integration of Science into the Practice of Law, 8 S. CAL. INTERDISC. L.J. 1, 32, 1998.
4Steven M. Barkan, Response to Schanck, On the Need for Critical Law Librarianship, or Are We All Legal Realists Now? 82 LAW LIBR. J. 23, 34-35 (1990).
5David L. Faigman, The Law’s Scientific Revolution: Reflections and Ruminations on the Law’s Use of Experts in Year Seven of the Revolution, 57 WASH. & LEE L. REV. 661, 683-4 (2000).
6Johan Roos, “The Renaissance We Need in Business Education,” Harvard Business Review, July 2, 2014.
7National Research Council, Knowledge and Policy: The Uncertain Connection (Washington, DC: Printing and Publishing Office, National Academy of Sciences, 1978).
8National Research Council, Using Science as Evidence in Public Policy (Washington, DC: The National Academies Press, 2012).
9As it began to frame the project, the committee developed a list scientific issues/topics that the modules might address. View the list of scientific issues/topics