The posting below looks at Global Learning VALUE Rubric and how it can \"address the challenges related to defining, mapping, implementing, and assessing global learning outcomes\". It is by Chad Anderson and David Blair and is from the Summer 2013, Vol. 16, No.3 issue of their publication, Diversity and Democracy http://www.aacu.org/diversitydemocracy/vol16no3/anderson_blair.cfm. Copyright © 2013 Association of American Colleges and Universities, 1818 R Street NW, Washington, DC 20009. All rights reserved. Reprinted with permission.
UP NEXT: Focusing Your Research at a Cocktail Party
------------------- 2,041 words ---------------------
Developing a Global Learning Rubric: Strengthening Teaching and Improving Learning
For over a decade, through its national, multi-project initiative Shared Futures: Global Learning and Social Responsibility, the Association of American Colleges and Universities (AAC&U) has collaborated with and learned from hundreds of campuses working to make high-impact, interdisciplinary, globally-focused learning pervasive across their institutions. Over time, each of these campuses has grappled with versions of the following questions:
What is global learning, and how can our institution build consensus around it?
What are the outcomes of global learning, and what knowledge, skills, and capacities are needed to ensure that students achieve those outcomes?
Is our institution effectively infusing global learning into the curriculum and cocurriculum?
Are our students successfully acquiring the aforementioned knowledge, skills, and capacities?
The Global Learning VALUE Rubric offers a common language and some foundational stepping stones to help campus practitioners answer these questions and address the challenges related to defining, mapping, implementing, and assessing global learning outcomes.
In 2011, the Shared Futures initiative's latest project, General Education for a Global Century (funded by the Henry Luce Foundation), gathered campus teams from thirty-two diverse colleges and universities at a summer institute to help develop a national agenda for aligning general education curricula with expectations for educating students for socially responsible and globally engaged leadership. Project participants, along with AAC&U staff, concluded that a global learning rubric would be a useful resource for initiating reflection and conversation on campus, garnering faculty and administrator buy-in, and building both common language and institutional capacity for global learning. As a result of that institute, a subcommittee of fourteen faculty, administrators, and assessment scholars-representing a variety of disciplines and institutional types-convened in September 2011 to develop such a tool.
The rubric development committee generally followed a process previously used by AAC&U's Valid Assessment of Learning in Undergraduate Education (VALUE) project (see Morgaine 2010). To begin, each committee member submitted a list of global learning outcomes they felt were essential for graduates to possess. Through months-long deliberation, the committee whittled down the resulting list to six dimensions (AAC&U 2013):
Personal and Social Responsibility
Understanding Global Systems
Applying Knowledge to Contemporary Global Contexts
For each dimension, the committee identified a set of criteria corresponding to levels one to four (see table 1 for an example).
Table 1. Understanding Global Systems
Understanding Global Systems
Uses deep knowledge of the historic and contemporary role and differential effects of human organizations and actions on global systems to develop and advocate for informed, appropriate action to solve complex problems in the human and natural worlds.
Analyzes major elements of global systems, including their historic and contemporary interconnections and the differential effects of human organizations and actions, to pose elementary solutions to complex problems in the human and natural worlds.
Examines the historical and contemporary roles, interconnections, and differential effects of human organizations and actions on global systems within the human and the natural worlds.
Identifies the basic role of some global and local institutions, ideas, and processes in the human and natural worlds.
Like all of AAC&U's VALUE rubrics, the Global Learning Rubric is meant to be read from left to right, starting at level four (capstone) and ending at level one (benchmark). This design encourages faculty to begin by picturing the highest level of student learning within a certain outcome before envisioning lower levels of achievement. Moreover, the Global Learning VALUE Rubric (like all VALUE rubrics) ends at level one rather than zero in order to emphasize positive capacities instead of negative deficiencies. The rubric thus underscores what should be present in students' work rather than what may currently exist.
To this end, level four captures what faculty members representing a variety of academic disciplines, campus roles, institutional types, enrollment sizes, and geographic regions describe as their highest aspirational goals for graduates and alumni, often inspired and embodied by their most exemplary students. While many students may not reach this level, faculty and other campus professionals should be cognizant of the language used to describe it so they can articulate expectations to students and identify cases where student work embodies level-four achievement. In contrast, level one captures what a wide range of faculty members describe as the average capabilities of their first-year students (although student work completed in any academic year could land anywhere along the rubric's levels). While the rubric does not explicitly include a level zero, that level still implicitly exists for student work that does not reach the minimal benchmark.
From June 2012 to March 2013, the rubric underwent four nationwide evaluation cycles, each followed by extensive revisions. The volunteer evaluators-over one hundred people representing sixty-two higher education institutions and associations across the country-tested the rubric's clarity and usability using student work. Several groups of faculty, both on campuses and at AAC&U's General Education and Assessment Conference (working with the assistance of Teagle scholars), helped calibrate the rubric (see sidebar) and establish interrater reliability.
Tips for Calibrating Rubrics
Before reviewing student work, it is advisable to calibrate the rubric through an interactive group session. In this session,
Facilitators should encourage participants to think comprehensively about the curriculum, student work, and learning skills and to look beyond specific disciplines or content.
Once participants have familiarized themselves with the rubric's glossary, structure, language, and performance levels, the group should address any questions about how to interpret general and specific criteria.
After this preliminary discussion, the group should complete a practice round in which participants read and score a sample of student work.
Participants should review and discuss the scores one level and one dimension at a time. Each participant should describe how they scored the work, citing specific evidence from the work sample to support their reasoning.
Finally, the group should determine the average or common score for the sample in question based on scorers' choices. It is important to note that consensus is not necessary; a general understanding about how the group will interpret the rubric is sufficient.
After taking these steps to calibrate the rubric, participants are ready to begin scoring student work.
-Chad Anderson and David Blair
Adapted with permission from Finley (n.d.)
St. Edward's University was one institution that participated in the rubric design and pilot process. Faculty and administrators at St. Edward's had previously completed an inventory of the institution's strengths and gaps in its global learning programming, which prepared them to consider which parts of the rubric applied to their campus needs, goals, and definition of global learning. The team at St. Edward's used the pilot instrument to assess student learning demonstrated through samples of student work from courses specifically designed to encourage global learning. Information gathered from these sessions prompted faculty to revise a reflective essay assignment required of students enrolled in the Workshop for Global Understanding to better assess where third-year students are in terms of institutional global learning goals.
Armed with assessment data from this project, faculty and other practitioners at St. Edward's plan to integrate the rubric into other globally focused courses in the coming year. As a result of their participation, they have developed a deeper appreciation for the evolutionary nature of global learning and a clearer understanding of how students can move at different speeds toward increasingly sophisticated comprehension of global issues, social justice, and their own identities in relation to the world.
Faculty and administrators at Michigan State University (MSU) offer another example of how participating campuses have learned from the rubric development project. At MSU, faculty evaluated the pilot Global Learning VALUE Rubric as part of the Center for Integrative Studies in General Science's spring 2012 Faculty Learning Community (FLC). Many FLC participants had not used or developed rubrics before, and the FLC's rubric evaluation exercise provided participants the tools to craft and use rubrics in their own courses and offered them a deeper understanding of how rubrics can assess and improve pedagogical and curricular effectiveness. This professional development opportunity fostered and strengthened faculty community-building across departments and disciplines (Jardeleza et al. 2013).
Participating MSU faculty reported implementing new pedagogies as a result of the rubric evaluation exercise. For example, one faculty member asked students to analyze two comparable environmental issues, one local and one abroad, and examine potential solutions in their respective contexts. After completing this exercise-which the instructor had designed to align with the pilot rubric's six dimensions-students used the rubric to identify the assignment's learning outcomes and indicate the outcome levels at which they saw themselves. This self-reflective exercise allowed students to actively engage in their learning by offering them a framework for describing and concretizing their global perspective. It also provided them an aspirational goalpost for which to strive in their academic, professional, and personal lives (Jardeleza et al. 2013).
The tool resulting from the pilot evaluation process uses the six dimensions identified by the rubric committee to measure how well curricular and cocurricular programming equips students with the knowledge, skills, and values to address real-world, global challenges. These dimensions make the most sense when understood as programmatic outcomes, and the rubric encourages institutions to construct a curriculum that will lead to these outcomes (or to modified ones that are essential to institutional goals). The Global Learning VALUE Rubric is particularly effective for measuring students' progress over time-for example, for measuring learning demonstrated through students' e-portfolios. While faculty can use the rubric in the classroom, they should be prepared to craft assignments and syllabi that intentionally address each of the rubric outcomes.
AAC&U encourages campuses not only to adapt and revise the VALUE rubrics to best fit their particular contexts and goals, but also to use the rubrics as guides for reflection about current practices and for inventorying their curricular programs' strengths and gaps. For example, the Global Learning VALUE Rubric suggests student action as a desired outcome of global learning. If a faculty member's current pedagogy does not require this outcome, the rubric might prompt the faculty member to consider the steps and resources needed to develop a curriculum that does engage students in meaningful action. Of course, campus context is important: faculty members must think about what action is possible, both within the confines of their courses or programs and at their students' current stages of development. But giving students access to high-impact learning opportunities encourages them to apply knowledge, to challenge themselves, and to foster their own capacities to initiate change.
Whether adapting any of the VALUE rubrics or developing their own, campus professionals will find themselves on a long, winding passage through unfamiliar territory. They will have to identify and narrow a list of key learning outcomes; draft and redraft definitions, descriptors, and levels; navigate the conflicting perspectives of colleagues; and compromise without losing the rubric's original intent.
No matter how challenging or slow, the journey of developing a rubric leads to a rewarding and tangible, albeit sometimes imperfect, end: the ability to help strengthen how faculty teach and how students learn. The big questions of our time-climate change, human rights, technological advancement, and economic globalization, to name a few-require the next generation to think critically, creatively, and collaboratively and to take responsible action to address challenges that are, and have always been, global in nature. The Global Learning VALUE Rubric is the result of one group's ambitious vision and best effort to equip faculty and students with a map to embark on their own journeys, wherever they might lead.
To download the Global Learning Rubric and other VALUE rubrics, please visit http://www.aacu.org/value/index.cfm.
The members of the Global Learning VALUE Rubric Development Committee include Chad Anderson (AAC&U), David Blair (St. Edward's University), Karla Davis-Salazar (University of South Florida), Sarah Fatherly (Queens University of Charlotte), Ashley Finley (AAC&U), Wende Garrison (Virginia Tech), Eleanor Hall (AAC&U), Ranae Hanson (Minneapolis Community and Technical College), Bruce Keith (United States Military Academy), Andrew Lloyd (Delaware State University), James Lucas (Michigan State University), Kathryne McConnell (Virginia Tech), Caryn McTighe Musil (AAC&U), Janet L. S. Moore (University of South Florida), Patricia Mosto (Rider University), Christina Sanchez (University of North Carolina at Charlotte), and Nicholas Santilli (Notre Dame College).
The authors would like to acknowledge Ashley Finley for her contributions to this article.
Association of American Colleges and Universities (AAC&U). 2013. \"Global Learning VALUE Rubric.\" http://www.aacu.org/value/index.cfm.
Finley, Ashley. n.d. \"The Anatomy of a VALUE Rubric.\" PowerPoint presentation.
Jardeleza, Sarah, April Cognato, Michael Gottfried, Ryan Kimbirauskas, Julie Libarkin, Rachel Olson, Gabriel Ording, Jennifer Owen, Pamela Rasmussen, Jon Stoltzfus, and Stephen Thomas. 2013. \"The Value of Community Building: One Center's Story of How the AAC&U VALUE Rubrics Provided Common Ground.\" Unpublished manuscript, last modified March 19.
Morgaine, Wende. 2010. \"Developing Rubrics: Lessons Learned.\" In Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics, edited by Terrel L. Rhodes, 11-13. Washington, DC: Association of American Colleges and Universities.
Chad Anderson is a program associate at the Association of American Colleges and Universities, and David Blair is director of institutional assessment at St. Edward's University.