Tomorrow's Teaching and Learning
The posting below looks at the use of ePortfolios at different institutions in evaluating so-called VALUE (Valid Assessment of Learning in Undergraduate Education) rubrics, and is by J. Elizabeth Clark, professor of English, LaGuardia Community College and Bret Eynon, assistant dean, Center for Teaching and Learning, LaGuardia Community College. The article is from Peer Review, Fall 2011/Winter 2012, Vol. 13, No. 4/Vol. 14, No. 1. Peer Review is a publication of the Association of American Colleges and Universities [http://www.aacu.org/peerreview/about.cfm] Copyright © 2012, all rights reserved. Reprinted with permission.
UP NEXT: Learning Theory and Online Instruction
Tomorrow's Teaching and Learning
--------------------------------- 2,226 words -----------------------------
Measuring Student Progress with E-Portfolios
In his \"An Essay on Criticism,\" Alexander Pope defines wit thusly: \"True wit is nature to advantage dressed, /What oft was said, but ne'er so well expressed/Something whose truth convinced at sight we find, /That gives us back the image of our mind.\"
In higher education today, \"What oft was said, but ne'er so well expressed\" serves as a compelling metaphor for the progressive uses of institutional rubrics. Are our institutions doing what we say they are? Institutions often tout student progress and achievement based on new initiatives and proven programs and approaches built over time. What happens, however, when we connect these claims to authentic student work? Do they align? Can we talk about student work and achievements across institutions? Why would we want to do so?
Faculty, students, and administrators are engaged in exciting new conversations about definitions of educational progress. While many faculty have long used rubrics to measure student achievement in a course or on a specific assignment, institutions are increasingly using rubrics to assess authentic student learning based on student-produced work connected to larger institutional outcomes. Because of the national, collaborative work in creating the Association of American Colleges and Universities' VALUE rubrics, these rubrics lend themselves to extending the conversation about work in individual courses, in programs and majors, in institutions, and across institutions.
Academic Pathways Through Higher Education
The flexibility of the VALUE rubrics allows them to be used in different segments of higher education and adapted to serve multiple purposes. Students are the ultimate stakeholders and beneficiaries, and so a key national issue in higher education today is helping students to see a clear pathway to graduation and to achieving their personal academic and vocational goals.
Too often, students see their overall education as a disconnected series of courses and experiences. Situating a wide range of skills and areas of knowledge over a four-year scale, institutions can use the VALUE rubrics to help students see how their skills and knowledge build sequentially over time, from course to course, and sometimes, from institution to institution.
Jason C. Evans, chair of the English department at Prairie State College, believes the VALUE rubrics are helpful for engaging students in a common language around academic expectations. \"Since they're widely used in two-and four-year settings, we can feel comfortable knowing we're more or less in line with other colleges. They provide a helpful common language for our students to understand what they're learning and why.\"
Natalie McKnight explains that in Boston University's College of General Studies (CGS) \"The rubric is important because it makes faculty and students more conscious of what we are trying to accomplish; knowing and discussing our goals enables us to reach them more successfully. Plus students, faculty, and administrators can use the rubric as a gauge to assess the impact of our program, something we can all benefit from knowing.\" She adds, \"Articulating goals and then evaluating how well one has achieved those goals are essential to progress and learning in any field. The VALUE rubrics have played a key role in that process for us, as have e-portfolios.\"
For students, the four-year scale of the VALUE rubrics allows them to understand their present learning in connection with future learning at their present or future institutions. Dean Susan Solberg of Prairie State College believes the power of the VALUE rubrics lies in this inter-institutional credibility. \"I especially like the fact that we can say that we are adopting standards that have national credibility because many college and universities have worked on them collaboratively.\" Sarah Burns-Feyl, Joy Tatusko, and Linda Anstendig of Pace University note, \"The VALUE rubrics provide a gold standard for assessing learning outcomes and are grounded in theory and practice. After all, these rubrics were developed by teams of faculty over an eighteen-month period, and have been tested in various settings.\" Knowledge building becomes more than situational at a particular college or university, communicated instead as a cultural value and a benchmark of higher education.
Adapting the Rubrics to Meet Local Needs
A key component of VALUE rubrics is their cross-institutional construction and transferability to local campuses as institutions transform the rubrics to make them relevant for local student learning.
Alison S. Carson explains how Manhattanville College adapted the rubrics to find new ways to talk with students and faculty about evaluating students through their distinctive portfolio program, transforming from paper to e-portfolios: \"We began with the recognition that the rubric had to work for our unique portfolio requirements and learning outcomes, as well as be user-friendly, which meant not too long.\"
Burns-Feyl, Tatsuko, and Anstendig of Pace University also believe that adapting nationally normed rubrics for a specific, local educational mission allows them to think differently about student learning.
At Pace, we have had various initiatives and pilots to assess learning outcomes (especially related to our last Middle States accreditation evaluation), but now as our ePortfolio program evolves, we have the opportunity to pursue assessment in a much more systematic way, and the VALUE rubrics (as well as our adaptations of them) are a crucial part of this process.
Engaging Students in Self-Assessment
Another important emerging trend is using the rubrics with students. Rather than just \"assessing\" students, some colleges and universities are asking students to engage in self-reflection and self-analysis. How do the rubrics allow them to understand their own growth and development over time?
Boston University's CGS introduces students to a modified VALUE rubric in their freshman year. At the end of the year, students write a self-assessment essay or a video analyzing their own development.
At Stony Brook University, Nancy Wozniak engages her student e-portfolio consultants in the use of the VALUE rubrics focused on measuring \"intensity\" in student e-portfolios over time.
Understanding Student Learning
The VALUE rubrics have also provided an important benchmark for emerging work in e-portfolio research in individual courses, in college-wide e-portfolio programs, and across emerging national networks.
At Manhattanville College, Carson explains that the VALUE rubrics provide a way to establish a \"more consistent, objective and transparent way of evaluating Portfolios.\" The college has decided to use \"the integrative and written communication VALUE rubrics to align with the college's learning outcomes for their Portfolio system.\"
Virginia Tech is an early leader in the use of e-portfolio rubrics to measure student learning. Marc Zaldivar explains the university's pilot of three different rubrics to evaluate learning gains in first-year experience and the way the institution then hopes to use the emerging data:
The VALUE rubrics are a constant resource for Virginia Tech. As we work with new programs that are exploring assessment or learning with e-portfolio frameworks, we use these as a starter guide to defining specific learning outcomes (from the LEAP initiative) and how to assess those outcomes with the VALUE rubrics. Our largest project to adopt them is our growing first-year experiences, which is a combined, cross-campus effort for our Quality-Enhancement Plan for our recent SACS accreditation. In this, we are collecting artifacts and reflections from students on three specific outcomes: problem solving, integration of learning, and inquiry. For each of these outcomes, a modified version of the VALUE rubric will be employed to assess student learning in those three domains. The academic year 2010-2011 was our first year of this pilot effort, with six participating programs and roughly one thousand students. Assessment of those materials will begin in the next few months. The next academic year will expand to [assessment of] twelve different programs and approximately two thousand students. Our target is to reach 100 percent of the entering students, or roughly five thousand students, within the next three years. Other programs at Virginia Tech, such as the SERVE Living Community, are also exploring ways that other VALUE rubrics, such as the civic engagement and critical thinking rubrics, may be used to help them assess the effectiveness of their programs.
Like Manhattanville College and Virginia Tech University, LaGuardia Community College has also begun to use the VALUE rubrics to assess student learning. The college modified the information literacy rubric for its local research and information literacy (RIL) competency.
Recently LaGuardia conducted college-wide Benchmark Assessment Readings to assist the college in preparing for its Middle States Review comparing student work from students with twenty-five credits or less and forty-five credits or more. The college believes that the data gleaned from student artifacts will provide an additional understanding of the extent of student learning of general education outcomes. In an examination of student work submitted for the RIL competency, an interdisciplinary group of faculty scorers found that students made a gain of +1.45 on a six-point rubric, demonstrating key gains between an earlier and later point in students' tenures at the college.
For LaGuardia, this gain indicates that the college is effectively helping students to make gains in research and information literacy throughout the curriculum cumulatively over time. As students prepare to enter four-year colleges after graduation, they are better prepared for the rigors of research in the junior year. These findings are also helping the college to shape its conversation about student learning over time and preparing students for transfer to four-year institutions.
What emerged as one of the most significant findings of this evaluation process at LaGuardia, however, was the importance of reading and scoring student work as part of the college's faculty development. Faculty found this process most meaningful because of the way it helped them to understand their relationship to student learning, the college's core competencies, and the work of departments and disciplines. The assessment readings proved to be an enormously rich site of shared faculty exploration into the general education (core) competencies, and how those competencies play out in different courses. As a direct result of this process, several programs have already begun to revise assignments they use in order to ensure that what they ask students to do more effectively corresponds to the college's competencies and rubrics. More importantly, faculty were engaged in reading student artifacts, using the assessment interface, and defining the college's assessment outcomes.
Integrative Learning and E-Portfolios
Building on its work as a national e-portfolio leader, LaGuardia Community College recently launched the Connect to Learning project (C2L), a national three-year FIPSE-funded project coordinated by LaGuardia's Making Connections National Resource Center in partnership with the Association for Authentic, Experiential and Evidence-Based Learning. The C2L project works with a national network of twenty-two campuses-community colleges, private colleges, and research universities-to engage in a collective and recursive knowledge-generation process. The project focuses e-portfolios on reflective pedagogy and student learning, correlating improvement on student success measures, such as retention, with more nuanced assessment of student work using the VALUE rubrics.
As LaGuardia Community College, Manhattanville College, Pace University, Boston University, Stony Brook University, Virginia Tech, Prairie State College, and other campuses in the C2L network expand their campus-based use of various VALUE rubrics, the C2L network as a whole will begin experimenting with VALUE as a tool for cross-institutional exchange. Highlighting the use of e-portfolio to support and surface students' longitudinal development across semesters and disciplines, the C2L network will consider ways to use the VALUE rubrics to examine student work and compare student learning in diverse institutional settings.
This past summer, the teams from twenty-two campuses sampled student e-portfolios in a structured conversation around integrative learning and e-portfolio at the C2L Summer Institute. The teams used the integrative learning VALUE rubric to explore the range of integration in student e-portfolios. Every campus contributed a sample student e-portfolio. The campus teams rated the student e-portfolios in teams and small groups and then discussed the characteristics of integrative learning.
What emerged in a large group discussion is the importance of intentional e-portfolio development on our campuses. Reflection and integration have to be prompted and structured activities for students as they learn to be more reflective learners integrating their knowledge across disciplines. The group also concurred that asking students to integrate across disciplines can be particularly challenging when institutions do not model effective integration and cross-disciplinary practice for students. It is our collective hope that this will provide us with new ways to discuss e-portfolios as a pedagogical resource for higher education and point the way to future study.
Building a Relational Culture in Higher Education
Institutions of higher education have always been proficient at talking about what their campuses do well. The VALUE rubrics represent a radical shift, encouraging institutions to build a common discourse and to ground our exchange about pedagogy and innovation in a nuanced examination of student learning that these rubrics afford. Not meant to be prescriptive, the VALUE rubrics are broad and flexible enough to encourage campuses as diverse in their educational missions as Prairie State College, LaGuardia Community College, Manhattanville College, Pace University, Boston University, Virginia Tech, and Stony Brook University to assess student learning on their campuses in a relational way. They also present a unified view of the cultural, social, and economic importance of key skills like written communication and information literacy while also creating space for emerging concepts like integrative learning. This work allows us to talk concretely rather than abstractly about student learning and our shared vision for higher education in the United States. It \"[G]ives us back the image of our mind,\" for what higher education has always aspired to: creating a community of educated individuals poised to meet tomorrow's challenges.